Re: #22268 - values_list() on a ManyToManyField returns

[Russell Keith-Magee]

On Fri, Mar 28, 2014 at 2:29 PM, anubhav joshi wrote:

>
> values() and values_list() are both intended as optimisations for a
>> specific use case - retrieval of subsets of data without the overhead of
>> creating a model instance. This metaphor completely falls apart when
>> dealing with m2m relations, because the the "one row, one object" metaphor
>> that underpins most of the ORM falls apart.
>>
>> I agree with you on this. "one-row,one-object" cannot be implemented here.
>

Well, it *can* be implemented - just not in a way that is both
computationally efficient *and* backwards compatible.

Yours,
Russ Magee %-)

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at http://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/CAJxq84_WmUA0G%2BsZb_tWAzr7MFYPLo50EC1dDvGQ-jnR3iywbw%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

Re: #22268 - values_list() on a ManyToManyField returns

[anubhav joshi]

> values() and values_list() are both intended as optimisations for a 
> specific use case - retrieval of subsets of data without the overhead of 
> creating a model instance. This metaphor completely falls apart when 
> dealing with m2m relations, because the the "one row, one object" metaphor 
> that underpins most of the ORM falls apart. 
>
> I agree with you on this. "one-row,one-object" cannot be implemented here.
 

> While we could search for a deeper meaning for queries of this type, and 
> restructure the internal SQL to account for this, there's another issue to 
> consider. We're also talking about a part of the query API that has existed 
> since at least the query set refactor, and possibly as far back as magic 
> removal. That means there's between 5 and 7 years of legacy here, in which 
> time nobody else has raised a ticket about this issue. It's also reasonable 
> to assume that there will be a non-trivial number of people depending on 
> the API as currently implemented. 
>
> So, I'm inclined to say this is a known wart in the ORM, born of the leaky 
> abstraction of pushing relational data into an object-based representation. 
> Document the limitation, and move on.
>  
>

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at http://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/da7a89f3-25c8-4503-a443-e77c6b609d09%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: #22268 - values_list() on a ManyToManyField returns

[Russell Keith-Magee]

You're missing an important third option here: do nothing, and document the
fact that "values" over an m2m is an inherently nonsense operation in an
ORM context.

values() and values_list() are both intended as optimisations for a
specific use case - retrieval of subsets of data without the overhead of
creating a model instance. This metaphor completely falls apart when
dealing with m2m relations, because the the "one row, one object" metaphor
that underpins most of the ORM falls apart.

While we could search for a deeper meaning for queries of this type, and
restructure the internal SQL to account for this, there's another issue to
consider. We're also talking about a part of the query API that has existed
since at least the query set refactor, and possibly as far back as magic
removal. That means there's between 5 and 7 years of legacy here, in which
time nobody else has raised a ticket about this issue. It's also reasonable
to assume that there will be a non-trivial number of people depending on
the API as currently implemented.

So, I'm inclined to say this is a known wart in the ORM, born of the leaky
abstraction of pushing relational data into an object-based representation.
Document the limitation, and move on.

Yours,
Russ Magee %-)



On Thu, Mar 27, 2014 at 4:32 PM, anubhav joshi wrote:

> There has been quite a discussion on the ticket regarding this
> Refs. : https://code.djangoproject.com/ticket/22268
> I think we must discuss and decide what should be the solution we should
> be working towards.
> Until now there have been two strategies:
>
> 1.) The query generated is correct therefore,
> We just add pk to the query and then when final result is obtained we can
> club the dicts/tuples having same pk and remove pk.
> And also remove the ones where the pk is None (due to the LEFT OUTER JOIN
> ).
>
> 2.) Instead of retrieving the M2M or reverse foreign key values in the
> same SQL query, use a clever combination of:
>
>- prefetch_related()
>- subclassing dict and tuple
>
> So values() and values_list return these subclassed dicts and tuples,
> which lazily construct and return the relevant M2M or reverse foreign key
> elements when they are accessed.In the end, the subclassed dict and 
> tuplewould work somewhat like the Django model, except that you access things 
> in
> the form of dictionaries and tuples, and you limit the elements that can
> appear in them.
>
> --
> You received this message because you are subscribed to the Google Groups
> "Django developers" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to django-developers+unsubscr...@googlegroups.com.
> To post to this group, send email to django-developers@googlegroups.com.
> Visit this group at http://groups.google.com/group/django-developers.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/django-developers/4c1b12a4-822a-4582-8d57-3a8f04f62a58%40googlegroups.com
> .
> For more options, visit https://groups.google.com/d/optout.
>

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at http://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/CAJxq848PgOEq4dzzKWUpHDX9qcpPwK65sChGNFFUryVdFdw6SA%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

#22268 - values_list() on a ManyToManyField returns

[anubhav joshi]

There has been quite a discussion on the ticket regarding this
Refs. : https://code.djangoproject.com/ticket/22268
I think we must discuss and decide what should be the solution we should be 
working towards.
Until now there have been two strategies:

1.) The query generated is correct therefore,
We just add pk to the query and then when final result is obtained we can 
club the dicts/tuples having same pk and remove pk.
And also remove the ones where the pk is None (due to the LEFT OUTER JOIN).

2.) Instead of retrieving the M2M or reverse foreign key values in the same 
SQL query, use a clever combination of:

   - prefetch_related() 
   - subclassing dict and tuple 

So values() and values_list return these subclassed dicts and tuples, which 
lazily construct and return the relevant M2M or reverse foreign key 
elements when they are accessed.In the end, the subclassed dict and tuplewould 
work somewhat like the Django model, except that you access things in 
the form of dictionaries and tuples, and you limit the elements that can 
appear in them.

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at http://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/4c1b12a4-822a-4582-8d57-3a8f04f62a58%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.