#29229: QuerySet.values_list() combined with .extra() or .annotate() may produce
wrong .union()
-------------------------------------+-------------------------------------
Reporter: master | Owner: nobody
Type: Bug | Status: new
Component: Database layer | Version: 1.11
(models, ORM) |
Severity: Release blocker | Resolution:
Keywords: union values_list | Triage Stage: Ready for
| checkin
Has patch: 1 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------
Old description:
> Regression introduced in 2.0, still present in 2.0.3.
>
> Easy context to reproduce the problem: suppose a Message model, in a
> postman app. Only one ordinary field is enough for the demo.
>
> {{{#!python
> qs1 = Message.objects.extra(select={'count': 0}).values_list('id',
> 'count').order_by()
> print(qs1.query) # as expected:
> # SELECT (0) AS "count", "postman_message"."id"
> # FROM "postman_message"
>
> qs2 =
> Message.objects.values('somefield').annotate(count=models.Count('pk')).annotate(id=models.Max('pk')).values_list('id',
> 'count').order_by()
> print(qs2.query) # as expected:
> # SELECT COUNT("postman_message"."id") AS "count",
> MAX("postman_message"."id") AS "id"
> # FROM "postman_message" GROUP BY "postman_message"."somefield"
>
> print(qs1.union(qs2).query) # !! WRONG !! the qs2 part is truncated:
> # SELECT (0) AS "count", "postman_message"."id" FROM "postman_message"
> # UNION
> # SELECT MAX("postman_message"."id") AS "id" FROM "postman_message" GROUP
> BY "postman_message"."somefield"
> }}}
>
> Compared to version 1.11, it comes from this addition in
> db/models/sql/compiler.py/get_combinator_sql():
> {{{#!python
> if not compiler.query.values_select and self.query.values_select:
> compiler.query.set_values(self.query.values_select)
> }}}
> Here, self.query.values_select is ('id',).
> For qs2, values_select is indeed empty, as coded in
> db/models/sql/query.py/set_values(),
> because in this case the two values_list() arguments are managed in
> annotation_names[].
New description:
Regression introduced in 2.0, still present in 2.0.3 [Edit: and backported
in 1.11.8].
Easy context to reproduce the problem: suppose a Message model, in a
postman app. Only one ordinary field is enough for the demo.
{{{#!python
qs1 = Message.objects.extra(select={'count': 0}).values_list('id',
'count').order_by()
print(qs1.query) # as expected:
# SELECT (0) AS "count", "postman_message"."id"
# FROM "postman_message"
qs2 =
Message.objects.values('somefield').annotate(count=models.Count('pk')).annotate(id=models.Max('pk')).values_list('id',
'count').order_by()
print(qs2.query) # as expected:
# SELECT COUNT("postman_message"."id") AS "count",
MAX("postman_message"."id") AS "id"
# FROM "postman_message" GROUP BY "postman_message"."somefield"
print(qs1.union(qs2).query) # !! WRONG !! the qs2 part is truncated:
# SELECT (0) AS "count", "postman_message"."id" FROM "postman_message"
# UNION
# SELECT MAX("postman_message"."id") AS "id" FROM "postman_message" GROUP
BY "postman_message"."somefield"
}}}
Compared to version 1.11 [Edit: I run 1.11.7 precisely], it comes from
this addition in db/models/sql/compiler.py/get_combinator_sql():
{{{#!python
if not compiler.query.values_select and self.query.values_select:
compiler.query.set_values(self.query.values_select)
}}}
Here, self.query.values_select is ('id',).
For qs2, values_select is indeed empty, as coded in
db/models/sql/query.py/set_values(),
because in this case the two values_list() arguments are managed in
annotation_names[].
--
Comment (by master):
Precisions about versions.
--
Ticket URL: <https://code.djangoproject.com/ticket/29229#comment:3>
Django <https://code.djangoproject.com/>
The Web framework for perfectionists with deadlines.
--
You received this message because you are subscribed to the Google Groups
"Django updates" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to django-updates+unsubscr...@googlegroups.com.
To post to this group, send email to django-updates@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/django-updates/064.b779afb5add0ac0de72b34af8c0afa99%40djangoproject.com.
For more options, visit https://groups.google.com/d/optout.
