#27982: Possible race condition related to queryset union
-------------------------------------+-------------------------------------
Reporter: gigelu | Owner: nobody
Type: Bug | Status: new
Component: Database layer | Version: 1.11
(models, ORM) |
Severity: Normal | Resolution:
Keywords: | Triage Stage:
| Unreviewed
Has patch: 0 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------
Comment (by gigelu):
More info on the bug.
When `paginator.count`
(https://github.com/django/django/blob/master/django/core/paginator.py#L69)
is accessed, it calls the queryset's count
(https://github.com/django/django/blob/master/django/db/models/query.py#L339).
At this moment the queryset is not evaluated yet, so it doesn't have a
`_result_cache`, resulting in a call to its own query's `get_count()`
(https://github.com/django/django/blob/master/django/db/models/sql/query.py#L479),
which calls `get_aggregation` where we find at line 466 `result =
compiler.execute_sql(SINGLE)` which always returns only one object from
DB.
Now another interesting part: the result from DB is fetched here:
https://github.com/django/django/blob/master/django/db/models/sql/compiler.py#L885,
and it looks like `<class 'tuple'>: (1, 'simple', 'info', False,
datetime.datetime(2017, 3, 23, 13, 8, 42, 794329))`. From this tuple the
count gets its value, which in reality is the id of the object fetched
from database.
I've added another simple notification to db and I could confirm the bug
by running with `all=True`, which earlier returned `3`, now it returns `4`
(from `<class 'tuple'>: (4, 'simple', 'primary', False,
datetime.datetime(2017, 3, 24, 12, 40, 20, 376582))`).
The query that runs is the same as earlier:
{{{
(SELECT "notification_simplenotification"."id",
"notification_simplenotification"."type",
"notification_simplenotification"."popup_type",
"notification_simplenotification"."completed",
"notification_simplenotification"."date_created" FROM
"notification_simplenotification" ORDER BY
"notification_simplenotification"."date_created" DESC) UNION (SELECT
"notification_decisionnotification"."id",
"notification_decisionnotification"."type",
"notification_decisionnotification"."popup_type",
"notification_decisionnotification"."completed",
"notification_decisionnotification"."date_created" FROM
"notification_decisionnotification" ORDER BY
"notification_decisionnotification"."date_created" DESC)
}}}
It doesn't ask for `count` from what I can see here.
As a quick fix for the Django only version is to evaluate the queryset
before passing it to the paginator.
But this doesn't work with DRF because that querysets gets through
filtering which returns another queryset which again doesn't have the
`_result_cache` populated.
--
Ticket URL: <https://code.djangoproject.com/ticket/27982#comment:10>
Django <https://code.djangoproject.com/>
The Web framework for perfectionists with deadlines.
--
You received this message because you are subscribed to the Google Groups
"Django updates" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to [email protected].
To post to this group, send email to [email protected].
To view this discussion on the web visit
https://groups.google.com/d/msgid/django-updates/064.b8513a4f53774dd437c4c69a86835a11%40djangoproject.com.
For more options, visit https://groups.google.com/d/optout.
