#32554: Add Q.empty(), Q.TRUE, Q.FALSE, Q.any(), and Q.all()
-------------------------------------+-------------------------------------
Reporter: jonathan-golorry | Owner: jonathan-
| golorry
Type: New feature | Status: closed
Component: Database layer | Version: dev
(models, ORM) |
Severity: Normal | Resolution: wontfix
Keywords: Q objects, any, all | Triage Stage:
| Unreviewed
Has patch: 1 | Needs documentation: 1
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------
Comment (by jonathan-golorry):
Replying to [comment:4 Mariusz Felisiak]:
> Thanks for this ticket, however I have mixed-feelings. Django API is
already massive. IMO new hooks will make `Q` expressions more confusing
and are not necessary per se. `Q.any()` and `Q.all()` can be replaced with
`reduce()` (that's how we do this in Django itself, see
[https://github.com/django/django/blob/7c08f26bf0439c1ed593b51b51ad847f7e262bc1/django/contrib/admin/options.py#L1032
an example]). I don't see why the new API should be preferred over an
example with `reduce()` in
[https://docs.djangoproject.com/en/dev/topics/db/queries/#complex-lookups-
with-q-objects docs]. It has already been
[https://github.com/django/django/pull/13798#issuecomment-767492096
pointed out] that we need a good example of building `Q()` dynamically.
`reduce` isn't great. A common mistake I see in accepted answers on stack
overflow is that people forget to address empty iterators (potentially
leaking data due to empty `Q()` objects).
https://stackoverflow.com/questions/13076822/django-dynamically-filtering-
with-q-objects
https://stackoverflow.com/questions/852414/how-to-dynamically-compose-an-
or-query-filter-in-django
`reduce` turns that into a TypeError, which is better than leaking data,
but handling the case properly is still a pain. It's hard to explain why
some initializers work as defaults and others don't.
{{{
reduce(operator.and_, iterator, initializer=~Q(pk__in=[]) # "all",
defaulting to everything
reduce(operator.or_, iterator, initializer=Q(pk__in=[]) # "any",
defaulting to nothing
reduce(operator.and_, iterator, initializer=Q(pk__in=[]) # always
returns nothing
reduce(operator.or_, iterator, initializer=~Q(pk__in=[]) # always
returns everything
}}}
That's ignoring the absolute foot-gun that is `initializer=Q()`. You need
to add logically meaningless expressions to make the queries safe.
{{{
reduce(operator.and_, iterator, initializer=Q()) | Q(pk__in=[]) # "all",
defaulting to nothing
reduce(operator.or_, iterator, initializer=Q()) & ~Q(pk__in=[]) # "or",
defaulting to everything
}}}
Using manual checks to avoid needing initializer isn't great because of
edge cases around `bool(Q(Q()))` and `iterator=[Q()]`.
> About `Q.TRUE` and `Q.FALSE`, these constants are really tricky and
niche. I don't believe that folks will not how to use them properly (I
don't see immediate and wide use). Moreover, I'm not sure why Django
should recommend the `pk__in=[]` lookup (and not e.g. `pk=-1`), everything
depends on the context. It is so niche that folks should choose on their
own.
The main argument for having `Q.TRUE` and `Q.FALSE` is to make the above
code examples more readable. The logic of ANDing with TRUE is already
weird enough that it's easy to miss a `~`.
The query optimizer can handle `pk__in=[]`, not the others.
{{{
Article.objects.filter(Q(pk__in=[])) # doesn't hit DB
Article.objects.filter(Q(pk=None)) # hits DB and returns nothing
Article.objects.none() # doesn't hit DB
Article.objects.filter(Q()) # returns everything
}}}
> I strongly believe that we need a good example of building `Q()`
dynamically with the existing API, rather than adding yet another method
to do the same.
I think empty `Q()` objects are a significant security risk. The code
required to handle them properly is unintuitive and difficult to read. If
we just documented a good example, it would need to sufficiently explain
all these risks.
--
Ticket URL: <https://code.djangoproject.com/ticket/32554#comment:5>
Django <https://code.djangoproject.com/>
The Web framework for perfectionists with deadlines.
--
You received this message because you are subscribed to the Google Groups
"Django updates" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to [email protected].
To view this discussion on the web visit
https://groups.google.com/d/msgid/django-updates/074.a7b4ab419b4d8365f8182f66cfd1a8b7%40djangoproject.com.
