#36484: Add optional error-on-integer-overflow setting for QuerySet filters
-------------------------------------+-------------------------------------
Reporter: Yosuke Takeuchi | Type: New
| feature
Status: new | Component: Database
| layer (models, ORM)
Version: 5.0 | Severity: Normal
Keywords: | Triage Stage:
| Unreviewed
Has patch: 0 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------
== Overview
In Django 5.0 the behaviour for arithmetic that overflows an integer
column changed: a filter that would overflow now always returns an empty
QuerySet instead of being transparently cast to a wider integer type (the
historical behaviour on most back-ends). While this keeps runtime
semantics simple, it makes migrations from Django 4.x harder to
verify—tests may “pass” yet silently return fewer (or zero) rows. It is
also difficult for developers on 5.x to notice when an overflow is
happening because the condition is swallowed inside the ORM.
(see:
https://docs.djangoproject.com/en/5.2/releases/5.0/#:~:text=Filtering%20querysets%20against%20overflowing%20integer%20values%20now%20always%20returns%20an%20empty%20queryset.%20As%20a%20consequence%2C%20you%20may%20need%20to%20use%20ExpressionWrapper()%20to%20explicitly%20wrap%20arithmetic%20against%20integer%20fields%20in%20such%20cases)
To improve debuggability and give teams a migration aid, I propose an opt-
in setting that raises a specific exception the moment an overflow is
detected.
== Proposal
=== Add new parameter.
{{{
# settings.py
ERROR_IF_INTEGER_OVERFLOW_IN_FILTER_QUERYSET = True # default: False
}}}
=== Implementation sketch
* Add new exception
{{{
class EmptyResultSetDueToIntegerOverflow(Exception):
"""Raised when an integer overflow is detected while building a WHERE
clause. Subclasses plain Exception so it is *not* caught by existing
`EmptyResultSet` handlers."""
}}}
* Raise it at the source
In django.db.models.lookups.IntegerFieldOverflow (or equivalent part of
the lookup machinery), raise EmptyResultSetDueToIntegerOverflow instead of
EmptyResultSet.
* Handle it centrally
In django.db.models.sql.where.WhereNode.as_sql() wrap the call to
compiler.compile(child)
{{{
try:
...
except EmptyResultSetDueToIntegerOverflow:
if settings.ERROR_IF_INTEGER_OVERFLOW_IN_FILTER_QUERYSET:
raise
empty_needed -= 1 # current silent-empty logic
}}}
* Default behaviour unchanged
The setting defaults to False, so existing applications keep the current
silent-empty semantics unless they explicitly enable strict mode.
=== Benefits
* Safe migrations – Projects upgrading from 4.x can enable the flag in CI
to surface every place where arithmetic now truncates results.
* Debuggability – Developers on 5.x who run into an unexpected empty
queryset get an immediate stack-trace pointing at the offending
expression.
* Backwards-compatible – Opt-in toggle preserves the 5.0 behaviour by
default; no existing code breaks unless the project explicitly asks for
stricter checking.
* Path for future tightening – If the community agrees, the flag could
default to True in a future major release after a full deprecation cycle.
I am happy to prepare a pull request with tests and documentation if this
approach is acceptable. Thank you for considering this enhancement.
--
Ticket URL: <https://code.djangoproject.com/ticket/36484>
Django <https://code.djangoproject.com/>
The Web framework for perfectionists with deadlines.
--
You received this message because you are subscribed to the Google Groups
"Django updates" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to django-updates+unsubscr...@googlegroups.com.
To view this discussion visit
https://groups.google.com/d/msgid/django-updates/01070197bb81e3a6-cae1c355-8016-4565-a105-ccae943841fe-000000%40eu-central-1.amazonses.com.
