#23941: Aggregating over decimal field regression
-------------------------------------+-------------------------------------
Reporter: jarshwah | Owner: jarshwah
Type: Bug | Status: assigned
Component: Database layer | Version: master
(models, ORM) | Resolution:
Severity: Release blocker | Triage Stage: Accepted
Keywords: | Needs documentation: 0
Has patch: 1 | Patch needs improvement: 1
Needs tests: 0 | UI/UX: 0
Easy pickings: 0 |
-------------------------------------+-------------------------------------
Comment (by carljm):
It seems this issue is maybe being presented as broader than it really is?
We don't ever do "validation" (in the Django sense) on annotation outputs,
we just convert from db value to Python value. We have one special case
with `DecimalField` where that conversion itself can actually raise an
exception. The problem is not with validation, the problem is that
`max_digits` is not just a validation, it is a hard limit on the value
that the field knows how to represent (due to its use in setting the
decimal context precision). There is no parallel problem with e.g.
`max_length` on a `CharField` (or with any other field with validation
constraints), because the `max_length` (or any other validation
constraint) isn't checked anywhere in the conversion from DB value to
Python value. So in a way, this is simply a bug in `DecimalField` (or
perhaps in the way an internal-type of `DecimalField` is special-cased in
`ExpressionNode.convert_value`), in that its "validation" constraints are
applied (and result in hard errors) in data conversion, where normally
field validation does not apply at all.
(As an aside, it also seems a little odd to me that
`ExpressionNode.convert_value` assumes that any field with internal-type
of `DecimalField` will have a `format_number` method, when
`DecimalField.format_number()` has a docstring indicating that it only
exists for legacy code.)
In the case where no explicit `output_field` is given for an annotation,
we should definitely not apply the `max_digits` or `decimal_places` values
from the source field to the result; we should simply convert to Decimal
and return.
The trickier case is where an explicit `output_field` was given. I hear
Anssi's argument that we should respect the `max_digits` and
`decimal_places` in that case, otherwise we are "lying" to the user, but
I'm not convinced. Why should `DecimalField` be special? If you specify a
`CharField` as `output_field`, its `max_length` will not be respected in
converting from db to Python. If you specify a `PositiveIntegerField` as
`output_field` for an expression, you can get a negative result with no
error, etc. Conversion is not the same thing as validation.
It seems to me that the idea of applying field validation to the output of
an expression would be a major new feature addition that would need quite
a lot of separate discussion to figure out how it would work, or if its
even a good idea (there is currently no such thing as getting
`ValidationError` from attempting to do a database query in Django). In
the meantime, it's simply a bug (and a regression) that such validation is
sort-of applied in the case of `DecimalField`, and the correct fix is to
just stop doing that, as the current pull request does.
This does leave us in the odd situation where specifying `output_field`
sometimes requires that you provide field parameters which are effectively
ignored. I think the ideal API for this would be to allow (or require?)
`output_field` to be a field class, not a field instance. But that would
require making `get_internal_type` a classmethod rather than an instance
method on Fields. Which perhaps it should be, but that's also a major
invasive change for third-party field classes.
--
Ticket URL: <https://code.djangoproject.com/ticket/23941#comment:5>
Django <https://code.djangoproject.com/>
The Web framework for perfectionists with deadlines.
--
You received this message because you are subscribed to the Google Groups
"Django updates" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to [email protected].
To post to this group, send email to [email protected].
To view this discussion on the web visit
https://groups.google.com/d/msgid/django-updates/066.f804ccf7ef7528ef6aab4ffceed8ccb7%40djangoproject.com.
For more options, visit https://groups.google.com/d/optout.
