#29323: HTTPRequest QueryDict wrongly decodes binary encoded values in python2
-------------------------------------------+------------------------
Reporter: Thomas Riccardi | Owner: nobody
Type: Uncategorized | Status: new
Component: HTTP handling | Version: 1.11
Severity: Normal | Keywords:
Triage Stage: Unreviewed | Has patch: 0
Needs documentation: 0 | Needs tests: 0
Patch needs improvement: 0 | Easy pickings: 0
UI/UX: 0 |
-------------------------------------------+------------------------
`application/x-www-form-urlencoded` bodies allow binary values: just
percent-escape (most of) the bytes.
It doesn't work with python2 and django.
In python2, QueryDict returns an `unicode` string for values, which makes
it impossible to represent binary data.
Worse: it returns an `unicode` string whose unicode code points are the
raw expected bytes (caused by lax behavior of `str.decode('iso-8859-1')`)
so it silently fails.
Example:
{{{#!python
q = QueryDict('foo=%00%7f%80%ff')
q['foo']
# actual:
# u'\x00\x7f\x80\xff'
# expected:
# '\x00\x7f\x80\xff'
}}}
Relevant code:
https://github.com/django/django/blob/f89b11b879b83aa505dc8231da5f06ca4b1b062e/django/http/request.py#L397-L401
This was introduced by
https://github.com/django/django/commit/fa02120d360387bebbbe735e86686bb4c7c43db2
while trying to accept broken user agents that send non-ascii as
urlencoded raw values:
- the python 3 version seems fine: it tries to decode '''before''' calling
the query string parser (from `urllib`)
- the python 2 version made a mistake: it decodes '''after''' calling the
query string parser (and only on the value, not the key strangely)
I'm not sure how to fix this, as there are many locations where the key
and value are re-encoded (`appendlist` does it too..)
I have not (yet) tested with python3.
--
Ticket URL: <https://code.djangoproject.com/ticket/29323>
Django <https://code.djangoproject.com/>
The Web framework for perfectionists with deadlines.
--
You received this message because you are subscribed to the Google Groups
"Django updates" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to [email protected].
To post to this group, send email to [email protected].
To view this discussion on the web visit
https://groups.google.com/d/msgid/django-updates/058.69043153da29f9a7b789884dd05fea9b%40djangoproject.com.
For more options, visit https://groups.google.com/d/optout.
