#18004: Django should not use `force_unicode(..., errors='replace')` when
parsing
POST data.
-------------------------------------+-------------------------------------
Reporter: mrmachine | Owner: aaugustin
Type: Bug | Status: closed
Component: HTTP handling | Version: master
Severity: Normal | Resolution: needsinfo
Keywords: post data unicode | Triage Stage:
utf8 encode decode transaction | Unreviewed
aborted | Needs documentation: 0
Has patch: 1 | Patch needs improvement: 0
Needs tests: 0 | UI/UX: 0
Easy pickings: 0 |
-------------------------------------+-------------------------------------
Changes (by kmtracey):
* status: assigned => closed
* resolution: => needsinfo
Comment:
Indeed, we need some way to recreate and understand this issue before
attempting any fix. I'm not at all sure, based on this description, that
errors=replace is causing this. The "invalid byte sequence" error message
noted in the description identifies the invalid byte sequence as 0xea2020.
That is nothing at all like the expected utf-8 encoding of the unicode
replacement character (which would be 0xefbfbd). The whole point of
errors=replace is to take a bytestring we cannot figure out how to
correctly decode and turn it into a valid unicode string, possibly
containing the unicode replacement character if in fact the input
bytestring is "broken". The unicode replacement character itself is a
perfectly valid unicode character with a utf-8 encoding: for it to trigger
a database error would indicate a bug in the database.
The question is how the byte string 0xea2020 got stuffed onto the postgres
connection -- this, as logged, is not a valid utf-8 byte sequence and
therefore should never have been sent on the DB connection from Django.
But without some way for us to trigger that happening I don't see how we
can identify how it happened. Therefore closing this needsinfo since I
don't see we can do anything to address this problem without a recreation
scenario.
--
Ticket URL: <https://code.djangoproject.com/ticket/18004#comment:16>
Django <https://code.djangoproject.com/>
The Web framework for perfectionists with deadlines.
--
You received this message because you are subscribed to the Google Groups
"Django updates" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to [email protected].
To post to this group, send email to [email protected].
For more options, visit https://groups.google.com/groups/opt_out.
