#36371: JSONField.from_db_value crashes when DB returns parsed JSON despite
KeyTransform guard
-------------------------------------+-------------------------------------
Reporter: Mason Pitts | Type: Bug
Status: new | Component: Database
| layer (models, ORM)
Version: 5.2 | Severity: Normal
Keywords: jsonfield, | Triage Stage:
from_db_value, double-decoding, | Unreviewed
psycopg3, cx_oracle, python- |
oracledb |
Has patch: 1 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 1 | UI/UX: 0
-------------------------------------+-------------------------------------
In Django 5.2, the default implementation of JSONField.from_db_value()
only skips double-decoding when the ORM expression is a KeyTransform on a
non-string. However, many modern database drivers (e.g. PostgreSQL
psycopg3, Oracle DB_TYPE_JSON via cx_Oracle 8.1+/python-oracledb) will
automatically deserialize JSON columns into native Python types (dict,
list) before Django sees them. Since from_db_value() still unconditionally
calls json.loads() in most cases, you get:
{{{
TypeError: the JSON object must be str, bytes or bytearray, not list
}}}
even though the value is already a valid Python object.
Here is the current code below as of 5/6/2025.
{{{
def from_db_value(self, value, expression, connection):
if value is None:
return value
# Some backends (SQLite at least) extract non-string values in
their
# SQL datatypes.
if isinstance(expression, KeyTransform) and not isinstance(value,
str):
return value
try:
return json.loads(value, cls=self.decoder)
except json.JSONDecodeError:
return value
}}}
Here is a potential solution that attempts to return value if it is a
Python type.
{{{
def from_db_value(self, value, expression, connection):
if value is None:
return None
# Decode binary data first.
if isinstance(value, (bytes, bytearray)):
value = value.decode()
# If value isn’t a string at this point, the driver already gave
us
# a native Python type (dict, list, bool, int, float, ...).
if not isinstance(value, str):
return value
try:
return json.loads(value, cls=self.decoder)
except json.JSONDecodeError:
return value
}}}
Steps to reproduce:
1. Define a model with a models.JSONField().
2. Use a database and driver combination that natively decodes JSON
columns—for example:
- PostgreSQL with psycopg3
- Oracle 21c+ JSON column type with cx_Oracle 8.1+ or python-
oracledb in thin mode
* I encountered this problem using oracle_db in thin mode.
3. Query the model in the Django admin or via MyModel.objects.all().
4. Observe the traceback raising a TypeError when json.loads() is fed a
list or dict.
Version information:
Django: 5.2
Python: 3.12.10
Affected drivers/backends:
PostgreSQL with psycopg3
Oracle 21c+ with cx_Oracle 8.1+ / python-oracledb in thin mode
--
Ticket URL: <https://code.djangoproject.com/ticket/36371>
Django <https://code.djangoproject.com/>
The Web framework for perfectionists with deadlines.
--
You received this message because you are subscribed to the Google Groups
"Django updates" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to django-updates+unsubscr...@googlegroups.com.
To view this discussion visit
https://groups.google.com/d/msgid/django-updates/01070196a603383b-ea113402-7a25-4144-aa67-93aec22987bc-000000%40eu-central-1.amazonses.com.
