Hi,
I have a Django model working fine under Django 3.0 (i.e. "Django<3.1")
which looks like this:
===================
class Job(models.Model):
id = models.CharField(max_length=36, primary_key=True)
queue = models.CharField(max_length=40)
args = JSONField()
kwargs = JSONField()
type = models.CharField(max_length=80)
...
class Meta:
managed = False # <------ The table is implemented as a
Postgres FDW wrapper.
db_table = 'jobs'
===================
I am testing the update to Django 3.1.2 and hit an error in executing this
line:
jobs = list(models.Job.objects.filter(queue='celery',
state='scheduled'))
The error is as follows from pytest (i.e. stack trace with local variables
too):
==================
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
_ _
/usr/local/lib/python3.8/dist-packages/django/db/models/query.py:287: in
__iter__
self._fetch_all()
self = <QuerySet []>
/usr/local/lib/python3.8/dist-packages/django/db/models/query.py:1308: in
_fetch_all
self._result_cache = list(self._iterable_class(self))
self = <QuerySet []>
/usr/local/lib/python3.8/dist-packages/django/db/models/query.py:70: in
__iter__
for row in compiler.results_iter(results):
annotation_col_map = {}
compiler = <django.db.models.sql.compiler.SQLCompiler object at
0x7f8685e49160>
db = 'fdw'
init_list = ['id', 'queue', 'args', 'kwargs', 'type', 'state', ...]
klass_info = {'model': <class 'paiyroll.models.batch.Job'>,
'select_fields': [0, 1, 2, 3, 4, 5, ...]}
known_related_objects = []
model_cls = <class 'paiyroll.models.batch.Job'>
model_fields_end = 9
model_fields_start = 0
queryset = <QuerySet []>
related_populators = []
results = [[('8f4ab6b0-914f-4a75-972d-febbe55011fc', 'celery',
['paiyroll.tasks', 'call_to_string', 'call_to_string', [], {}], {},
'paiyroll.tasks.function_run', 'scheduled', ...)]]
select = [(Col(jobs, paiyroll.Job.id), ('"jobs"."id"', []),
None), (Col(jobs, paiyroll.Job.queue), ('"jobs"."queue"', []), None...,
paiyroll.Job.type), ('"jobs"."type"', []), None), (Col(jobs,
paiyroll.Job.state), ('"jobs"."state"', []), None), ...]
select_fields = [0, 1, 2, 3, 4, 5, ...]
self = <django.db.models.query.ModelIterable object at
0x7f86836f3040>
/usr/local/lib/python3.8/dist-packages/django/db/models/sql/compiler.py:1100:
in apply_converters
value = converter(value, expression, connection)
connection = <django.db.backends.postgresql.base.DatabaseWrapper
object at 0x7f869a321670>
converter = <bound method JSONField.from_db_value of
<django.contrib.postgres.fields.jsonb.JSONField: args>>
converters = [(2, ([<bound method JSONField.from_db_value of
<django.contrib.postgres.fields.jsonb.JSONField: args>>], Col(jobs,
pa...NField.from_db_value of
<django.contrib.postgres.fields.jsonb.JSONField: details>>], Col(jobs,
paiyroll.Job.details)))]
convs = [<bound method JSONField.from_db_value of
<django.contrib.postgres.fields.jsonb.JSONField: args>>]
expression = Col(jobs, paiyroll.Job.args)
pos = 2
row = ['8f4ab6b0-914f-4a75-972d-febbe55011fc', 'celery',
['paiyroll.tasks', 'call_to_string', 'call_to_string', [], {}], {},
'paiyroll.tasks.function_run', 'scheduled', ...]
rows = <itertools.chain object at 0x7f8683ae7520>
self = <django.db.models.sql.compiler.SQLCompiler object at
0x7f8685e49160>
value = ['paiyroll.tasks', 'call_to_string', 'call_to_string',
[], {}]
*/usr/local/lib/python3.8/dist-packages/django/db/models/fields/json.py:74:
in from_db_value return json.loads(value, cls=self.decoder)
connection = <django.db.backends.postgresql.base.DatabaseWrapper
object at 0x7f869a321670> expression = Col(jobs, paiyroll.Job.args)
self = <django.contrib.postgres.fields.jsonb.JSONField: args>
value = ['paiyroll.tasks', 'call_to_string', 'call_to_string',
[], {}] *
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
_ _
s = ['paiyroll.tasks', 'call_to_string', 'call_to_string', [], {}], cls =
None
object_hook = None, parse_float = None, parse_int = None, parse_constant =
None
object_pairs_hook = None, kw = {}
def loads(s, *, cls=None, object_hook=None, parse_float=None,
parse_int=None, parse_constant=None, object_pairs_hook=None,
**kw):
"""Deserialize ``s`` (a ``str``, ``bytes`` or ``bytearray`` instance
containing a JSON document) to a Python object.
``object_hook`` is an optional function that will be called with the
result of any object literal decode (a ``dict``). The return value
of
``object_hook`` will be used instead of the ``dict``. This feature
can be used to implement custom decoders (e.g. JSON-RPC class
hinting).
``object_pairs_hook`` is an optional function that will be called
with the
result of any object literal decoded with an ordered list of pairs.
The
return value of ``object_pairs_hook`` will be used instead of the
``dict``.
This feature can be used to implement custom decoders. If
``object_hook``
is also defined, the ``object_pairs_hook`` takes priority.
``parse_float``, if specified, will be called with the string
of every JSON float to be decoded. By default this is equivalent to
float(num_str). This can be used to use another datatype or parser
for JSON floats (e.g. decimal.Decimal).
``parse_int``, if specified, will be called with the string
of every JSON int to be decoded. By default this is equivalent to
int(num_str). This can be used to use another datatype or parser
for JSON integers (e.g. float).
``parse_constant``, if specified, will be called with one of the
following strings: -Infinity, Infinity, NaN.
This can be used to raise an exception if invalid JSON numbers
are encountered.
To use a custom ``JSONDecoder`` subclass, specify it with the
``cls``
kwarg; otherwise ``JSONDecoder`` is used.
The ``encoding`` argument is ignored and deprecated since Python
3.1.
"""
if isinstance(s, str):
if s.startswith('\ufeff'):
raise JSONDecodeError("Unexpected UTF-8 BOM (decode using
utf-8-sig)",
s, 0)
else:
if not isinstance(s, (bytes, bytearray)):
> raise TypeError(f'the JSON object must be str, bytes or
bytearray, '
f'not {s.__class__.__name__}')
E TypeError: the JSON object must be str, bytes or bytearray,
not list
cls = None
kw = {}
object_hook = None
object_pairs_hook = None
parse_constant = None
parse_float = None
parse_int = None
s = ['paiyroll.tasks', 'call_to_string', 'call_to_string', [], {}]
===============================
As you can perhaps see from the bolded part (for
/usr/local/lib/python3.8/dist-packages/django/db/models/fields/json.py:74),
the value being written into the JSONField called "args" is a Python list,
(shown as "s" on the last line of the traceback-with-values). I am aware of
documented changes around serializers but did not think that affected me;
however, given that I am using an FDW to return the data, is it that I
should now be serialising returned data into a string?
Any pointers appreciated.
Thanks, Shaheed
--
You received this message because you are subscribed to the Google Groups
"Django users" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to [email protected].
To view this discussion on the web visit
https://groups.google.com/d/msgid/django-users/CAHAc2jcTVRUg1Q9vVdNL6xpnZpBMCPZu8BoQKcacfEwYDFN6BA%40mail.gmail.com.
