kadubhumika commented on code in PR #74093:
URL: https://github.com/apache/airflow/pull/74093#discussion_r4178548305
##########
airflow-core/src/airflow/utils/sqlalchemy.py:
##########
@@ -291,6 +291,73 @@ def load_dialect_impl(self, dialect):
return super().load_dialect_impl(dialect)
+class DagRunConfJSON(TypeDecorator):
Review Comment:
Thanks @ashb, you're right. The failure isn't in storage: Postgres JSONB
keeps the
value as a huge numeric, which Python reads back as an int, and msgspec then
raises
OverflowError because it can't encode integers outside the 64-bit range.
I'll drop DagRunConfJSON and the DagRun model change, and instead convert
any int
outside the int64/uint64 range to a float right before the msgpack encoding.
I'll
keep it scoped to the conf payload, handle nested dicts/lists, and leave
normal ints
and bools untouched.
I'll also remove the unrelated zip-inflation commit from this branch and
rebase on main.
One question: some ints (e.g. 400+ digits) are too large to fit in a float
at all.
For those, would you prefer we raise a clear error, or convert to inf? I'd
lean
toward raising a clear error, since inf isn't valid JSON.
I'll push the revised approach shortly.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
