dhkim1920 opened a new pull request, #73783: URL: https://github.com/apache/airflow/pull/73783
Python tasks fail before their task body runs when Dag run conf contains an integer such as `18446744073709551616`. The Execution API preserves the value, but encoding the supervisor's startup message raises `OverflowError` because it exceeds MessagePack's native integer range. Preserve these values across Python subprocess communication using a MessagePack extension with a signed binary payload. Both directions restore the original Python `int`, including nested values. Conversion runs only after an encoding overflow, preserving the ordinary encoding path. External language-SDK subprocesses retain their existing behavior, whether or not they specify a schema version: unsupported integers raise before any socket write. This change does not add large-integer support to external runtimes or dictionary keys. Validation in Breeze with Python 3.10 and SQLite: - Reproduced positive and negative overflow before the fix in real Python task startup and DB/Execution API tests. - Communication, supervisor, subprocess coordinator, and schema integration tests: 425 passed, 1 platform-specific skip. - DB/Execution API regression tests: 5 passed. - External-runtime guard tests cover specified and unspecified schema versions and verify no socket write; they use the actual sender class with mock sockets. - Ruff, Task SDK and core mypy, and applicable static checks passed. No live Celery/Kubernetes deployment was tested. Broader validation: the API, Core, CLI, Other, and Serialization groups passed. Always also passed (2,142 passed, 3 skipped, 1 xfailed) after rebuilding the local image to resolve a missing Beam provider. The full Task SDK run with four workers and `--dist loadfile` reported 3,225 passed, 7 skipped, and two `TestContextDetection` failures; the same two failures reproduce on unmodified upstream `1d916545b2` (3,204 passed, 7 skipped). The affected secrets and object-storage test files pass in isolation (67 passed). An initial `loadgroup` run also exposed an object-store cache ordering failure. The full Task SDK suite is not claimed as fully green. closes: #73712 --- ##### Was generative AI tooling used to co-author this PR? - [X] Yes — Codex (GPT-6) Generated-by: Codex (GPT-6) following [the guidelines](https://github.com/apache/airflow/blob/main/contributing-docs/05_pull_requests.rst#gen-ai-assisted-contributions) --- Drafted-by: Codex (GPT-6); reviewed by @dhkim1920 before posting -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
