The GitHub Actions job "Tests (AMD)" on airflow.git/v3-3-test has failed. Run started by GitHub user potiuk (triggered by potiuk).
Head commit for run: ae7c36b5effd8cf2d1af7f2f987295a45a72f7a3 / Jarek Potiuk <[email protected]> [v3-3-test] Limit pandas to < 3 for DataFrame XComs (#70791) (#70912) * Limit pandas to < 3 for DataFrame XComs Under pandas 3 a DataFrame does not round trip through an XCom the way it does under pandas 2. The public classes moved to the `pandas` namespace, so the same object qualifies as `pandas.DataFrame` rather than `pandas.core.frame.DataFrame` and the serde registry -- keyed on the latter -- does not dispatch to the pandas serializer at all. The caller gets serde's generic "cannot serialize object of type" with nothing pointing at pandas. Beyond that, the dtypes differ: an `object` column reads back as `str` and missing values as `nan` rather than `None`, so the reader's pandas version, not the writer's, decides what a task receives. Declare `pandas<3` where Airflow declares pandas, and back it with a runtime check, since pandas is an optional dependency and the constraint alone cannot be relied on -- a deployment may install pandas 3 directly or pull it in through another package. The check needs the pandas 3 qualname registered to be reachable at all: without `pandas.DataFrame` in the registry the request never reaches this module. It is registered for that reason only, to refuse the value with a message that names pandas and says what to do, not to support it. Reads are refused as well as writes. A payload written under pandas 2 comes back with pandas 3 dtypes, so accepting it would hand the task different data than was pushed, with no signal. Moving from pandas 2 to pandas 3 is a deliberate migration for users to make. Support for it is not dropped, only deferred. Generated-by: Claude Opus 5 (1M context) following the guidelines at https://github.com/apache/airflow/blob/main/contributing-docs/05_pull_requests.rst#gen-ai-assisted-contributions * Sync uv.lock with the pandas < 3 constraint * Add significant newsfragment for the pandas < 3 limit * Apply the pandas < 3 limit to the remaining providers The XCom limit only binds where Airflow declares pandas. Thirteen providers pin pandas independently of common-sql, so a deployment installing any of them could still resolve pandas 3 and reach the runtime refusal rather than being held on a supported version by the constraint. pandas-gbq is deliberately untouched -- a different package, unaffected by this. Provider READMEs and uv.lock are regenerated to match. * Regenerate provider docs index for the pandas < 3 pins The update-providers-build-files hook renders the extras table in each provider's docs/index.rst from its pyproject dependencies. Fourteen providers show the pandas extra, so the pin change widens those tables. * Explain the pandas upper bound at every site and drop a vacuous test contributing-docs/13_airflow_dependencies_and_extras.rst asks for a comment saying why whenever a dependency is upper-bound, and AGENTS.md asks for the tracking URL in the file rather than only in a PR comment. All 45 specifiers across 15 files landed without either. Add the rationale and a pointer to the pandas 3 support PR above each block, so whoever decides when the cap comes off does not have to reconstruct it. test_pandas_2_dataframe_xcom_still_round_trips passed unchanged on main: nothing read pd.__version__ before this PR, so monkeypatching it to a 2.x value while the installed pandas is already 2.x changed nothing observable, and the round trip is already covered by test_pandas_serializers. Dropped. --------- (cherry picked from commit 415cb70f87784cc074d6db7844a07579587b5f0f) Co-authored-by: Rahul Vats <[email protected]> Report URL: https://github.com/apache/airflow/actions/runs/30708410052 With regards, GitHub Actions via GitBox --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
