andygrove opened a new pull request, #1750: URL: https://github.com/apache/datafusion-ballista/pull/1750
# Which issue does this PR close? Closes #. # Rationale for this change The workspace crates were bumped to 53.0.0 in #1486 (Mar 2026), but \`python/Cargo.toml\` has stayed at 52.0.0. So the python wheels coming out of every recent RC's CI run (most recently \`53.0.0-rc1-pypitest-1\`) are actually labeled \`ballista-52.0.0-*.whl\` and embed 52.0.0 code pulled from crates.io — not the 53.0.0 code in this repo. Two compounding causes: 1. **The bump scripts didn't include \`python/Cargo.toml\`.** Every prior bump (#1394 51→52, #1363 50→51, etc.) left python behind. \`dev/update_ballista_versions.py\` and \`dev/update_datafusion_versions.py\` only iterate the workspace crates. 2. **\`python/Cargo.toml\` depended on ballista crates via crates.io versions, not path deps.** So even if the version field had been bumped, the wheel build would fail when ballista 53.0.0 isn't yet on crates.io — which it isn't until after the release vote. # What changes are included in this PR? \`python/Cargo.toml\` - \`[package].version\` 52.0.0 → 53.0.0. - Convert \`ballista\`, \`ballista-core\`, \`ballista-executor\`, \`ballista-scheduler\` dependencies to use \`path = "../ballista/<crate>"\` while keeping \`version = "53.0.0"\` for \`cargo publish\` metadata. Wheels now build against in-repo source, decoupling python from the crates.io publish cadence. - \`datafusion\`, \`datafusion-proto\`, \`datafusion-python\` 52 → 53 to match the workspace. - \`pyo3\`, \`pyo3-build-config\` 0.26 → 0.28. Required by \`datafusion-python\` 53 — cargo refused to resolve with a \`pyo3-ffi\` links-collision otherwise. \`python/pyproject.toml\` - \`datafusion==52\` → \`datafusion==53\`. \`python/Cargo.lock\`, \`python/uv.lock\` - Regenerated. \`dev/update_ballista_versions.py\` - Add \`python\` to the crate iteration list and to the path check that bumps \`[package].version\`. Future bumps now touch python automatically. \`dev/update_datafusion_versions.py\` - Add \`datafusion-python\` to the recognized crates so its dep version gets bumped in \`python/Cargo.toml\`. - Add \`python/Cargo.toml\` (as \`pyballista\`) to the downstream-version iteration. - Add \`update_pyproject_toml\` to bump the \`"datafusion==X"\` pin in \`python/pyproject.toml\`. - Skip upstream-datafusion paths that don't exist in this repo (the script's first loop was previously erroring out and the script could not be run end-to-end here). \`python/README.md\` - Replace the stale "versioned independently from the main Ballista project" line with an accurate description of lockstep-versioning and the path-dep build model. # Are there any user-facing changes? Python users will see \`ballista-53.0.0-*.whl\` going forward (instead of the unchanged 52.0.0 wheels that have shipped under three RC labels). No code API changes; just the version skew gets fixed. Rust workspace consumers are not affected (this only touches \`python/\`). Note: this PR depends on #1749 (lint debt + wheel gating) landing first for CI to be fully green — the ruff failures in `python/python/tests/test_context.py` and friends are pre-existing. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
