Hi all, Ballista upgraded to DataFusion 55 five weeks ago and still can't release.
Our release includes the Python client, which can't ship until datafusion-python 55.0.0 is published, and that release vote hasn't started yet. This isn't a one-off. Over the last eleven DataFusion releases, datafusion-python has followed DataFusion by a median of 31 days, and by more than seven weeks at its longest. That's not a criticism of datafusion-python, which has good reasons for its own schedule, but it means Ballista releases will usually trail DataFusion by at least a month. Meanwhile, more than 300 commits on Ballista's main branch are waiting to ship, including fixes for wrong results from distributed NOT IN, executors being OOM-killed on heavy queries, and jobs hanging when all executors are lost. I've opened an issue proposing that we release the Python client separately, with its own vote once datafusion-python has caught up: https://github.com/apache/datafusion-ballista/issues/2511 It has the release data, a summary of our earlier discussion, and a few open questions, including whether to start the Ballista 55.0.0 release now. I'd especially like to hear from anyone using the Ballista Python client. It would be great to keep the discussion on the issue so it's all in one place, but replies here are welcome too. Thanks, Andy --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
