There is an epic PR/proposal [1] that I would like everyone to be aware of.
The PR has a lot of detail to explain the motivation, but the TL;DR is that this PR trims Ballista down to be a library that people can extend rather than an out-of-the-box application. Copied from the PR description: This PR removes: - python support (datafusion python is the main effort, we should focus making it run on ballista) - helm support (this can be contrib project) - good part of ci directory (ci works without it) - plugin sub-system (never finalized) - caching support (this can be a contrib project) - support for hdfs (library too old, there are newer library to be used) - UI (can be a contrib project) - made keda, rest-api, flight-sql optional (we could move them to contrib project) - key value store is gone, only memory store remains - removes some dependencies which were detected as not used Personally, I like this direction, although I have not yet reviewed the PR. Please leave comments on the PR if you have opinions. Thanks, Andy. [1] https://github.com/apache/datafusion-ballista/pull/1066
