I would like to add the following item to the agenda call for the next Rust
sync call:

Dependencies

Background: As the dependency stack gets larger, it will be harder to use
DataFusion as an embedded query engine and the compile / dev times will get
higher.

As we expand the supported functions of DataFusion this problem is likely
to get worse. For example
https://github.com/apache/arrow/pull/9243#discussion_r575716759 and
https://github.com/apache/arrow/pull/9139

Proposal: Add Rust "features" to the datafusion crate and make many of the
new dependencies optional (so that we had features like regex and unicode
and hash which would only pull in the dependencies / have those functions
if the features were enabled.) This approach has worked well for Arrow
(which has only chrono and num as required dependencies)

Reply via email to