alamb commented on issue #14777: URL: https://github.com/apache/datafusion/issues/14777#issuecomment-2682572875
> I would like to come to a consensus as to what we would accept in a more specific worded manner, but perhaps more importantly, can we provide a home for non-core functions where the community could maintain them outside of DataFusion core? > It has some downsides too (being non-Apache limits contribution from corporations; has unpredictable release cycles) I agree this is a major downside of datafusion-extra, despire @demetribu 's great work there. > I would prefer separate crates within this repository for the top-popular function collections. No more than 6. I think this would be a reasonable approach in my mind. I am not sure that I would pick exactly 6 but I would definitely pick some number on that order of magnitude I think the key requirements in my mind are: 1. Existing users of datafusion dont' get more functions unless they opt in (e.g. via feature flag) 2. Existing development flows (`cargo test --test sqllogictests` for example) aren't impacted 3. It is easy to test / enable / disable the relevant function packages. So in other words, as long as the other functions live in a crate that isn't required to run most of the DataFusion tests it would be good in my opinion > Is the release burden proportional to amount of code being shipped? Is it proportional to number of releases being made? Can the burden be minimized into pure automation? It's 21st century... I am less worried about the mechanics of release which new crates will not affect, but more like "how much effort it is to keep tests going, debug the CI runners when they run out of disk space, or take too long to run the tests" -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: github-unsubscr...@datafusion.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org --------------------------------------------------------------------- To unsubscribe, e-mail: github-unsubscr...@datafusion.apache.org For additional commands, e-mail: github-h...@datafusion.apache.org
