rtyler commented on issue #16622: URL: https://github.com/apache/datafusion/issues/16622#issuecomment-5777005227
> > In terms of more work for downstream crates, I am not sure it is any easier / harder on them as today they need to chase multiple DataFusion versions potentially anyways > > Also worth hearing out from some maintainers here (re: option 2, i.e. LTS releases/branches), [@JanKaul](https://github.com/JanKaul), [@rtyler](https://github.com/rtyler), [@ion-elgreco](https://github.com/ion-elgreco) and [@gabotechs](https://github.com/gabotechs) come to mind. _crawls out of the sewers_ I have been summoned! From the delta-io/delta-rs perspective Datafusion major releases are tedious but not painfully so, it is usually a case of "who moved my cheese" trying to figure out which APIs moved around and what they mean. ("This was a PlanConfiguration now it's an ConfiguredExecPlan? okie doke") The `arrow` and `object_store` breaking changes have been excruciating because we basically have to align the planets between delta-io/delta-kernel-rs, which must necessarily support multiple arrow versions through feature flags, all the way up the stack through datafusion, delta-rs, and then some of the extended ecosystem as well (e.g. datafusion-ffi / python). @comphead's point about the Spark release cycle is the _de facto_ world we have to live in because these planets align typically only once or twice a year for major release versions. The thing about API breaking changes is that for most users, myself included, the trtadeoff is better performance or capability, most of us will jump through as many hoops as you present in order to get those new capabilities 😄 I appreciate the ping on the thread and the consideration here 🫡 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
