alamb commented on issue #6470: URL: https://github.com/apache/arrow-datafusion/issues/6470#issuecomment-1566124974
Thanks @sergiimk -- this is quite interesting I think one reason for the long cycle time is not just re-link but also that datafusion-core is large and any change to it requires recompiling the entire crate. Another approach I have seen taken in the rust community is to break such large crates into smaller ones. Not only does this tend to increase test cycle times it also encourages more encapsulation. We have been chipping away at doing this (e.g. https://github.com/apache/arrow-datafusion/issues/1754) but extracting `datafusion-physical-plan` and `datafusion-datasource` are especially connected. Maybe it is time to invest some more time trying to split out those crates 🤔 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
