2010YOUY01 commented on PR #23414: URL: https://github.com/apache/datafusion/pull/23414#issuecomment-4942407209
Thank you for the detailed feedback, @alamb! I agree that the current solution adds friction for new developers. I think this approach could eventually become frictionless, but it needs more work. Let me run some experiments to see whether I can make even the initial version easier to use. Here are a few ideas: - Provide a one-click setup/bootstrap script that installs the required tools. - Skip checks whose tools are unavailable. With only the Rust toolchain installed, most critical checks should already be runnable. - Provide help messages showing the exact underlying commands, for example: `cargo xtask ci step fmt help` → `command: cargo fmt`. One minor clarification: `xtask` itself does not require installing any additional tools. It is more like a Cargo builtin feature (or convention): https://github.com/matklad/cargo-xtask ### Shell scripts vs. xtask commands To reach our end goal, the leaf nodes (atomic checks), should be wrapped as reusable units. The main design question is whether those units should be shell scripts or `xtask` steps. I still think `xtask` will be easier to use and maintain in the long term. The existing leaf scripts already contain additional logic rather than just invoking plain commands. Personally, I find Rust/`xtask` easier to work with: Rust is DataFusion's native language, and it provides more structure than shell scripts. Let me explore ways to reduce the setup friction and see how the approach looks afterward. ### Additional references Here are a few projects I would like to examine more closely. My impression is that most projects do not provide a great developer experience for running tests locally or reproducing CI. ChatGPT suggested these three as particularly strong examples, and I would like to study their approaches and consider how we could implement something similar: - ClickHouse: https://clickhouse.com/docs/development/continuous-integration - Materialize: https://github.com/MaterializeInc/materialize/blob/main/doc/developer/mzcompose.md - RisingWave: https://risingwave.com/blog/developer-experience-counts-behind-the-scenes-of-risedev/ -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
