alamb opened a new pull request, #6682: URL: https://github.com/apache/arrow-datafusion/pull/6682
# Which issue does this PR close? Part of https://github.com/apache/arrow-datafusion/issues/6676 # Rationale for this change We are running into builder space problems so reducing the space needed to run `cargo test` is required. Also it would be nice to get faster CI runs. Locally for me on my mac, it seems like each test binary takes a few seconds to compile and 100+MB of disk space. Thus combining them into fewer larger tests (so we have fewer binaries) will be an improvement # What changes are included in this PR? Combine `dataframe` and `dataframe_function` tests: # Are these changes tested? Yes by existing CI # Are there any user-facing changes? No -- this is development process # Measurements ## dataframe_functions (126MB) ```shell cd /Users/alamb/Software/arrow-datafusion2/ && RUST_BACKTRACE=1 CARGO_TARGET_DIR=/Users/alamb/Software/target-df2 cargo test --test dataframe_functions Finished test [unoptimized + debuginfo] target(s) in 0.28s Running tests/dataframe_functions.rs (/Users/alamb/Software/target-df2/debug/deps/dataframe_functions-6259330cccb05fd4) ... (arrow_dev) alamb@MacBook-Pro-8:~/Software/arrow-datafusion2$ du -s -h /Users/alamb/Software/target-df2/debug/deps/dataframe_functions-6259330cccb05fd4 126M /Users/alamb/Software/target-df2/debug/deps/dataframe_functions-6259330cccb05fd4 ``` ## dataframe (pre-PR) (133MB) ```rust cd /Users/alamb/Software/arrow-datafusion2/ && RUST_BACKTRACE=1 CARGO_TARGET_DIR=/Users/alamb/Software/target-df2 cargo test --test dataframe Compiling datafusion v26.0.0 (/Users/alamb/Software/arrow-datafusion2/datafusion/core) Finished test [unoptimized + debuginfo] target(s) in 4.65s Running tests/dataframe.rs (/Users/alamb/Software/target-df2/debug/deps/dataframe-bb6b871995f5ee03) (arrow_dev) alamb@MacBook-Pro-8:~/Software/arrow-datafusion2$ du -s -h /Users/alamb/Software/target-df2/debug/deps/dataframe-bb6b871995f5ee03 133M /Users/alamb/Software/target-df2/debug/deps/dataframe-bb6b871995f5ee03 ``` ## dataframe (with this PR) (134MB) By combining them, the result is 134 MB, A savings of 125MB. ```rust cd /Users/alamb/Software/arrow-datafusion2/ && RUST_BACKTRACE=1 CARGO_TARGET_DIR=/Users/alamb/Software/target-df2 cargo test --test dataframe Compiling datafusion v26.0.0 (/Users/alamb/Software/arrow-datafusion2/datafusion/core) Finished test [unoptimized + debuginfo] target(s) in 4.98s Running tests/dataframe.rs (/Users/alamb/Software/target-df2/debug/deps/dataframe-bb6b871995f5ee03) ... du -s -h /Users/alamb/Software/target-df2/debug/deps/dataframe-bb6b871995f5ee03 134M /Users/alamb/Software/target-df2/debug/deps/dataframe-bb6b871995f5ee03 ``` Admittedly these measurements are on my mac, but I think they will hold generally -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
