JingsongLi opened a new pull request, #542: URL: https://github.com/apache/paimon-rust/pull/542
## What changed - Add a non-published `benchmarks/tpcds` workspace crate with `load` and `run` commands. - Load external TPC-DS-generated Parquet data into append-only Paimon tables. - Run the same external query files against Paimon or native Parquet through DataFusion. - Record planning, execution, row-count, spill, and available physical metrics in a versioned JSON report. - Add configurable `SessionConfig` and `RuntimeEnv` constructors to `SQLContext` for benchmark memory, parallelism, and spill settings. - Document SF10/SF100/SF1000 operation, cache protocol, and result interpretation. ## Why The DataFusion integration has focused functional tests but no reproducible macro benchmark for analytical joins, aggregations, sorts, and spill behavior. This harness provides an end-to-end comparison using the same binary, runtime settings, and external query files for both sources. The repository does not include TPC tools, generated data, or query text. Reports are explicitly labeled as TPC-DS-derived non-TPC results. ## Validation - `cargo build -p paimon-tpcds-bench` - `cargo test -p paimon-tpcds-bench` — 7 unit tests and 4 smoke tests passed - `cargo test -p paimon-datafusion --test sql_context_tests` — 48 tests passed - `cargo clippy -p paimon-tpcds-bench -p paimon-datafusion --all-targets -- -D warnings` - `cargo fmt --all -- --check` - `git diff --check` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
