JingsongLi opened a new pull request, #1234: URL: https://github.com/apache/paimon-rust/pull/1234
### Purpose Integration tests currently require Docker and Spark to regenerate the same Paimon warehouse before Rust, DataFusion, Python, and Go can read it. Replace that setup with a committed Java-written golden warehouse so these suites run directly from a checkout. ### Brief change log - Add a 116 KiB archive containing 34 tables and 658 files generated by Spark 3.5.3 / Paimon 1.3.1. Record the generator source, checksums, and coverage; retain historical schemas, snapshots, tags, manifests, indexes, deletion vectors, and data files. - Extract private temporary warehouses in all four suites, while retaining `PAIMON_TEST_WAREHOUSE` for explicit overrides. - Fail on missing fixtures instead of generating replacement tables with the Rust writer or skipping tests. - Enable the previously ignored Java/Rust dynamic-bucket HASH-index comparison. - Remove Spark provisioning, Docker Compose, the provisioning Makefile, and CI container setup/teardown. ### Tests - Rust integration suite: 80 passed. - DataFusion: 362 library tests, 12 dynamic-bucket tests, and 47 read tests passed. Two existing Lumina read tests initially failed because the native library is unavailable on this macOS ARM64 host; rerunning the read suite with those two tests excluded passed. One existing test remains ignored. - Python: all 47 tests in `tests/test_datafusion.py` passed using a binding built from this checkout. - Go: all 29 integration tests passed using a C binding built from this checkout. - Rustfmt, gofmt, and `cargo +1.98.0 clippy --locked -p paimon-integration-tests -p paimon-datafusion --all-targets -- -D warnings` passed. - Dependency reports and all 21 generated binary legal files verified. - All 658 archived files match the Java output byte for byte and contain no references to the original warehouse path. ### API and Format This changes test infrastructure only; public APIs and persisted table formats are unchanged. The golden archive fixes one producer version; future compatibility cases should add independently generated Java fixtures with explicit behavioral assertions. ### Documentation Update contributor guides, getting-started instructions, and Python/Go test documentation for automatic fixture loading. Document fixture provenance and the existing warehouse override in `testdata/warehouse/README.md`. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
