2010YOUY01 opened a new pull request, #27606: URL: https://github.com/apache/datafusion/pull/27606
## Which issue does this PR close? Part of #27082. Stack 3/3: memory accounting → timestamp arithmetic → stream merge. These three refactors are independent. They are stacked only to avoid merge conflicts in the shared fuzz registration and dependency files. ## Rationale for this change Merging sorted streams should produce the same values as sorting their combined input, regardless of input batch boundaries or empty batches. For example, merging two copies of `0..100` should return each value twice, in order. The shared fuzz runner varies the batches and reports seeds for replay. ## What changes are included in this PR? Consolidate the four existing input layouts into one `MergeFuzzer`. Each dataset randomly selects a layout and splits its inputs into batches; cases assert the merged values and require every nonfinal output batch to have the requested size. Reading order: 1. `datafusion/fuzz/tests/suites/merge.rs::MERGE_LAYOUTS`: the four input shapes. 2. `MergeFuzzer`: dataset generation and merge assertions. 3. `datafusion/fuzz/tests/fuzz.rs::merge`: four-second campaign with up to seven batch sizes per dataset. Layout coverage is sampled across datasets; a single timed run does not guarantee every layout. ## What is the testing strategy for this PR? After rebasing the three-commit stack onto main (`a56c519bc`), all three migrated suites passed together: ```bash CARGO_CACHE_RUSTC_INFO=0 cargo test --locked --profile ci -p datafusion-fuzz --features extended_tests --test fuzz -- --skip spill_pool_scenario --skip key_range_bitmap_union --skip lru_queue --skip sort_topk --nocapture --test-threads=1 ``` ```text Fuzz suite merge: 20818 cases, 2975 datasets, 4.000042s Fuzz suite record_batch_memory_tracker: 2535 cases, 26 datasets, 1.000325083s Fuzz suite timestamp_arithmetic: 30459 cases, 31 datasets, 1.000006625s test result: ok. 3 passed; 0 failed; 0 ignored; 0 measured; 4 filtered out; finished in 6.00s ``` `cargo fmt --all -- --check` and `git diff --check upstream/main..HEAD` also passed. ## Are there any user-facing changes? No. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
