LuciferYang opened a new issue, #10292: URL: https://github.com/apache/paimon/issues/10292
### Search before asking - [X] I searched in the [issues](https://github.com/apache/paimon/issues) and found no similar issues. ### Paimon version master (1.5-SNAPSHOT) ### Compute Engine Any engine doing an external sort that spills (large sort / merge). ### Minimal reproduce step 1. Set `local-sort.max-num-file-handles = 1`. 2. Run a sort large enough to produce more than one spill file. ### What doesn't meet your expectations? The job crashes with an `IllegalArgumentException` (a negative `subList` argument), but only after the second spill file appears. `local-sort.max-num-file-handles` (the external-merge fan-in, default 128) has no lower-bound validation, so it can be set to 1. With fan-in = 1, `AbstractBinaryExternalMerger.mergeChannelList` divides by `maxFanIn - 1 = 0`, yielding `Integer.MAX_VALUE` merges and a negative `subList` bound. Because it only triggers on the second spill, it passes small-data testing and then crash-loops in production as data grows (the job restarts on the same config and fails again). It should fail fast at construction with a clear message. ### Anything else? A fan-in of 1 is degenerate for a merge sort (it can never converge N runs), so rejecting it up front is correct. ### Are you willing to submit a PR? - [X] I'm willing to submit a PR! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
