LuciferYang opened a new pull request, #10293: URL: https://github.com/apache/paimon/pull/10293
### Purpose `local-sort.max-num-file-handles` (the external-merge fan-in, default 128) has no lower-bound validation, so it can be set to 1. With fan-in = 1, `AbstractBinaryExternalMerger.mergeChannelList` divides by `maxFanIn - 1 = 0` once a second spill file forms, producing `Integer.MAX_VALUE` merges and a negative `subList` argument, so the sort crashes with `IllegalArgumentException`. Because it only triggers on the second spill, a fan-in of 1 passes small-data testing and then crash-loops in production as data grows (the job restarts on the same config and fails again). This adds a `checkArgument(maxFanIn >= 2, ...)` in the `AbstractBinaryExternalMerger` constructor naming `local-sort.max-num-file-handles`, converting the latent, data-volume-dependent crash into a deterministic, actionable configuration error at construction time. The merger constructor is the single chokepoint through which every fan-in call path passes. A fan-in of 1 is degenerate for a merge sort (it can never converge N runs), so this rejects no legitimate configuration. This closes #10292. ### Tests - `BinaryExternalSortBufferTest#testFanInBelowTwoFailsFast` pins that constructing with a fan-in below two fails fast. Without the guard, construction succeeds and the crash only surfaces later on the second spill. ### API and Format No. ### Documentation No. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
