ccaominh opened a new pull request #10089: URL: https://github.com/apache/druid/pull/10089
Fixes #10057. ### Description Native batch range partitioning was only considering the partition dimension value when grouping rows instead of using all of the row's partition values. Thus, for schemas with multiple dimensions, the rollup was overestimated, which would cause too many dimension values to be packed into the same range partition. The resulting segments would then be overly large (and not honor the target or max partition sizes). <hr> This PR has: - [x] been self-reviewed. - [x] added Javadocs for most classes and all non-trivial methods. Linked related entities via Javadoc links. - [x] added comments explaining the "why" and the intent of the code wherever would not be obvious for an unfamiliar reader. - [x] added unit tests or modified existing tests to cover new code paths, ensuring the threshold for [code coverage](https://github.com/apache/druid/blob/master/dev/code-review/code-coverage.md) is met. <hr> ##### Key changed/added classes in this PR * `PartialDimensionDistributionTask`: Consider all dimension values when grouping row * `RangePartitionMultiPhaseParallelIndexingTest`: Regression test by having input with rows that should roll up and rows that should not roll up ---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
