jihoonson commented on a change in pull request #8925: Parallel indexing single dim partitions URL: https://github.com/apache/incubator-druid/pull/8925#discussion_r349809274
########## File path: docs/ingestion/native-batch.md ########## @@ -241,18 +241,37 @@ Currently only one splitHintSpec, i.e., `segments`, is available. ### `partitionsSpec` -PartitionsSpec is to describe the secondary partitioning method. +PartitionsSpec is used to describe the secondary partitioning method. You should use different partitionsSpec depending on the [rollup mode](../ingestion/index.md#rollup) you want. -For perfect rollup, you should use `hashed`. +For perfect rollup, you should use either `hashed` (partitioning based on the hash of dimensions in each row) or +`single_dim` (based on ranges of a single dimension. For best-effort rollup, you should use `dynamic`. + +Hashed partitioning is recommended in most cases, as it will improve indexing performance and create more uniformly Review comment: I'm not sure how hashed partitioning can improve indexing performance or create more uniformly sized data segments relative to dynamic partitioning. With dynamic partitioning, the parallel indexing task will run in a single phase mode whereas hash-based partitioning requires to run in two phases mode. Also, the uniformity in segment size with hashed partitioning will depend on the partition key distribution whereas dynamic partitioning guarantees a max size for segments. Am I missing something? ---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected] With regards, Apache Git Services --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
