kbuci commented on code in PR #18191:
URL: https://github.com/apache/hudi/pull/18191#discussion_r2818564666
##########
hudi-client/hudi-client-common/src/main/java/org/apache/hudi/config/HoodieClusteringConfig.java:
##########
@@ -347,6 +347,18 @@ public class HoodieClusteringConfig extends HoodieConfig {
+ "Please exercise caution while setting this config, especially
when clustering is done very frequently. This could lead to race condition in "
+ "rare scenarios, for example, when the clustering completes after
instants are fetched but before rollback completed.");
+ public static final ConfigProperty<Boolean>
COMPUTE_PLAN_PER_PARTITION_PARALLEL = ConfigProperty
+ .key("hoodie.clustering.plan.compute.partition.parallel")
+ .defaultValue(true)
+ .sinceVersion("1.1.0")
+ .withDocumentation("Compute clustering groups for each partition
independently in parallel (using engine context) when generating "
+ + "clustering plan. For example, with Spark engine setting this to
true would lead to a separate spark task computing all clustering "
Review Comment:
good point, let me use the word `locally` since that's more accurate
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
