kbuci commented on code in PR #18191:
URL: https://github.com/apache/hudi/pull/18191#discussion_r2875892970
##########
hudi-client/hudi-client-common/src/main/java/org/apache/hudi/table/action/cluster/strategy/PartitionAwareClusteringPlanStrategy.java:
##########
@@ -164,7 +165,14 @@ public Option<HoodieClusteringPlan>
generateClusteringPlan(ClusteringPlanActionE
return Option.empty();
}
- List<Pair<List<HoodieClusteringGroup>, String>> res =
getEngineContext().map(partitionPaths, partitionPath -> {
+ final HoodieEngineContext engineContext;
+ if (getWriteConfig().isClusteringPlanPartitionParallel()) {
+ engineContext = getEngineContext();
+ } else {
+ engineContext = new
HoodieLocalEngineContext(getEngineContext().getStorageConf());
Review Comment:
Oh I see, I'm assuming that for non-partitioned it currently just builds all
clustering groups for `.` empty partition? Should I create a followup issue or
attempt handling that in this PR? For context, I am anyway planning on creating
another PR to make `SparkStreamCopyClusteringPlanStrategy` get schema hashes
via spark engine context if
`hoodie.clustering.plan.generation.use.local.engine.context` is enabled
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
