nsivabalan commented on code in PR #14026:
URL: https://github.com/apache/hudi/pull/14026#discussion_r2396108331
##########
hudi-client/hudi-spark-client/src/main/java/org/apache/hudi/client/SparkRDDTableServiceClient.java:
##########
@@ -101,14 +101,15 @@ protected HoodieWriteMetadata<HoodieData<WriteStatus>>
partialUpdateTableMetadat
String instantTime,
WriteOperationType writeOperationType) {
if (isStreamingWriteToMetadataEnabled(table)) {
- boolean enforceCoalesceWithRepartition = writeOperationType ==
WriteOperationType.CLUSTER && config.getBulkInsertSortMode() ==
BulkInsertSortMode.NONE;
+ boolean enforceCoalesceWithRepartition = writeOperationType ==
WriteOperationType.CLUSTER; // for other table services,
enforceCoalesceWithRepartition will be false.
if (enforceCoalesceWithRepartition) {
- // check clustering plan for sort columns. only if there are no sort
columns, then we might still set enforceCoalesceWithRepartition to true.
+ // check clustering plan for sort columns. only if there are no sort
columns, then fallback to sort mode from write config.
HoodieClusteringPlan clusteringPlan =
ClusteringUtils.getClusteringPlan(
table.getMetaClient(),
ClusteringUtils.getRequestedClusteringInstant(instantTime,
table.getActiveTimeline(), table.getInstantGenerator()).get())
.map(Pair::getRight).orElseThrow(() -> new
HoodieClusteringException(
"Unable to read clustering plan for instant: " + instantTime));
- enforceCoalesceWithRepartition =
!clusteringPlan.getStrategy().getStrategyParams().containsKey(PLAN_STRATEGY_SORT_COLUMNS.key());
+ enforceCoalesceWithRepartition =
(!clusteringPlan.getStrategy().getStrategyParams().containsKey(PLAN_STRATEGY_SORT_COLUMNS.key())
Review Comment:
sure. pushed a commit to address this
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
