TheR1sing3un commented on code in PR #12451:
URL: https://github.com/apache/hudi/pull/12451#discussion_r1896355013
##########
hudi-client/hudi-spark-client/src/main/java/org/apache/hudi/execution/bulkinsert/RDDConsistentBucketBulkInsertPartitioner.java:
##########
@@ -49,25 +48,48 @@
/**
* A partitioner for (consistent hashing) bucket index used in bulk_insert
*/
-public class RDDConsistentBucketBulkInsertPartitioner<T> extends
RDDBucketIndexPartitioner<T> implements
ConsistentHashingBucketInsertPartitioner {
+public class RDDConsistentBucketBulkInsertPartitioner<T> extends
RDDBucketIndexPartitioner<T> {
- private final Map<String, List<ConsistentHashingNode>> hashingChildrenNodes;
+ private Map<String/*partition*/, List<ConsistentHashingNode/*pending
resizing related child nodes*/>> hashingChildrenNodes;
+
+ // mark if this partitioner is used for writing to uncommitted buckets. Only
for case that clustering service executes bucket resizing.
+ private boolean isExecutingClustering = false;
Review Comment:
> The flag can be eliminated.
Removed~
##########
hudi-client/hudi-spark-client/src/main/java/org/apache/hudi/execution/bulkinsert/ConsistentBucketIndexBulkInsertPartitionerWithRows.java:
##########
@@ -54,14 +52,16 @@
/**
* Bulk_insert partitioner of Spark row using consistent hashing bucket index.
*/
-public class ConsistentBucketIndexBulkInsertPartitionerWithRows
- extends BucketSortBulkInsertPartitioner<Dataset<Row>> implements
ConsistentHashingBucketInsertPartitioner {
+public class ConsistentBucketIndexBulkInsertPartitionerWithRows extends
BucketSortBulkInsertPartitioner<Dataset<Row>> {
private final String indexKeyFields;
private final List<String> fileIdPfxList = new ArrayList<>();
- private final Map<String, List<ConsistentHashingNode>> hashingChildrenNodes;
+ private Map<String/*partition*/, List<ConsistentHashingNode>/*pending
resizing related child nodes*/> hashingChildrenNodes;
+
+ // mark if this partitioner is used for writing to uncommitted buckets. Only
for case that clustering service executes bucket resizing.
+ private boolean isExecutingClustering = false;
Review Comment:
> The flag can be eliminated.
Removed~
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
