satishkotha commented on a change in pull request #1484: [HUDI-316] : Hbase qps
repartition writestatus
URL: https://github.com/apache/incubator-hudi/pull/1484#discussion_r407868223
##########
File path:
hudi-client/src/main/java/org/apache/hudi/index/hbase/HBaseIndex.java
##########
@@ -498,4 +554,37 @@ public boolean isImplicitWithStorage() {
public void setHbaseConnection(Connection hbaseConnection) {
HBaseIndex.hbaseConnection = hbaseConnection;
}
+
+ /**
+ * Partitions each WriteStatus with inserts into a unique single partition.
WriteStatus without inserts will be
+ * assigned to random partitions. This partitioner will be useful to utilize
max parallelism with spark operations
+ * that are based on inserts in each WriteStatus.
+ */
+ public class WriteStatusPartitioner extends Partitioner {
Review comment:
Could you please make this a static class if its not using any instance
variables of outer class
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
With regards,
Apache Git Services
