ndimiduk commented on code in PR #6067:
URL: https://github.com/apache/hbase/pull/6067#discussion_r1686581455
##########
hbase-backup/src/main/java/org/apache/hadoop/hbase/backup/impl/BackupSystemTable.java:
##########
@@ -1902,4 +1905,19 @@ private static void ensureTableEnabled(Admin admin,
TableName tableName) throws
}
}
}
+
+ /**
+ * Executes the given operations in partitioned batches of size {@link
#BATCH_SIZE}
+ */
+ private static void executePartitionedBatches(Table table, List<? extends
Row> operations)
+ throws IOException {
+ List<? extends List<? extends Row>> operationBatches =
Lists.partition(operations, BATCH_SIZE);
+ for (List<? extends Row> batch : operationBatches) {
+ try {
+ table.batch(batch, new Object[batch.size()]);
Review Comment:
Look at the implementation of `HTable`. `Table#put(List)` does exactly this
already. If you want to change the edit-shipping semantics, this won't do it.
##########
hbase-backup/src/main/java/org/apache/hadoop/hbase/backup/impl/BackupSystemTable.java:
##########
@@ -101,6 +103,7 @@
public final class BackupSystemTable implements Closeable {
private static final Logger LOG =
LoggerFactory.getLogger(BackupSystemTable.class);
+ private static final int BATCH_SIZE = 1000;
Review Comment:
Are all these `Put`s/`Delete`s against the same rowkey? If yes, you're
implicitly getting an atomic row update via the current code path, I think.
If not, or you don't need row-level atomicity, you can replace this with a
loop over calls to `BufferedMutator#mutate()`, and the that class will batch
them up for RPC according to its configuration. Thus, it's still
user-configurable, but using nobs that already exist. Unless you think that
this particular operation needs some buffer settings that are specialized over
the defaults.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
