2005hithlj commented on code in PR #4808:
URL: https://github.com/apache/hbase/pull/4808#discussion_r1002517110
##########
hbase-replication/src/main/java/org/apache/hadoop/hbase/replication/ReplicationQueueStorage.java:
##########
@@ -184,4 +185,12 @@ void removeLastSequenceIds(String peerId, List<String>
encodedRegionNames)
* @return Whether the replication queue table exists
*/
boolean hasData() throws ReplicationException;
+
+ // the below 3 methods are used for migrating
+ void batchUpdate(ServerName serverName, List<ReplicationQueueData> datas)
+ throws ReplicationException;
+
+ void batchUpdate(List<ZkLastPushedSeqId> lastPushedSeqIds) throws
ReplicationException;
+
+ void batchUpdate(String peerId, List<String> hfileRefs) throws
ReplicationException;
Review Comment:
The names of these three methods can contain specific function?
##########
hbase-replication/src/main/java/org/apache/hadoop/hbase/replication/TableReplicationQueueStorage.java:
##########
@@ -541,4 +538,58 @@ public boolean hasData() throws ReplicationException {
throw new ReplicationException("failed to get replication queue table",
e);
}
}
+
+ @Override
+ public void batchUpdate(ServerName serverName, List<ReplicationQueueData>
datas)
+ throws ReplicationException {
+ List<Put> puts = new ArrayList<>();
+ for (ReplicationQueueData data : datas) {
+ if (data.getOffsets().isEmpty()) {
+ continue;
+ }
+ Put put = new Put(Bytes.toBytes(data.getId().toString()));
+ data.getOffsets().forEach((walGroup, offset) -> {
+ put.addColumn(QUEUE_FAMILY, Bytes.toBytes(walGroup),
Bytes.toBytes(offset.toString()));
+ });
+ puts.add(put);
Review Comment:
‘puts.add(put)’ should be moved after ‘put.addColumn’？
##########
hbase-replication/src/main/java/org/apache/hadoop/hbase/replication/TableReplicationQueueStorage.java:
##########
@@ -541,4 +538,58 @@ public boolean hasData() throws ReplicationException {
throw new ReplicationException("failed to get replication queue table",
e);
}
}
+
+ @Override
+ public void batchUpdate(ServerName serverName, List<ReplicationQueueData>
datas)
+ throws ReplicationException {
+ List<Put> puts = new ArrayList<>();
+ for (ReplicationQueueData data : datas) {
+ if (data.getOffsets().isEmpty()) {
+ continue;
+ }
+ Put put = new Put(Bytes.toBytes(data.getId().toString()));
+ data.getOffsets().forEach((walGroup, offset) -> {
+ put.addColumn(QUEUE_FAMILY, Bytes.toBytes(walGroup),
Bytes.toBytes(offset.toString()));
+ });
+ puts.add(put);
+ }
+ try (Table table = conn.getTable(tableName)) {
+ table.put(puts);
+ } catch (IOException e) {
+ throw new ReplicationException("failed to batch update queues", e);
+ }
+ }
+
+ @Override
+ public void batchUpdate(List<ZkLastPushedSeqId> lastPushedSeqIds) throws
ReplicationException {
+ Map<String, Put> peerId2Put = new HashMap<>();
+ for (ZkLastPushedSeqId lastPushedSeqId : lastPushedSeqIds) {
+ peerId2Put
+ .computeIfAbsent(lastPushedSeqId.getPeerId(), peerId -> new
Put(Bytes.toBytes(peerId)))
+ .addColumn(LAST_SEQUENCE_ID_FAMILY,
Bytes.toBytes(lastPushedSeqId.getEncodedRegionName()),
+ Bytes.toBytes(lastPushedSeqId.getLastPushedSeqId()));
+ }
+ try (Table table = conn.getTable(tableName)) {
+ table
+ .put(peerId2Put.values().stream().filter(p ->
!p.isEmpty()).collect(Collectors.toList()));
+ } catch (IOException e) {
+ throw new ReplicationException("failed to batch update last pushed
sequence ids", e);
+ }
+ }
+
+ @Override
+ public void batchUpdate(String peerId, List<String> hfileRefs) throws
ReplicationException {
+ if (hfileRefs.isEmpty()) {
+ return;
+ }
+ Put put = new Put(Bytes.toBytes(peerId));
+ for (String ref : hfileRefs) {
+ put.addColumn(HFILE_REF_FAMILY, Bytes.toBytes(ref),
HConstants.EMPTY_BYTE_ARRAY);
+ }
+ try (Table table = conn.getTable(tableName)) {
+ table.put(put);
Review Comment:
‘table.put(put)’ should be moved into the for loop?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
