Apache9 commented on code in PR #5241:
URL: https://github.com/apache/hbase/pull/5241#discussion_r1202369028
##########
hbase-server/src/test/java/org/apache/hadoop/hbase/replication/regionserver/TestMetaRegionReplicaReplicationEndpoint.java:
##########
@@ -225,6 +227,38 @@ public void
testCatalogReplicaReplicationWithFlushAndCompaction() throws Excepti
}
}
+ @Test
+ public void testCatalogReplicaReplicationWALRolledAndDeleted() throws
Exception {
+ Connection connection =
ConnectionFactory.createConnection(HTU.getConfiguration());
+ TableName tableName = TableName.valueOf("hbase:meta");
+ Table table = connection.getTable(tableName);
+ try {
+ MiniHBaseCluster cluster = HTU.getHBaseCluster();
+ HRegionServer hrs =
cluster.getRegionServer(cluster.getServerHoldingMeta());
+ ReplicationSource source = (ReplicationSource)
hrs.getReplicationSourceService()
+ .getReplicationManager().catalogReplicationSource.get();
+ ((ReplicationPeerImpl) source.replicationPeer).setPeerState(false);
+ // load the data to the table
+ for (int i = 0; i < 5; i++) {
+ LOG.info("Writing data from " + i * 1000 + " to " + (i * 1000 + 1000));
+ HTU.loadNumericRows(table, HConstants.CATALOG_FAMILY, i * 1000, i *
1000 + 1000);
+ LOG.info("flushing table");
+ HTU.flush(tableName);
+ LOG.info("compacting table");
+ if (i < 4) {
+ HTU.compact(tableName, false);
+ }
+ }
+
HTU.getHBaseCluster().getMaster().getLogCleaner().triggerCleanerNow().get(1,
+ TimeUnit.SECONDS);
+ ((ReplicationPeerImpl) source.replicationPeer).setPeerState(true);
+ verifyReplication(tableName, numOfMetaReplica, 0, 5000,
HConstants.CATALOG_FAMILY);
+ } finally {
+ table.close();
Review Comment:
Use try with resources.
##########
hbase-server/src/test/java/org/apache/hadoop/hbase/replication/regionserver/TestMetaRegionReplicaReplicationEndpoint.java:
##########
@@ -225,6 +227,38 @@ public void
testCatalogReplicaReplicationWithFlushAndCompaction() throws Excepti
}
}
+ @Test
+ public void testCatalogReplicaReplicationWALRolledAndDeleted() throws
Exception {
+ Connection connection =
ConnectionFactory.createConnection(HTU.getConfiguration());
+ TableName tableName = TableName.valueOf("hbase:meta");
+ Table table = connection.getTable(tableName);
+ try {
+ MiniHBaseCluster cluster = HTU.getHBaseCluster();
+ HRegionServer hrs =
cluster.getRegionServer(cluster.getServerHoldingMeta());
+ ReplicationSource source = (ReplicationSource)
hrs.getReplicationSourceService()
+ .getReplicationManager().catalogReplicationSource.get();
+ ((ReplicationPeerImpl) source.replicationPeer).setPeerState(false);
Review Comment:
Is this enough to make sure that the replication is already stopped? The
check is in another thread, we'd better add some checks here to confirm that
the replication source is stopped.
##########
hbase-server/src/test/java/org/apache/hadoop/hbase/replication/regionserver/TestMetaRegionReplicaReplicationEndpoint.java:
##########
@@ -225,6 +227,38 @@ public void
testCatalogReplicaReplicationWithFlushAndCompaction() throws Excepti
}
}
+ @Test
+ public void testCatalogReplicaReplicationWALRolledAndDeleted() throws
Exception {
+ Connection connection =
ConnectionFactory.createConnection(HTU.getConfiguration());
+ TableName tableName = TableName.valueOf("hbase:meta");
+ Table table = connection.getTable(tableName);
+ try {
+ MiniHBaseCluster cluster = HTU.getHBaseCluster();
+ HRegionServer hrs =
cluster.getRegionServer(cluster.getServerHoldingMeta());
+ ReplicationSource source = (ReplicationSource)
hrs.getReplicationSourceService()
+ .getReplicationManager().catalogReplicationSource.get();
+ ((ReplicationPeerImpl) source.replicationPeer).setPeerState(false);
+ // load the data to the table
+ for (int i = 0; i < 5; i++) {
+ LOG.info("Writing data from " + i * 1000 + " to " + (i * 1000 + 1000));
+ HTU.loadNumericRows(table, HConstants.CATALOG_FAMILY, i * 1000, i *
1000 + 1000);
+ LOG.info("flushing table");
+ HTU.flush(tableName);
+ LOG.info("compacting table");
+ if (i < 4) {
+ HTU.compact(tableName, false);
+ }
+ }
+
HTU.getHBaseCluster().getMaster().getLogCleaner().triggerCleanerNow().get(1,
+ TimeUnit.SECONDS);
+ ((ReplicationPeerImpl) source.replicationPeer).setPeerState(true);
+ verifyReplication(tableName, numOfMetaReplica, 0, 5000,
HConstants.CATALOG_FAMILY);
Review Comment:
Here we just checked whether data can be read? I think the main thing here
is that we should make sure the ReplicationSource can still relicate things
out, i.e, it is not stuck forever.
So maybe we should load some data, flush, delete the WAL, enable the
replication source, and then load more data, without flushing, to make sure
that we could read all the data?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
