smengcl commented on a change in pull request #2998:
URL: https://github.com/apache/hadoop/pull/2998#discussion_r647197192
##########
File path:
hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/BPServiceActor.java
##########
@@ -1104,6 +1122,34 @@ private void sendLifeline() throws IOException {
}
}
+ class IBRTaskHandler implements Runnable {
+
+ @Override
+ public void run() {
+ LOG.info("Starting IBR Task Handler.");
+ while (shouldRun()) {
+ try {
+ final long startTime = scheduler.monotonicNow();
+ final boolean sendHeartbeat = scheduler.isHeartbeatDue(startTime);
+ if (!dn.areIBRDisabledForTests() &&
+ (ibrManager.sendImmediately() || sendHeartbeat)) {
+ synchronized (sendIBRLock) {
+ ibrManager.sendIBRs(bpNamenode, bpRegistration,
+ bpos.getBlockPoolId(), getRpcMetricSuffix());
+ }
+ }
+ // There is no work to do; sleep until hearbeat timer elapses,
Review comment:
```suggestion
// There is no work to do; sleep until heartbeat timer elapses,
```
Same typo at line 751 where this line is copied from. Let's fix that as well.
##########
File path:
hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestDatanodeReport.java
##########
@@ -172,8 +172,20 @@ public void testDatanodeReportMissingBlock() throws
Exception {
// all bad datanodes
}
cluster.triggerHeartbeats(); // IBR delete ack
- lb = fs.getClient().getLocatedBlocks(p.toString(), 0).get(0);
- assertEquals(0, lb.getLocations().length);
+ int c = 0;
+ while (true) {
+ lb = fs.getClient().getLocatedBlocks(p.toString(), 0).get(0);
+ if (0 != lb.getLocations().length) {
+ c++;
+ if (c > 7) {
+ assertEquals(0, lb.getLocations().length);
Review comment:
The `assertEquals` condition is contradictory to the condition `(0 !=
lb.getLocations().length)`.
If the intent is to fail the test here, just use
`Assert.fail("getLocatedBlocks failed after 7 retries")` instead.
##########
File path:
hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/BPServiceActor.java
##########
@@ -1104,6 +1122,34 @@ private void sendLifeline() throws IOException {
}
}
+ class IBRTaskHandler implements Runnable {
+
+ @Override
+ public void run() {
+ LOG.info("Starting IBR Task Handler.");
+ while (shouldRun()) {
+ try {
+ final long startTime = scheduler.monotonicNow();
+ final boolean sendHeartbeat = scheduler.isHeartbeatDue(startTime);
+ if (!dn.areIBRDisabledForTests() &&
+ (ibrManager.sendImmediately() || sendHeartbeat)) {
+ synchronized (sendIBRLock) {
+ ibrManager.sendIBRs(bpNamenode, bpRegistration,
+ bpos.getBlockPoolId(), getRpcMetricSuffix());
+ }
+ }
+ // There is no work to do; sleep until hearbeat timer elapses,
+ // or work arrives, and then iterate again.
+ ibrManager.waitTillNextIBR(scheduler.getHeartbeatWaitTime());
Review comment:
This implies this IBR timer will expire around the same time as the FBR
one (`offerService()`). This should be fine.
With IBR separated in a new thread, maybe later we could have a new config
key that controls IBR interval separately, or add a configurable constant
offset (from the FBR timer) to the IBR timer. This isn't something we need to
add to this jira. Just a thought.
Just in case I miss anything, @sodonnel would you like to take a quick look
at this?
##########
File path:
hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/datanode/TestIncrementalBlockReports.java
##########
@@ -167,13 +168,24 @@ public void testReportBlockDeleted() throws
InterruptedException, IOException {
// Trigger a heartbeat, this also triggers an IBR.
DataNodeTestUtils.triggerHeartbeat(singletonDn);
- Thread.sleep(2000);
// Ensure that the deleted block is reported.
- Mockito.verify(nnSpy, times(1)).blockReceivedAndDeleted(
- any(DatanodeRegistration.class),
- anyString(),
- any(StorageReceivedDeletedBlocks[].class));
+ int c = 0;
Review comment:
Same here
##########
File path:
hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestDatanodeReport.java
##########
@@ -172,8 +172,20 @@ public void testDatanodeReportMissingBlock() throws
Exception {
// all bad datanodes
}
cluster.triggerHeartbeats(); // IBR delete ack
- lb = fs.getClient().getLocatedBlocks(p.toString(), 0).get(0);
- assertEquals(0, lb.getLocations().length);
+ int c = 0;
Review comment:
nit: Try to use meaningful variable names, e.g.
`countRetries`/`count`/`counter` instead of just `c`.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
