[
https://issues.apache.org/jira/browse/HDFS-11182?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15762444#comment-15762444
]
ASF GitHub Bot commented on HDFS-11182:
---------------------------------------
Github user arp7 commented on a diff in the pull request:
https://github.com/apache/hadoop/pull/168#discussion_r93129689
--- Diff:
hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/checker/DatasetVolumeChecker.java
---
@@ -299,26 +327,35 @@ public void checkVolume(
private class ResultHandler
implements FutureCallback<VolumeCheckResult> {
private final FsVolumeReference reference;
- private final Set<StorageLocation> failedVolumes;
- private final Set<StorageLocation> healthyVolumes;
- private final CountDownLatch latch;
- private final AtomicLong numVolumes;
+ private final Set<FsVolumeSpi> failedVolumes;
+ private final Set<FsVolumeSpi> healthyVolumes;
+ private final Semaphore semaphore;
@Nullable
private final Callback callback;
+ /**
+ *
+ * @param reference FsVolumeReference to be released when the check is
+ * complete.
+ * @param healthyVolumes set of healthy volumes. If the disk check is
+ * successful, add the volume here.
+ * @param failedVolumes set of failed volumes. If the disk check fails,
+ * add the volume here.
+ * @param semaphore semaphore used to trigger callback invocation.
--- End diff --
CountDownLatch#countDown returns no value so there is no easy way to detect
when the count falls to zero and the callback can be invoked (it must be
invoked once only). I was using an AtomicLong to detect the 0->1 transition but
it had a bug.
The semaphore approach fixes it. We still need the CountDownLatch which we
can use as an event. I could have used an Object mutex instead but that would
have required extra code to deal with the spurious wakeup problem which
CountDownLatch does not suffer from.
> Update DataNode to use DatasetVolumeChecker
> -------------------------------------------
>
> Key: HDFS-11182
> URL: https://issues.apache.org/jira/browse/HDFS-11182
> Project: Hadoop HDFS
> Issue Type: Sub-task
> Components: datanode
> Reporter: Arpit Agarwal
> Assignee: Arpit Agarwal
>
> Update DataNode to use the DatasetVolumeChecker class introduced in
> HDFS-11149 to parallelize disk checks.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
