virajjasani commented on code in PR #4264:
URL: https://github.com/apache/hadoop/pull/4264#discussion_r866460800
##########
hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/tools/DFSAdmin.java:
##########
@@ -1915,12 +1919,57 @@ public int reconfig(String[] argv, int i) throws
IOException {
}
int startReconfiguration(final String nodeThpe, final String address)
- throws IOException {
- return startReconfiguration(nodeThpe, address, System.out, System.err);
+ throws IOException, InterruptedException {
+ return startReconfigurationUtil(nodeThpe, address, System.out, System.err);
+ }
+
+ int startReconfigurationUtil(final String nodeType, final String address,
final PrintStream out,
+ final PrintStream err) throws IOException, InterruptedException {
+ if (!"livenodes".equals(address)) {
+ return startReconfiguration(nodeType, address, out, err);
+ }
+ if (!"datanode".equals(nodeType)) {
+ err.println("Only datanode type supports reconfiguration in bulk.");
+ return 1;
+ }
+ ExecutorService executorService = Executors.newFixedThreadPool(5);
+ DistributedFileSystem dfs = getDFS();
+ DatanodeInfo[] nodes = dfs.getDataNodeStats(DatanodeReportType.LIVE);
+ AtomicInteger successCount = new AtomicInteger();
+ AtomicInteger failCount = new AtomicInteger();
+ if (nodes != null) {
+ for (DatanodeInfo node : nodes) {
+ executorService.submit(() -> {
+ int status = startReconfiguration(nodeType, node.getIpcAddr(false),
out, err);
+ if (status == 0) {
+ successCount.incrementAndGet();
+ } else {
+ failCount.incrementAndGet();
+ }
+ });
+ }
+ while ((successCount.get() + failCount.get()) < nodes.length) {
Review Comment:
Agree, countDownLatch would be better but since we are also keeping count of
both success and failure, and using these counts to return final status (0 or
1), it is difficult to allocate count down number initially. For instance, if
we use two latches - successCountDownLatch and failCountDownLatch, we can't
know what initial count to initiate both latches with. Hence, if we still want
to use CountDownLatch, we will have to use all 3 of them:
```
AtomicInteger successCount = new AtomicInteger();
AtomicInteger failCount = new AtomicInteger();
CountDownLatch countDownLatch = new CountDownLatch(nodes.length); //
total count as nodes array length
```
Hence, rather than using additional CountDownLatch, maybe we are good with
only two AtomicIntegers? WDYT?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
