[
https://issues.apache.org/jira/browse/HDFS-16568?focusedWorklogId=767026&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-767026
]
ASF GitHub Bot logged work on HDFS-16568:
-----------------------------------------
Author: ASF GitHub Bot
Created on: 06/May/22 06:02
Start Date: 06/May/22 06:02
Worklog Time Spent: 10m
Work Description: tomscut commented on code in PR #4264:
URL: https://github.com/apache/hadoop/pull/4264#discussion_r866512560
##########
hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/tools/DFSAdmin.java:
##########
@@ -1915,12 +1919,57 @@ public int reconfig(String[] argv, int i) throws
IOException {
}
int startReconfiguration(final String nodeThpe, final String address)
- throws IOException {
- return startReconfiguration(nodeThpe, address, System.out, System.err);
+ throws IOException, InterruptedException {
+ return startReconfigurationUtil(nodeThpe, address, System.out, System.err);
+ }
+
+ int startReconfigurationUtil(final String nodeType, final String address,
final PrintStream out,
+ final PrintStream err) throws IOException, InterruptedException {
+ if (!"livenodes".equals(address)) {
+ return startReconfiguration(nodeType, address, out, err);
+ }
+ if (!"datanode".equals(nodeType)) {
+ err.println("Only datanode type supports reconfiguration in bulk.");
+ return 1;
+ }
+ ExecutorService executorService = Executors.newFixedThreadPool(5);
+ DistributedFileSystem dfs = getDFS();
+ DatanodeInfo[] nodes = dfs.getDataNodeStats(DatanodeReportType.LIVE);
+ AtomicInteger successCount = new AtomicInteger();
+ AtomicInteger failCount = new AtomicInteger();
+ if (nodes != null) {
+ for (DatanodeInfo node : nodes) {
+ executorService.submit(() -> {
+ int status = startReconfiguration(nodeType, node.getIpcAddr(false),
out, err);
+ if (status == 0) {
+ successCount.incrementAndGet();
+ } else {
+ failCount.incrementAndGet();
+ }
+ });
+ }
+ while ((successCount.get() + failCount.get()) < nodes.length) {
Review Comment:
Thanks @virajjasani for your explanation. Sounds good.
Issue Time Tracking
-------------------
Worklog Id: (was: 767026)
Time Spent: 1.5h (was: 1h 20m)
> dfsadmin -reconfig option to start/query reconfig on all live datanodes
> -----------------------------------------------------------------------
>
> Key: HDFS-16568
> URL: https://issues.apache.org/jira/browse/HDFS-16568
> Project: Hadoop HDFS
> Issue Type: New Feature
> Reporter: Viraj Jasani
> Assignee: Viraj Jasani
> Priority: Major
> Labels: pull-request-available
> Time Spent: 1.5h
> Remaining Estimate: 0h
>
> DFSAdmin provides option to initiate or query the status of reconfiguration
> operation on only specific host based on host:port provided by user. It would
> be good to provide an ability to initiate such operations in bulk, on all
> live datanodes.
--
This message was sent by Atlassian Jira
(v8.20.7#820007)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
