[jira] [Updated] (HDFS-16243) The available disk space is less than the reserved space, and no log message is displayed
[ https://issues.apache.org/jira/browse/HDFS-16243?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hualong Zhang updated HDFS-16243: - Attachment: HDFS-16243.patch > The available disk space is less than the reserved space, and no log message > is displayed > - > > Key: HDFS-16243 > URL: https://issues.apache.org/jira/browse/HDFS-16243 > Project: Hadoop HDFS > Issue Type: Improvement > Components: datanode >Affects Versions: 2.7.2 >Reporter: Hualong Zhang >Priority: Major > Attachments: HDFS-16243.patch > > > When I submitted a task to the hadoop test cluster, it appeared "could only > be replicated to 0 nodes instead of minReplication (=1)" > I checked the namenode and datanode logs and did not find any error logs. It > was not until the use of dfsadmin -report that the available capacity was 0 > and I realized that it may be a configuration problem. > Checking the configuration found that the value of the > "dfs.datanode.du.reserved" configuration is greater than the available disk > space of HDFS, which caused this problem > It seems that there should be some warnings or errors in the log. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Updated] (HDFS-16243) The available disk space is less than the reserved space, and no log message is displayed
[ https://issues.apache.org/jira/browse/HDFS-16243?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hualong Zhang updated HDFS-16243: - Attachment: (was: HDFS-16243.patch) > The available disk space is less than the reserved space, and no log message > is displayed > - > > Key: HDFS-16243 > URL: https://issues.apache.org/jira/browse/HDFS-16243 > Project: Hadoop HDFS > Issue Type: Improvement > Components: datanode >Affects Versions: 2.7.2 >Reporter: Hualong Zhang >Priority: Major > > When I submitted a task to the hadoop test cluster, it appeared "could only > be replicated to 0 nodes instead of minReplication (=1)" > I checked the namenode and datanode logs and did not find any error logs. It > was not until the use of dfsadmin -report that the available capacity was 0 > and I realized that it may be a configuration problem. > Checking the configuration found that the value of the > "dfs.datanode.du.reserved" configuration is greater than the available disk > space of HDFS, which caused this problem > It seems that there should be some warnings or errors in the log. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-16243) The available disk space is less than the reserved space, and no log message is displayed
[
https://issues.apache.org/jira/browse/HDFS-16243?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17421924#comment-17421924
]
Renukaprasad C commented on HDFS-16243:
---
Thanks [~zhttylz] for the issue & the patch.
LOG.warn("Configured reserved space is higher than Disk capacity"); - Here can
you print values as well.
I think you created patch for the specific version. Is it applicable to trunk
as well?
Also, you can raise a PR, which will be easier for review & trace.
> The available disk space is less than the reserved space, and no log message
> is displayed
> -
>
> Key: HDFS-16243
> URL: https://issues.apache.org/jira/browse/HDFS-16243
> Project: Hadoop HDFS
> Issue Type: Improvement
> Components: datanode
>Affects Versions: 2.7.2
>Reporter: Hualong Zhang
>Priority: Major
> Attachments: HDFS-16243.patch
>
>
> When I submitted a task to the hadoop test cluster, it appeared "could only
> be replicated to 0 nodes instead of minReplication (=1)"
> I checked the namenode and datanode logs and did not find any error logs. It
> was not until the use of dfsadmin -report that the available capacity was 0
> and I realized that it may be a configuration problem.
> Checking the configuration found that the value of the
> "dfs.datanode.du.reserved" configuration is greater than the available disk
> space of HDFS, which caused this problem
> It seems that there should be some warnings or errors in the log.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Updated] (HDFS-16043) HDFS : Add markedDeleteBlockScrubberThread to delete blcoks asynchronously
[ https://issues.apache.org/jira/browse/HDFS-16043?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiangyi Zhu updated HDFS-16043: --- Summary: HDFS : Add markedDeleteBlockScrubberThread to delete blcoks asynchronously (was: HDFS : Delete performance optimization) > HDFS : Add markedDeleteBlockScrubberThread to delete blcoks asynchronously > -- > > Key: HDFS-16043 > URL: https://issues.apache.org/jira/browse/HDFS-16043 > Project: Hadoop HDFS > Issue Type: Improvement > Components: hdfs, namanode >Affects Versions: 3.4.0 >Reporter: Xiangyi Zhu >Assignee: Xiangyi Zhu >Priority: Major > Labels: pull-request-available > Attachments: 20210527-after.svg, 20210527-before.svg > > Time Spent: 5h > Remaining Estimate: 0h > > The deletion of the large directory caused NN to hold the lock for too long, > which caused our NameNode to be killed by ZKFC. > Through the flame graph, it is found that its main time-consuming > calculation is QuotaCount when removingBlocks(toRemovedBlocks) and deleting > inodes, and removeBlocks(toRemovedBlocks) takes a higher proportion of time. > h3. solution: > 1. RemoveBlocks is processed asynchronously. A thread is started in the > BlockManager to process the deleted blocks and control the lock time. > 2. QuotaCount calculation optimization, this is similar to the optimization > of this Issue HDFS-16000. > h3. Comparison before and after optimization: > Delete 1000w Inode and 1000w block test. > *before:* > remove inode elapsed time: 7691 ms > remove block elapsed time :11107 ms > *after:* > remove inode elapsed time: 4149 ms > remove block elapsed time :0 ms -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Work logged] (HDFS-16242) JournalMetrics should add JournalId MetricTag to distinguish different nameservice journal metrics.
[
https://issues.apache.org/jira/browse/HDFS-16242?focusedWorklogId=656978=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-656978
]
ASF GitHub Bot logged work on HDFS-16242:
-
Author: ASF GitHub Bot
Created on: 29/Sep/21 02:28
Start Date: 29/Sep/21 02:28
Worklog Time Spent: 10m
Work Description: Neilxzn commented on pull request #3494:
URL: https://github.com/apache/hadoop/pull/3494#issuecomment-929771633
cc @aajisaka @adamantal . Would you take some time to review this patch?
Thank you.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 656978)
Time Spent: 50m (was: 40m)
> JournalMetrics should add JournalId MetricTag to distinguish different
> nameservice journal metrics.
>
>
> Key: HDFS-16242
> URL: https://issues.apache.org/jira/browse/HDFS-16242
> Project: Hadoop HDFS
> Issue Type: Improvement
> Components: journal-node
>Affects Versions: 3.4.0
>Reporter: Max Xie
>Priority: Minor
> Labels: pull-request-available
> Time Spent: 50m
> Remaining Estimate: 0h
>
> JournalMetrics should add JournalId MetricTag to distinguish different
> nameservice journal metrics when JournalNode use ganglia sink or prometheus
> sink.
> *JMX*
> Before this patch, journal' jmx :
> {code:java}
> // jmx json
> {
> "name" : "Hadoop:service=JournalNode,name=Journal-nntest1",
> "Syncs60sNumOps" : 0,
> ...
> },
> {
>"name" : "Hadoop:service=JournalNode,name=Journal-nntest2",
>"Syncs60sNumOps" : 0,
>...
>}
> {code}
>
> After this patch, journal' jmx :
>
> {code:java}
> // jmx json
> {
> "name" : "Hadoop:service=JournalNode,name=Journal-nntest1",
> "tag.JournalId" : "nntest1", // add this tag
> "Syncs60sNumOps" : 0,
>...
> },
> {
>"name" : "Hadoop:service=JournalNode,name=Journal-nntest2",
> "tag.JournalId" : "nntest2",
>"Syncs60sNumOps" : 0,
> ...
> }
> {code}
>
> *PrometheusSink*
> Before this patch, journal' prometheus export :
>
> {code:java}
> journal_node_syncs60s_num_ops{context="dfs",hostname="host"} 2
> {code}
> After this patch, journal' prometheus export :
> {code:java}
> journal_node_syncs60s_num_ops{context="dfs",journalid="nntest2",hostname="host"}
> 2
> journal_node_syncs60s_num_ops{context="dfs",journalid="nntest1",hostname="host"}
> 75
> {code}
>
>
>
>
>
>
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Work logged] (HDFS-16043) HDFS : Delete performance optimization
[ https://issues.apache.org/jira/browse/HDFS-16043?focusedWorklogId=656974=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-656974 ] ASF GitHub Bot logged work on HDFS-16043: - Author: ASF GitHub Bot Created on: 29/Sep/21 02:17 Start Date: 29/Sep/21 02:17 Worklog Time Spent: 10m Work Description: jojochuang commented on pull request #3063: URL: https://github.com/apache/hadoop/pull/3063#issuecomment-929766828 Can we update the subject of this PR? As mentioned in a previous comment this PR makes block removal asynchronous and does actually speed up delete. https://github.com/apache/hadoop/pull/3063#issuecomment-868476838 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 656974) Time Spent: 5h (was: 4h 50m) > HDFS : Delete performance optimization > -- > > Key: HDFS-16043 > URL: https://issues.apache.org/jira/browse/HDFS-16043 > Project: Hadoop HDFS > Issue Type: Improvement > Components: hdfs, namanode >Affects Versions: 3.4.0 >Reporter: Xiangyi Zhu >Assignee: Xiangyi Zhu >Priority: Major > Labels: pull-request-available > Attachments: 20210527-after.svg, 20210527-before.svg > > Time Spent: 5h > Remaining Estimate: 0h > > The deletion of the large directory caused NN to hold the lock for too long, > which caused our NameNode to be killed by ZKFC. > Through the flame graph, it is found that its main time-consuming > calculation is QuotaCount when removingBlocks(toRemovedBlocks) and deleting > inodes, and removeBlocks(toRemovedBlocks) takes a higher proportion of time. > h3. solution: > 1. RemoveBlocks is processed asynchronously. A thread is started in the > BlockManager to process the deleted blocks and control the lock time. > 2. QuotaCount calculation optimization, this is similar to the optimization > of this Issue HDFS-16000. > h3. Comparison before and after optimization: > Delete 1000w Inode and 1000w block test. > *before:* > remove inode elapsed time: 7691 ms > remove block elapsed time :11107 ms > *after:* > remove inode elapsed time: 4149 ms > remove block elapsed time :0 ms -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-14575) LeaseRenewer#daemon threads leak in DFSClient
[ https://issues.apache.org/jira/browse/HDFS-14575?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17421893#comment-17421893 ] Wei-Chiu Chuang commented on HDFS-14575: Now that HDFS-16235 is resolved, we should consider backporting this and HDFS-16235 to branch-3.3/3.2 if deemed stable. > LeaseRenewer#daemon threads leak in DFSClient > - > > Key: HDFS-14575 > URL: https://issues.apache.org/jira/browse/HDFS-14575 > Project: Hadoop HDFS > Issue Type: Bug >Affects Versions: 3.1.0 >Reporter: Tao Yang >Assignee: Renukaprasad C >Priority: Major > Fix For: 3.4.0 > > Attachments: HDFS-14575.001.patch, HDFS-14575.002.patch, > HDFS-14575.003.patch, HDFS-14575.004.patch > > > Currently LeaseRenewer (and its daemon thread) without clients should be > terminated after a grace period which defaults to 60 seconds. A race > condition may happen when a new request is coming just after LeaseRenewer > expired. > Reproduce this race condition: > # Client#1 creates File#1: creates LeaseRenewer#1 and starts Daemon#1 > thread, after a few seconds, File#1 is closed , there is no clients in > LeaseRenewer#1 now. > # 60 seconds (grace period) later, LeaseRenewer#1 just expires but daemon#1 > thread is still in sleep, Client#1 creates File#2, lead to the creation of > Daemon#2. > # Daemon#1 is awake then exit, after that, LeaseRenewer#1 is removed from > factory. > # File#2 is closed after a few seconds, LeaseRenewer#2 is created since it > can’t get renewer from factory. > Daemon#2 thread leaks from now on, since Client#1 in it can never be removed > and it won't have a chance to stop. > To solve this problem, IIUIC, a simple way I think is to make sure that all > clients are cleared when LeaseRenewer is removed from factory. Please feel > free to give your suggestions. Thanks! -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Work logged] (HDFS-16238) Improve comments related to EncryptionZoneManager
[
https://issues.apache.org/jira/browse/HDFS-16238?focusedWorklogId=656971=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-656971
]
ASF GitHub Bot logged work on HDFS-16238:
-
Author: ASF GitHub Bot
Created on: 29/Sep/21 01:45
Start Date: 29/Sep/21 01:45
Worklog Time Spent: 10m
Work Description: jianghuazhu commented on pull request #3483:
URL: https://github.com/apache/hadoop/pull/3483#issuecomment-929753291
Thanks @virajjasani for the comment and review.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 656971)
Time Spent: 1h 50m (was: 1h 40m)
> Improve comments related to EncryptionZoneManager
> -
>
> Key: HDFS-16238
> URL: https://issues.apache.org/jira/browse/HDFS-16238
> Project: Hadoop HDFS
> Issue Type: Improvement
> Components: documentation, encryption, namenode
>Reporter: JiangHua Zhu
>Assignee: JiangHua Zhu
>Priority: Minor
> Labels: pull-request-available
> Time Spent: 1h 50m
> Remaining Estimate: 0h
>
> In EncryptionZoneManager, there are some missing
> The description of the relevant comment. The purpose of this jira is to
> perfect them.
> E.g:
>/**
> * Re-encrypts the given encryption zone path. If the given path is not the
> * root of an encryption zone, an exception is thrown.
> * @param zoneIIP
> * @param keyVersionName
> * @throws IOException
> */
>List reencryptEncryptionZone(final INodesInPath zoneIIP,
>final String keyVersionName) throws IOException {
> ..
> }
> The description of zoneIIP and keyVersionName is missing here.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Work logged] (HDFS-16238) Improve comments related to EncryptionZoneManager
[
https://issues.apache.org/jira/browse/HDFS-16238?focusedWorklogId=656864=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-656864
]
ASF GitHub Bot logged work on HDFS-16238:
-
Author: ASF GitHub Bot
Created on: 28/Sep/21 20:26
Start Date: 28/Sep/21 20:26
Worklog Time Spent: 10m
Work Description: virajjasani commented on pull request #3483:
URL: https://github.com/apache/hadoop/pull/3483#issuecomment-929483751
+1 (non-binding) from my side, thanks for the work @jianghuazhu. @ayushtkn
will be able to review this further.
Thanks
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 656864)
Time Spent: 1h 40m (was: 1.5h)
> Improve comments related to EncryptionZoneManager
> -
>
> Key: HDFS-16238
> URL: https://issues.apache.org/jira/browse/HDFS-16238
> Project: Hadoop HDFS
> Issue Type: Improvement
> Components: documentation, encryption, namenode
>Reporter: JiangHua Zhu
>Assignee: JiangHua Zhu
>Priority: Minor
> Labels: pull-request-available
> Time Spent: 1h 40m
> Remaining Estimate: 0h
>
> In EncryptionZoneManager, there are some missing
> The description of the relevant comment. The purpose of this jira is to
> perfect them.
> E.g:
>/**
> * Re-encrypts the given encryption zone path. If the given path is not the
> * root of an encryption zone, an exception is thrown.
> * @param zoneIIP
> * @param keyVersionName
> * @throws IOException
> */
>List reencryptEncryptionZone(final INodesInPath zoneIIP,
>final String keyVersionName) throws IOException {
> ..
> }
> The description of zoneIIP and keyVersionName is missing here.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Work logged] (HDFS-16239) XAttr#toString doesnt print the attribute value in readable format
[ https://issues.apache.org/jira/browse/HDFS-16239?focusedWorklogId=656849=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-656849 ] ASF GitHub Bot logged work on HDFS-16239: - Author: ASF GitHub Bot Created on: 28/Sep/21 20:24 Start Date: 28/Sep/21 20:24 Worklog Time Spent: 10m Work Description: prasad-acit opened a new pull request #3491: URL: https://github.com/apache/hadoop/pull/3491 …le format ### Description of PR ### How was this patch tested? ### For code changes: - [ ] Does the title or this PR starts with the corresponding JIRA issue id (e.g. 'HADOOP-17799. Your PR title ...')? - [ ] Object storage: have the integration tests been executed and the endpoint declared according to the connector-specific documentation? - [ ] If adding new dependencies to the code, are these dependencies licensed in a way that is compatible for inclusion under [ASF 2.0](http://www.apache.org/legal/resolved.html#category-a)? - [ ] If applicable, have you updated the `LICENSE`, `LICENSE-binary`, `NOTICE-binary` files? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 656849) Time Spent: 40m (was: 0.5h) > XAttr#toString doesnt print the attribute value in readable format > -- > > Key: HDFS-16239 > URL: https://issues.apache.org/jira/browse/HDFS-16239 > Project: Hadoop HDFS > Issue Type: Bug >Reporter: Renukaprasad C >Assignee: Renukaprasad C >Priority: Major > Labels: pull-request-available > Time Spent: 40m > Remaining Estimate: 0h > > org.apache.hadoop.fs.XAttr#toString prints the value of attribute in bytes. > return "XAttr [ns=" + ns + ", name=" + name + ", value=" > + Arrays.toString(value) + "]"; > XAttr [ns=SYSTEM, name=az.expression, value=[82, 69, 80, 91, 50, 93..] > This should be converted to String rather than printing to Array of bytes. > -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Work logged] (HDFS-16240) Replace unshaded guava in HttpFSServerWebServer
[ https://issues.apache.org/jira/browse/HDFS-16240?focusedWorklogId=656846=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-656846 ] ASF GitHub Bot logged work on HDFS-16240: - Author: ASF GitHub Bot Created on: 28/Sep/21 20:24 Start Date: 28/Sep/21 20:24 Worklog Time Spent: 10m Work Description: iwasakims commented on pull request #3492: URL: https://github.com/apache/hadoop/pull/3492#issuecomment-928765424 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 656846) Time Spent: 1.5h (was: 1h 20m) > Replace unshaded guava in HttpFSServerWebServer > --- > > Key: HDFS-16240 > URL: https://issues.apache.org/jira/browse/HDFS-16240 > Project: Hadoop HDFS > Issue Type: Bug > Components: httpfs >Reporter: Masatake Iwasaki >Assignee: Masatake Iwasaki >Priority: Major > Labels: pull-request-available > Fix For: 3.4.0 > > Time Spent: 1.5h > Remaining Estimate: 0h > > HDFS-16129 added use of com.google.common.annotations.VisibleForTesting to > HttpFSServerWebServer. It is replaced by replace-guava replacer of > HADOOP-17288 on every build time. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Work logged] (HDFS-16240) Replace unshaded guava in HttpFSServerWebServer
[ https://issues.apache.org/jira/browse/HDFS-16240?focusedWorklogId=656821=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-656821 ] ASF GitHub Bot logged work on HDFS-16240: - Author: ASF GitHub Bot Created on: 28/Sep/21 20:22 Start Date: 28/Sep/21 20:22 Worklog Time Spent: 10m Work Description: iwasakims opened a new pull request #3492: URL: https://github.com/apache/hadoop/pull/3492 https://issues.apache.org/jira/browse/HDFS-16240 [HDFS-16129](https://issues.apache.org/jira/browse/HDFS-16129) added use of com.google.common.annotations.VisibleForTesting to HttpFSServerWebServer. It is replaced by replace-guava replacer of [HADOOP-17288](https://issues.apache.org/jira/browse/HDFS-17288) on every build. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 656821) Time Spent: 1h 20m (was: 1h 10m) > Replace unshaded guava in HttpFSServerWebServer > --- > > Key: HDFS-16240 > URL: https://issues.apache.org/jira/browse/HDFS-16240 > Project: Hadoop HDFS > Issue Type: Bug > Components: httpfs >Reporter: Masatake Iwasaki >Assignee: Masatake Iwasaki >Priority: Major > Labels: pull-request-available > Fix For: 3.4.0 > > Time Spent: 1h 20m > Remaining Estimate: 0h > > HDFS-16129 added use of com.google.common.annotations.VisibleForTesting to > HttpFSServerWebServer. It is replaced by replace-guava replacer of > HADOOP-17288 on every build time. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Work logged] (HDFS-16237) Record the BPServiceActor information that communicates with Standby
[ https://issues.apache.org/jira/browse/HDFS-16237?focusedWorklogId=656800=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-656800 ] ASF GitHub Bot logged work on HDFS-16237: - Author: ASF GitHub Bot Created on: 28/Sep/21 20:20 Start Date: 28/Sep/21 20:20 Worklog Time Spent: 10m Work Description: ferhui commented on pull request #3479: URL: https://github.com/apache/hadoop/pull/3479#issuecomment-928824215 @jianghuazhu Thanks for contribution. @virajjasani @tomscut Thanks for review! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 656800) Time Spent: 3.5h (was: 3h 20m) > Record the BPServiceActor information that communicates with Standby > > > Key: HDFS-16237 > URL: https://issues.apache.org/jira/browse/HDFS-16237 > Project: Hadoop HDFS > Issue Type: Improvement > Components: datanode >Reporter: JiangHua Zhu >Assignee: JiangHua Zhu >Priority: Major > Labels: pull-request-available > Fix For: 3.4.0 > > Time Spent: 3.5h > Remaining Estimate: 0h > > When BPServiceActor communicates with Standby, the specific BPServiceActor > information should be recorded. Now it is directly filtered. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Work logged] (HDFS-16216) RBF: Wrong path when get mount point status
[
https://issues.apache.org/jira/browse/HDFS-16216?focusedWorklogId=656794=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-656794
]
ASF GitHub Bot logged work on HDFS-16216:
-
Author: ASF GitHub Bot
Created on: 28/Sep/21 20:19
Start Date: 28/Sep/21 20:19
Worklog Time Spent: 10m
Work Description: tomscut commented on a change in pull request #3415:
URL: https://github.com/apache/hadoop/pull/3415#discussion_r717704397
##
File path:
hadoop-hdfs-project/hadoop-hdfs-rbf/src/test/java/org/apache/hadoop/hdfs/server/federation/router/TestRouterMountTable.java
##
@@ -19,6 +19,7 @@
import static org.junit.Assert.assertEquals;
import static org.junit.Assert.assertFalse;
+import static org.junit.Assert.assertNull;
Review comment:
We should remove this unused import.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 656794)
Time Spent: 50m (was: 40m)
> RBF: Wrong path when get mount point status
> ---
>
> Key: HDFS-16216
> URL: https://issues.apache.org/jira/browse/HDFS-16216
> Project: Hadoop HDFS
> Issue Type: Bug
>Reporter: zhuobin zheng
>Assignee: zhuobin zheng
>Priority: Major
> Labels: pull-request-available
> Time Spent: 50m
> Remaining Estimate: 0h
>
> When we get mount point status, we will get wrong path in file status.
> {code:java}
> // prepare mount env
> hdfs dfsrouteradmin -add /test ns1 /test
> hdfs dfsrouteradmin -add /test/test1/test2/test3 ns1 /test/test1/test2/test3
> {code}
> {code:java}
> // java code
> public class TmpOp {
> public static void main(String[] args) throws Exception {
> Configuration conf = new HdfsConfiguration();
> FileSystem fs = FileSystem.get(conf);
> FileStatus test = fs.getFileStatus(new Path("/test"));
> FileStatus test1 = fs.getFileStatus(new Path("/test/test1"));
> FileStatus test2 = fs.getFileStatus(new Path("/test/test1/test2"));
> System.out.println(test.getPath());
> System.out.println(test1.getPath());
> System.out.println(test2.getPath());
> }
> }
> {code}
>
> {code:java}
> // result
> hdfs://ns1/test/test
> hdfs://ns1/test/test1/test1
> hdfs://ns1/test/test1/test2/test2
> {code}
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Work logged] (HDFS-16238) Improve comments related to EncryptionZoneManager
[
https://issues.apache.org/jira/browse/HDFS-16238?focusedWorklogId=656784=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-656784
]
ASF GitHub Bot logged work on HDFS-16238:
-
Author: ASF GitHub Bot
Created on: 28/Sep/21 20:18
Start Date: 28/Sep/21 20:18
Worklog Time Spent: 10m
Work Description: jianghuazhu commented on pull request #3483:
URL: https://github.com/apache/hadoop/pull/3483#issuecomment-928589828
@virajjasani, would you mind reviewing it again?
thank you very much.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 656784)
Time Spent: 1.5h (was: 1h 20m)
> Improve comments related to EncryptionZoneManager
> -
>
> Key: HDFS-16238
> URL: https://issues.apache.org/jira/browse/HDFS-16238
> Project: Hadoop HDFS
> Issue Type: Improvement
> Components: documentation, encryption, namenode
>Reporter: JiangHua Zhu
>Assignee: JiangHua Zhu
>Priority: Minor
> Labels: pull-request-available
> Time Spent: 1.5h
> Remaining Estimate: 0h
>
> In EncryptionZoneManager, there are some missing
> The description of the relevant comment. The purpose of this jira is to
> perfect them.
> E.g:
>/**
> * Re-encrypts the given encryption zone path. If the given path is not the
> * root of an encryption zone, an exception is thrown.
> * @param zoneIIP
> * @param keyVersionName
> * @throws IOException
> */
>List reencryptEncryptionZone(final INodesInPath zoneIIP,
>final String keyVersionName) throws IOException {
> ..
> }
> The description of zoneIIP and keyVersionName is missing here.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Work logged] (HDFS-16242) JournalMetrics should add JournalId MetricTag to distinguish different nameservice journal metrics.
[
https://issues.apache.org/jira/browse/HDFS-16242?focusedWorklogId=656665=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-656665
]
ASF GitHub Bot logged work on HDFS-16242:
-
Author: ASF GitHub Bot
Created on: 28/Sep/21 20:09
Start Date: 28/Sep/21 20:09
Worklog Time Spent: 10m
Work Description: hadoop-yetus commented on pull request #3494:
URL: https://github.com/apache/hadoop/pull/3494#issuecomment-929498362
:confetti_ball: **+1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|::|--:|:|::|:---:|
| +0 :ok: | reexec | 0m 47s | | Docker mode activated. |
_ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | | No case conflicting files
found. |
| +0 :ok: | codespell | 0m 1s | | codespell was not available. |
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain
any @author tags. |
| +1 :green_heart: | test4tests | 0m 0s | | The patch appears to
include 2 new or modified test files. |
_ trunk Compile Tests _ |
| +0 :ok: | mvndep | 12m 53s | | Maven dependency ordering for branch |
| +1 :green_heart: | mvninstall | 24m 14s | | trunk passed |
| +1 :green_heart: | compile | 23m 44s | | trunk passed with JDK
Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04 |
| +1 :green_heart: | compile | 21m 49s | | trunk passed with JDK
Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10 |
| +1 :green_heart: | checkstyle | 4m 20s | | trunk passed |
| +1 :green_heart: | mvnsite | 3m 16s | | trunk passed |
| +1 :green_heart: | javadoc | 2m 24s | | trunk passed with JDK
Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04 |
| +1 :green_heart: | javadoc | 3m 28s | | trunk passed with JDK
Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10 |
| +1 :green_heart: | spotbugs | 5m 41s | | trunk passed |
| +1 :green_heart: | shadedclient | 21m 45s | | branch has no errors
when building and testing our client artifacts. |
_ Patch Compile Tests _ |
| +0 :ok: | mvndep | 0m 27s | | Maven dependency ordering for patch |
| +1 :green_heart: | mvninstall | 2m 8s | | the patch passed |
| +1 :green_heart: | compile | 20m 26s | | the patch passed with JDK
Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04 |
| +1 :green_heart: | javac | 20m 26s | | the patch passed |
| +1 :green_heart: | compile | 18m 29s | | the patch passed with JDK
Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10 |
| +1 :green_heart: | javac | 18m 29s | | the patch passed |
| +1 :green_heart: | blanks | 0m 0s | | The patch has no blanks
issues. |
| +1 :green_heart: | checkstyle | 3m 33s | | the patch passed |
| +1 :green_heart: | mvnsite | 3m 11s | | the patch passed |
| +1 :green_heart: | javadoc | 2m 17s | | the patch passed with JDK
Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04 |
| +1 :green_heart: | javadoc | 3m 28s | | the patch passed with JDK
Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10 |
| +1 :green_heart: | spotbugs | 6m 7s | | the patch passed |
| +1 :green_heart: | shadedclient | 22m 8s | | patch has no errors
when building and testing our client artifacts. |
_ Other Tests _ |
| +1 :green_heart: | unit | 17m 15s | | hadoop-common in the patch
passed. |
| +1 :green_heart: | unit | 225m 5s | | hadoop-hdfs in the patch
passed. |
| +1 :green_heart: | asflicense | 1m 10s | | The patch does not
generate ASF License warnings. |
| | | 449m 58s | | |
| Subsystem | Report/Notes |
|--:|:-|
| Docker | ClientAPI=1.41 ServerAPI=1.41 base:
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3494/1/artifact/out/Dockerfile
|
| GITHUB PR | https://github.com/apache/hadoop/pull/3494 |
| Optional Tests | dupname asflicense compile javac javadoc mvninstall
mvnsite unit shadedclient spotbugs checkstyle codespell |
| uname | Linux ec622a565571 4.15.0-156-generic #163-Ubuntu SMP Thu Aug 19
23:31:58 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/bin/hadoop.sh |
| git revision | trunk / 84f3331ed7d67d3310a1372451f93f96d57ceaad |
| Default Java | Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10 |
| Multi-JDK versions |
/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04
/usr/lib/jvm/java-8-openjdk-amd64:Private
Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10 |
| Test Results |
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3494/1/testReport/ |
| Max. process+thread count | 3106 (vs. ulimit of 5500) |
| modules | C: hadoop-common-project/hadoop-common
hadoop-hdfs-project/hadoop-hdfs U:
[jira] [Work logged] (HDFS-16240) Replace unshaded guava in HttpFSServerWebServer
[
https://issues.apache.org/jira/browse/HDFS-16240?focusedWorklogId=656658=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-656658
]
ASF GitHub Bot logged work on HDFS-16240:
-
Author: ASF GitHub Bot
Created on: 28/Sep/21 20:08
Start Date: 28/Sep/21 20:08
Worklog Time Spent: 10m
Work Description: hadoop-yetus commented on pull request #3492:
URL: https://github.com/apache/hadoop/pull/3492#issuecomment-928861177
:broken_heart: **-1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|::|--:|:|::|:---:|
| +0 :ok: | reexec | 0m 44s | | Docker mode activated. |
_ Prechecks _ |
| +1 :green_heart: | dupname | 0m 1s | | No case conflicting files
found. |
| +0 :ok: | codespell | 0m 0s | | codespell was not available. |
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain
any @author tags. |
| -1 :x: | test4tests | 0m 0s | | The patch doesn't appear to include
any new or modified tests. Please justify why no new tests are needed for this
patch. Also please list what manual steps were performed to verify this patch.
|
_ trunk Compile Tests _ |
| +1 :green_heart: | mvninstall | 32m 5s | | trunk passed |
| +1 :green_heart: | compile | 0m 33s | | trunk passed with JDK
Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04 |
| +1 :green_heart: | compile | 0m 30s | | trunk passed with JDK
Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10 |
| +1 :green_heart: | checkstyle | 0m 27s | | trunk passed |
| +1 :green_heart: | mvnsite | 0m 41s | | trunk passed |
| +1 :green_heart: | javadoc | 0m 31s | | trunk passed with JDK
Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04 |
| +1 :green_heart: | javadoc | 0m 28s | | trunk passed with JDK
Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10 |
| +1 :green_heart: | spotbugs | 0m 50s | | trunk passed |
| +1 :green_heart: | shadedclient | 19m 11s | | branch has no errors
when building and testing our client artifacts. |
_ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 0m 25s | | the patch passed |
| +1 :green_heart: | compile | 0m 25s | | the patch passed with JDK
Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04 |
| +1 :green_heart: | javac | 0m 25s | | the patch passed |
| +1 :green_heart: | compile | 0m 22s | | the patch passed with JDK
Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10 |
| +1 :green_heart: | javac | 0m 22s | | the patch passed |
| +1 :green_heart: | blanks | 0m 0s | | The patch has no blanks
issues. |
| +1 :green_heart: | checkstyle | 0m 16s | | the patch passed |
| +1 :green_heart: | mvnsite | 0m 30s | | the patch passed |
| +1 :green_heart: | javadoc | 0m 21s | | the patch passed with JDK
Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04 |
| +1 :green_heart: | javadoc | 0m 20s | | the patch passed with JDK
Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10 |
| +1 :green_heart: | spotbugs | 0m 51s | | the patch passed |
| +1 :green_heart: | shadedclient | 18m 33s | | patch has no errors
when building and testing our client artifacts. |
_ Other Tests _ |
| +1 :green_heart: | unit | 5m 54s | | hadoop-hdfs-httpfs in the patch
passed. |
| +1 :green_heart: | asflicense | 0m 35s | | The patch does not
generate ASF License warnings. |
| | | 87m 7s | | |
| Subsystem | Report/Notes |
|--:|:-|
| Docker | ClientAPI=1.41 ServerAPI=1.41 base:
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3492/1/artifact/out/Dockerfile
|
| GITHUB PR | https://github.com/apache/hadoop/pull/3492 |
| Optional Tests | dupname asflicense compile javac javadoc mvninstall
mvnsite unit shadedclient spotbugs checkstyle codespell |
| uname | Linux 56f2d30fc148 4.15.0-112-generic #113-Ubuntu SMP Thu Jul 9
23:41:39 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/bin/hadoop.sh |
| git revision | trunk / 57f1fe06ec927a6b0755df1cbecc6926dfd54e76 |
| Default Java | Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10 |
| Multi-JDK versions |
/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04
/usr/lib/jvm/java-8-openjdk-amd64:Private
Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10 |
| Test Results |
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3492/1/testReport/ |
| Max. process+thread count | 737 (vs. ulimit of 5500) |
| modules | C: hadoop-hdfs-project/hadoop-hdfs-httpfs U:
hadoop-hdfs-project/hadoop-hdfs-httpfs |
| Console output |
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3492/1/console |
|
[jira] [Work logged] (HDFS-16231) Fix TestDataNodeMetrics#testReceivePacketSlowMetrics
[
https://issues.apache.org/jira/browse/HDFS-16231?focusedWorklogId=656608=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-656608
]
ASF GitHub Bot logged work on HDFS-16231:
-
Author: ASF GitHub Bot
Created on: 28/Sep/21 20:03
Start Date: 28/Sep/21 20:03
Worklog Time Spent: 10m
Work Description: haiyang1987 commented on pull request #3471:
URL: https://github.com/apache/hadoop/pull/3471#issuecomment-928867034
@ferhui Thanks your review and merge.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 656608)
Time Spent: 2h 40m (was: 2.5h)
> Fix TestDataNodeMetrics#testReceivePacketSlowMetrics
>
>
> Key: HDFS-16231
> URL: https://issues.apache.org/jira/browse/HDFS-16231
> Project: Hadoop HDFS
> Issue Type: Sub-task
>Reporter: Haiyang Hu
>Assignee: Haiyang Hu
>Priority: Major
> Labels: pull-request-available
> Fix For: 3.4.0
>
> Time Spent: 2h 40m
> Remaining Estimate: 0h
>
> TestDataNodeMetrics#testReceivePacketSlowMetrics fails with stacktrace:
> {code:java}
> java.lang.AssertionError: Expected exactly one metric for name
> TotalPacketsReceived
> Expected :1
> Actual :0
>
> at org.junit.Assert.fail(Assert.java:89)
> at org.junit.Assert.failNotEquals(Assert.java:835)
> at org.junit.Assert.assertEquals(Assert.java:647)
> at
> org.apache.hadoop.test.MetricsAsserts.checkCaptured(MetricsAsserts.java:278)
> at
> org.apache.hadoop.test.MetricsAsserts.getLongCounter(MetricsAsserts.java:237)
> at
> org.apache.hadoop.hdfs.server.datanode.TestDataNodeMetrics.testReceivePacketSlowMetrics(TestDataNodeMetrics.java:200)
> {code}
> {code:java}
> // Error MetricsName in current code，e.g
> TotalPacketsReceived，TotalPacketsSlowWriteToMirror，TotalPacketsSlowWriteToDisk，TotalPacketsSlowWriteToOsCache
> MetricsRecordBuilder dnMetrics =
> getMetrics(datanode.getMetrics().name());
> assertTrue("More than 1 packet received",
> getLongCounter("TotalPacketsReceived", dnMetrics) > 1L);
> assertTrue("More than 1 slow packet to mirror",
> getLongCounter("TotalPacketsSlowWriteToMirror", dnMetrics) > 1L);
> assertCounter("TotalPacketsSlowWriteToDisk", 1L, dnMetrics);
> assertCounter("TotalPacketsSlowWriteToOsCache", 0L, dnMetrics);
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Work logged] (HDFS-16231) Fix TestDataNodeMetrics#testReceivePacketSlowMetrics
[
https://issues.apache.org/jira/browse/HDFS-16231?focusedWorklogId=656579=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-656579
]
ASF GitHub Bot logged work on HDFS-16231:
-
Author: ASF GitHub Bot
Created on: 28/Sep/21 20:01
Start Date: 28/Sep/21 20:01
Worklog Time Spent: 10m
Work Description: ferhui commented on pull request #3471:
URL: https://github.com/apache/hadoop/pull/3471#issuecomment-928825711
@haiyang1987 Thanks for contribution. Merged
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 656579)
Time Spent: 2.5h (was: 2h 20m)
> Fix TestDataNodeMetrics#testReceivePacketSlowMetrics
>
>
> Key: HDFS-16231
> URL: https://issues.apache.org/jira/browse/HDFS-16231
> Project: Hadoop HDFS
> Issue Type: Sub-task
>Reporter: Haiyang Hu
>Assignee: Haiyang Hu
>Priority: Major
> Labels: pull-request-available
> Fix For: 3.4.0
>
> Time Spent: 2.5h
> Remaining Estimate: 0h
>
> TestDataNodeMetrics#testReceivePacketSlowMetrics fails with stacktrace:
> {code:java}
> java.lang.AssertionError: Expected exactly one metric for name
> TotalPacketsReceived
> Expected :1
> Actual :0
>
> at org.junit.Assert.fail(Assert.java:89)
> at org.junit.Assert.failNotEquals(Assert.java:835)
> at org.junit.Assert.assertEquals(Assert.java:647)
> at
> org.apache.hadoop.test.MetricsAsserts.checkCaptured(MetricsAsserts.java:278)
> at
> org.apache.hadoop.test.MetricsAsserts.getLongCounter(MetricsAsserts.java:237)
> at
> org.apache.hadoop.hdfs.server.datanode.TestDataNodeMetrics.testReceivePacketSlowMetrics(TestDataNodeMetrics.java:200)
> {code}
> {code:java}
> // Error MetricsName in current code，e.g
> TotalPacketsReceived，TotalPacketsSlowWriteToMirror，TotalPacketsSlowWriteToDisk，TotalPacketsSlowWriteToOsCache
> MetricsRecordBuilder dnMetrics =
> getMetrics(datanode.getMetrics().name());
> assertTrue("More than 1 packet received",
> getLongCounter("TotalPacketsReceived", dnMetrics) > 1L);
> assertTrue("More than 1 slow packet to mirror",
> getLongCounter("TotalPacketsSlowWriteToMirror", dnMetrics) > 1L);
> assertCounter("TotalPacketsSlowWriteToDisk", 1L, dnMetrics);
> assertCounter("TotalPacketsSlowWriteToOsCache", 0L, dnMetrics);
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Work logged] (HDFS-16241) Standby close reconstruction thread
[
https://issues.apache.org/jira/browse/HDFS-16241?focusedWorklogId=656568=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-656568
]
ASF GitHub Bot logged work on HDFS-16241:
-
Author: ASF GitHub Bot
Created on: 28/Sep/21 20:00
Start Date: 28/Sep/21 20:00
Worklog Time Spent: 10m
Work Description: hadoop-yetus commented on pull request #3493:
URL: https://github.com/apache/hadoop/pull/3493#issuecomment-929131317
:broken_heart: **-1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|::|--:|:|::|:---:|
| +0 :ok: | reexec | 0m 41s | | Docker mode activated. |
_ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | | No case conflicting files
found. |
| +0 :ok: | codespell | 0m 1s | | codespell was not available. |
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain
any @author tags. |
| -1 :x: | test4tests | 0m 0s | | The patch doesn't appear to include
any new or modified tests. Please justify why no new tests are needed for this
patch. Also please list what manual steps were performed to verify this patch.
|
_ trunk Compile Tests _ |
| +1 :green_heart: | mvninstall | 32m 11s | | trunk passed |
| +1 :green_heart: | compile | 1m 23s | | trunk passed with JDK
Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04 |
| +1 :green_heart: | compile | 1m 16s | | trunk passed with JDK
Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10 |
| +1 :green_heart: | checkstyle | 1m 2s | | trunk passed |
| +1 :green_heart: | mvnsite | 1m 22s | | trunk passed |
| +1 :green_heart: | javadoc | 0m 55s | | trunk passed with JDK
Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04 |
| +1 :green_heart: | javadoc | 1m 28s | | trunk passed with JDK
Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10 |
| +1 :green_heart: | spotbugs | 3m 6s | | trunk passed |
| +1 :green_heart: | shadedclient | 21m 17s | | branch has no errors
when building and testing our client artifacts. |
_ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 1m 12s | | the patch passed |
| +1 :green_heart: | compile | 1m 13s | | the patch passed with JDK
Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04 |
| +1 :green_heart: | javac | 1m 13s | | the patch passed |
| +1 :green_heart: | compile | 1m 7s | | the patch passed with JDK
Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10 |
| +1 :green_heart: | javac | 1m 7s | | the patch passed |
| +1 :green_heart: | blanks | 0m 0s | | The patch has no blanks
issues. |
| +1 :green_heart: | checkstyle | 0m 52s | | the patch passed |
| +1 :green_heart: | mvnsite | 1m 17s | | the patch passed |
| +1 :green_heart: | javadoc | 0m 45s | | the patch passed with JDK
Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04 |
| +1 :green_heart: | javadoc | 1m 21s | | the patch passed with JDK
Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10 |
| +1 :green_heart: | spotbugs | 3m 8s | | the patch passed |
| +1 :green_heart: | shadedclient | 20m 47s | | patch has no errors
when building and testing our client artifacts. |
_ Other Tests _ |
| +1 :green_heart: | unit | 234m 13s | | hadoop-hdfs in the patch
passed. |
| +1 :green_heart: | asflicense | 0m 41s | | The patch does not
generate ASF License warnings. |
| | | 329m 23s | | |
| Subsystem | Report/Notes |
|--:|:-|
| Docker | ClientAPI=1.41 ServerAPI=1.41 base:
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3493/1/artifact/out/Dockerfile
|
| GITHUB PR | https://github.com/apache/hadoop/pull/3493 |
| Optional Tests | dupname asflicense compile javac javadoc mvninstall
mvnsite unit shadedclient spotbugs checkstyle codespell |
| uname | Linux 32505d960d16 4.15.0-156-generic #163-Ubuntu SMP Thu Aug 19
23:31:58 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/bin/hadoop.sh |
| git revision | trunk / 657d6e7b600a5361767eb9a8827a1a236823c788 |
| Default Java | Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10 |
| Multi-JDK versions |
/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04
/usr/lib/jvm/java-8-openjdk-amd64:Private
Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10 |
| Test Results |
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3493/1/testReport/ |
| Max. process+thread count | 3696 (vs. ulimit of 5500) |
| modules | C: hadoop-hdfs-project/hadoop-hdfs U:
hadoop-hdfs-project/hadoop-hdfs |
| Console output |
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3493/1/console |
| versions |
[jira] [Work logged] (HDFS-16240) Replace unshaded guava in HttpFSServerWebServer
[ https://issues.apache.org/jira/browse/HDFS-16240?focusedWorklogId=656540=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-656540 ] ASF GitHub Bot logged work on HDFS-16240: - Author: ASF GitHub Bot Created on: 28/Sep/21 19:58 Start Date: 28/Sep/21 19:58 Worklog Time Spent: 10m Work Description: iwasakims merged pull request #3492: URL: https://github.com/apache/hadoop/pull/3492 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 656540) Time Spent: 1h (was: 50m) > Replace unshaded guava in HttpFSServerWebServer > --- > > Key: HDFS-16240 > URL: https://issues.apache.org/jira/browse/HDFS-16240 > Project: Hadoop HDFS > Issue Type: Bug > Components: httpfs >Reporter: Masatake Iwasaki >Assignee: Masatake Iwasaki >Priority: Major > Labels: pull-request-available > Fix For: 3.4.0 > > Time Spent: 1h > Remaining Estimate: 0h > > HDFS-16129 added use of com.google.common.annotations.VisibleForTesting to > HttpFSServerWebServer. It is replaced by replace-guava replacer of > HADOOP-17288 on every build time. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Work logged] (HDFS-16237) Record the BPServiceActor information that communicates with Standby
[ https://issues.apache.org/jira/browse/HDFS-16237?focusedWorklogId=656530=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-656530 ] ASF GitHub Bot logged work on HDFS-16237: - Author: ASF GitHub Bot Created on: 28/Sep/21 19:57 Start Date: 28/Sep/21 19:57 Worklog Time Spent: 10m Work Description: ferhui merged pull request #3479: URL: https://github.com/apache/hadoop/pull/3479 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 656530) Time Spent: 3h 20m (was: 3h 10m) > Record the BPServiceActor information that communicates with Standby > > > Key: HDFS-16237 > URL: https://issues.apache.org/jira/browse/HDFS-16237 > Project: Hadoop HDFS > Issue Type: Improvement > Components: datanode >Reporter: JiangHua Zhu >Assignee: JiangHua Zhu >Priority: Major > Labels: pull-request-available > Fix For: 3.4.0 > > Time Spent: 3h 20m > Remaining Estimate: 0h > > When BPServiceActor communicates with Standby, the specific BPServiceActor > information should be recorded. Now it is directly filtered. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Work logged] (HDFS-16239) XAttr#toString doesnt print the attribute value in readable format
[
https://issues.apache.org/jira/browse/HDFS-16239?focusedWorklogId=656529=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-656529
]
ASF GitHub Bot logged work on HDFS-16239:
-
Author: ASF GitHub Bot
Created on: 28/Sep/21 19:57
Start Date: 28/Sep/21 19:57
Worklog Time Spent: 10m
Work Description: hadoop-yetus commented on pull request #3491:
URL: https://github.com/apache/hadoop/pull/3491#issuecomment-928241628
:broken_heart: **-1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|::|--:|:|::|:---:|
| +0 :ok: | reexec | 1m 15s | | Docker mode activated. |
_ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | | No case conflicting files
found. |
| +0 :ok: | codespell | 0m 1s | | codespell was not available. |
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain
any @author tags. |
| -1 :x: | test4tests | 0m 0s | | The patch doesn't appear to include
any new or modified tests. Please justify why no new tests are needed for this
patch. Also please list what manual steps were performed to verify this patch.
|
_ trunk Compile Tests _ |
| +1 :green_heart: | mvninstall | 37m 25s | | trunk passed |
| +1 :green_heart: | compile | 1m 6s | | trunk passed with JDK
Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04 |
| +1 :green_heart: | compile | 0m 53s | | trunk passed with JDK
Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10 |
| +1 :green_heart: | checkstyle | 0m 27s | | trunk passed |
| +1 :green_heart: | mvnsite | 0m 59s | | trunk passed |
| +1 :green_heart: | javadoc | 0m 42s | | trunk passed with JDK
Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04 |
| +1 :green_heart: | javadoc | 0m 35s | | trunk passed with JDK
Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10 |
| +1 :green_heart: | spotbugs | 2m 52s | | trunk passed |
| +1 :green_heart: | shadedclient | 25m 42s | | branch has no errors
when building and testing our client artifacts. |
_ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 0m 58s | | the patch passed |
| +1 :green_heart: | compile | 1m 4s | | the patch passed with JDK
Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04 |
| +1 :green_heart: | javac | 1m 4s | | the patch passed |
| +1 :green_heart: | compile | 0m 56s | | the patch passed with JDK
Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10 |
| +1 :green_heart: | javac | 0m 56s | | the patch passed |
| +1 :green_heart: | blanks | 0m 0s | | The patch has no blanks
issues. |
| -0 :warning: | checkstyle | 0m 21s |
[/results-checkstyle-hadoop-hdfs-project_hadoop-hdfs-client.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3491/1/artifact/out/results-checkstyle-hadoop-hdfs-project_hadoop-hdfs-client.txt)
| hadoop-hdfs-project/hadoop-hdfs-client: The patch generated 1 new + 8
unchanged - 0 fixed = 9 total (was 8) |
| +1 :green_heart: | mvnsite | 0m 54s | | the patch passed |
| +1 :green_heart: | javadoc | 0m 42s | | the patch passed with JDK
Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04 |
| +1 :green_heart: | javadoc | 0m 34s | | the patch passed with JDK
Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10 |
| -1 :x: | spotbugs | 3m 36s |
[/new-spotbugs-hadoop-hdfs-project_hadoop-hdfs-client.html](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3491/1/artifact/out/new-spotbugs-hadoop-hdfs-project_hadoop-hdfs-client.html)
| hadoop-hdfs-project/hadoop-hdfs-client generated 1 new + 0 unchanged - 0
fixed = 1 total (was 0) |
| +1 :green_heart: | shadedclient | 26m 42s | | patch has no errors
when building and testing our client artifacts. |
_ Other Tests _ |
| +1 :green_heart: | unit | 2m 33s | | hadoop-hdfs-client in the patch
passed. |
| +1 :green_heart: | asflicense | 0m 32s | | The patch does not
generate ASF License warnings. |
| | | 108m 58s | | |
| Reason | Tests |
|---:|:--|
| SpotBugs | module:hadoop-hdfs-project/hadoop-hdfs-client |
| | Found reliance on default encoding in
org.apache.hadoop.fs.XAttr.toString():in org.apache.hadoop.fs.XAttr.toString():
new String(byte[]) At XAttr.java:[line 160] |
| Subsystem | Report/Notes |
|--:|:-|
| Docker | ClientAPI=1.41 ServerAPI=1.41 base:
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3491/1/artifact/out/Dockerfile
|
| GITHUB PR | https://github.com/apache/hadoop/pull/3491 |
| Optional Tests | dupname asflicense compile javac javadoc mvninstall
mvnsite unit shadedclient spotbugs checkstyle codespell |
| uname | Linux 918b11097bf1 4.15.0-147-generic
[jira] [Work logged] (HDFS-16155) Allow configurable exponential backoff in DFSInputStream refetchLocations
[ https://issues.apache.org/jira/browse/HDFS-16155?focusedWorklogId=656503=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-656503 ] ASF GitHub Bot logged work on HDFS-16155: - Author: ASF GitHub Bot Created on: 28/Sep/21 19:55 Start Date: 28/Sep/21 19:55 Worklog Time Spent: 10m Work Description: bbeaudreault commented on pull request #3271: URL: https://github.com/apache/hadoop/pull/3271#issuecomment-929150515 Any other comments on this patch? As tests demonstrate, it should have no impact on existing use-cases, aside from the requested backoff ceiling. It will enable operators to unlock faster retries if desired, and is much easier to read and test code. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 656503) Time Spent: 2h 40m (was: 2.5h) > Allow configurable exponential backoff in DFSInputStream refetchLocations > - > > Key: HDFS-16155 > URL: https://issues.apache.org/jira/browse/HDFS-16155 > Project: Hadoop HDFS > Issue Type: Improvement > Components: dfsclient >Reporter: Bryan Beaudreault >Assignee: Bryan Beaudreault >Priority: Minor > Labels: pull-request-available > Time Spent: 2h 40m > Remaining Estimate: 0h > > The retry policy in > [DFSInputStream#refetchLocations|https://github.com/apache/hadoop/blob/trunk/hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/DFSInputStream.java#L1018-L1040] > was first written many years ago. It allows configuration of the base time > window, but subsequent retries double in an un-configurable way. This retry > strategy makes sense in some clusters as it's very conservative and will > avoid DDOSing the namenode in certain systemic failure modes – for example, > if a file is being read by a large hadoop job and the underlying blocks are > moved by the balancer. In this case, enough datanodes would be added to the > deadNodes list and all hadoop tasks would simultaneously try to refetch the > blocks. The 3s doubling with random factor helps break up that stampeding > herd. > However, not all cluster use-cases are created equal, so there are other > cases where a more aggressive initial backoff is preferred. For example in a > low-latency single reader scenario. In this case, if the balancer moves > enough blocks, the reader hits this 3s backoff which is way too long for a > low latency use-case. > One could configure the the window very low (10ms), but then you can hit > other systemic failure modes which would result in readers DDOSing the > namenode again. For example, if blocks went missing due to truly dead > datanodes. In this case, many readers might be refetching locations for > different files with retry backoffs like 10ms, 20ms, 40ms, etc. It takes a > while to backoff enough to avoid impacting the namenode with that strategy. > I suggest adding a configurable multiplier to the backoff strategy so that > operators can tune this as they see fit for their use-case. In the above low > latency case, one could set the base very low (say 2ms) and the multiplier > very high (say 50). This gives an aggressive first retry that very quickly > backs off. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Work logged] (HDFS-16242) JournalMetrics should add JournalId MetricTag to distinguish different nameservice journal metrics.
[
https://issues.apache.org/jira/browse/HDFS-16242?focusedWorklogId=656472=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-656472
]
ASF GitHub Bot logged work on HDFS-16242:
-
Author: ASF GitHub Bot
Created on: 28/Sep/21 19:52
Start Date: 28/Sep/21 19:52
Worklog Time Spent: 10m
Work Description: Neilxzn opened a new pull request #3494:
URL: https://github.com/apache/hadoop/pull/3494
### Description of PR
https://issues.apache.org/jira/browse/HDFS-16242
JournalMetrics should add JournalId MetricTag to distinguish different
nameservice journal metrics when JournalNode use ganglia sink or prometheus
sink.
JMX
Before this patch, journal' jmx :
// jmx json
{
"name" : "Hadoop:service=JournalNode,name=Journal-nntest1",
"Syncs60sNumOps" : 0,
...
},
{
"name" : "Hadoop:service=JournalNode,name=Journal-nntest2",
"Syncs60sNumOps" : 0,
...
}
After this patch, journal' jmx :
// jmx json
{
"name" : "Hadoop:service=JournalNode,name=Journal-nntest1",
"tag.JournalId" : "nntest1", // add this tag
"Syncs60sNumOps" : 0,
...
},
{
"name" : "Hadoop:service=JournalNode,name=Journal-nntest2",
"tag.JournalId" : "nntest2",
"Syncs60sNumOps" : 0,
...
}
PrometheusSink
Before this patch, journal' prometheus export :
journal_node_syncs60s_num_ops{context="dfs",hostname="host"} 2
After this patch, journal' prometheus export :
journal_node_syncs60s_num_ops{context="dfs",journalid="nntest2",hostname="host"}
2
journal_node_syncs60s_num_ops{context="dfs",journalid="nntest1",hostname="host"}
75
### How was this patch tested?
add test testJournalMetricTags
### For code changes:
add JournalMetrics$getJournalId
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 656472)
Time Spent: 0.5h (was: 20m)
> JournalMetrics should add JournalId MetricTag to distinguish different
> nameservice journal metrics.
>
>
> Key: HDFS-16242
> URL: https://issues.apache.org/jira/browse/HDFS-16242
> Project: Hadoop HDFS
> Issue Type: Improvement
> Components: journal-node
>Affects Versions: 3.4.0
>Reporter: Max Xie
>Priority: Minor
> Labels: pull-request-available
> Time Spent: 0.5h
> Remaining Estimate: 0h
>
> JournalMetrics should add JournalId MetricTag to distinguish different
> nameservice journal metrics when JournalNode use ganglia sink or prometheus
> sink.
> *JMX*
> Before this patch, journal' jmx :
> {code:java}
> // jmx json
> {
> "name" : "Hadoop:service=JournalNode,name=Journal-nntest1",
> "Syncs60sNumOps" : 0,
> ...
> },
> {
>"name" : "Hadoop:service=JournalNode,name=Journal-nntest2",
>"Syncs60sNumOps" : 0,
>...
>}
> {code}
>
> After this patch, journal' jmx :
>
> {code:java}
> // jmx json
> {
> "name" : "Hadoop:service=JournalNode,name=Journal-nntest1",
> "tag.JournalId" : "nntest1", // add this tag
> "Syncs60sNumOps" : 0,
>...
> },
> {
>"name" : "Hadoop:service=JournalNode,name=Journal-nntest2",
> "tag.JournalId" : "nntest2",
>"Syncs60sNumOps" : 0,
> ...
> }
> {code}
>
> *PrometheusSink*
> Before this patch, journal' prometheus export :
>
> {code:java}
> journal_node_syncs60s_num_ops{context="dfs",hostname="host"} 2
> {code}
> After this patch, journal' prometheus export :
> {code:java}
> journal_node_syncs60s_num_ops{context="dfs",journalid="nntest2",hostname="host"}
> 2
> journal_node_syncs60s_num_ops{context="dfs",journalid="nntest1",hostname="host"}
> 75
> {code}
>
>
>
>
>
>
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Work logged] (HDFS-16231) Fix TestDataNodeMetrics#testReceivePacketSlowMetrics
[
https://issues.apache.org/jira/browse/HDFS-16231?focusedWorklogId=656439=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-656439
]
ASF GitHub Bot logged work on HDFS-16231:
-
Author: ASF GitHub Bot
Created on: 28/Sep/21 19:49
Start Date: 28/Sep/21 19:49
Worklog Time Spent: 10m
Work Description: ferhui merged pull request #3471:
URL: https://github.com/apache/hadoop/pull/3471
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 656439)
Time Spent: 2h 20m (was: 2h 10m)
> Fix TestDataNodeMetrics#testReceivePacketSlowMetrics
>
>
> Key: HDFS-16231
> URL: https://issues.apache.org/jira/browse/HDFS-16231
> Project: Hadoop HDFS
> Issue Type: Sub-task
>Reporter: Haiyang Hu
>Assignee: Haiyang Hu
>Priority: Major
> Labels: pull-request-available
> Fix For: 3.4.0
>
> Time Spent: 2h 20m
> Remaining Estimate: 0h
>
> TestDataNodeMetrics#testReceivePacketSlowMetrics fails with stacktrace:
> {code:java}
> java.lang.AssertionError: Expected exactly one metric for name
> TotalPacketsReceived
> Expected :1
> Actual :0
>
> at org.junit.Assert.fail(Assert.java:89)
> at org.junit.Assert.failNotEquals(Assert.java:835)
> at org.junit.Assert.assertEquals(Assert.java:647)
> at
> org.apache.hadoop.test.MetricsAsserts.checkCaptured(MetricsAsserts.java:278)
> at
> org.apache.hadoop.test.MetricsAsserts.getLongCounter(MetricsAsserts.java:237)
> at
> org.apache.hadoop.hdfs.server.datanode.TestDataNodeMetrics.testReceivePacketSlowMetrics(TestDataNodeMetrics.java:200)
> {code}
> {code:java}
> // Error MetricsName in current code，e.g
> TotalPacketsReceived，TotalPacketsSlowWriteToMirror，TotalPacketsSlowWriteToDisk，TotalPacketsSlowWriteToOsCache
> MetricsRecordBuilder dnMetrics =
> getMetrics(datanode.getMetrics().name());
> assertTrue("More than 1 packet received",
> getLongCounter("TotalPacketsReceived", dnMetrics) > 1L);
> assertTrue("More than 1 slow packet to mirror",
> getLongCounter("TotalPacketsSlowWriteToMirror", dnMetrics) > 1L);
> assertCounter("TotalPacketsSlowWriteToDisk", 1L, dnMetrics);
> assertCounter("TotalPacketsSlowWriteToOsCache", 0L, dnMetrics);
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Work logged] (HDFS-16242) JournalMetrics should add JournalId MetricTag to distinguish different nameservice journal metrics.
[
https://issues.apache.org/jira/browse/HDFS-16242?focusedWorklogId=656301=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-656301
]
ASF GitHub Bot logged work on HDFS-16242:
-
Author: ASF GitHub Bot
Created on: 28/Sep/21 18:04
Start Date: 28/Sep/21 18:04
Worklog Time Spent: 10m
Work Description: hadoop-yetus commented on pull request #3494:
URL: https://github.com/apache/hadoop/pull/3494#issuecomment-929498362
:confetti_ball: **+1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|::|--:|:|::|:---:|
| +0 :ok: | reexec | 0m 47s | | Docker mode activated. |
_ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | | No case conflicting files
found. |
| +0 :ok: | codespell | 0m 1s | | codespell was not available. |
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain
any @author tags. |
| +1 :green_heart: | test4tests | 0m 0s | | The patch appears to
include 2 new or modified test files. |
_ trunk Compile Tests _ |
| +0 :ok: | mvndep | 12m 53s | | Maven dependency ordering for branch |
| +1 :green_heart: | mvninstall | 24m 14s | | trunk passed |
| +1 :green_heart: | compile | 23m 44s | | trunk passed with JDK
Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04 |
| +1 :green_heart: | compile | 21m 49s | | trunk passed with JDK
Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10 |
| +1 :green_heart: | checkstyle | 4m 20s | | trunk passed |
| +1 :green_heart: | mvnsite | 3m 16s | | trunk passed |
| +1 :green_heart: | javadoc | 2m 24s | | trunk passed with JDK
Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04 |
| +1 :green_heart: | javadoc | 3m 28s | | trunk passed with JDK
Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10 |
| +1 :green_heart: | spotbugs | 5m 41s | | trunk passed |
| +1 :green_heart: | shadedclient | 21m 45s | | branch has no errors
when building and testing our client artifacts. |
_ Patch Compile Tests _ |
| +0 :ok: | mvndep | 0m 27s | | Maven dependency ordering for patch |
| +1 :green_heart: | mvninstall | 2m 8s | | the patch passed |
| +1 :green_heart: | compile | 20m 26s | | the patch passed with JDK
Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04 |
| +1 :green_heart: | javac | 20m 26s | | the patch passed |
| +1 :green_heart: | compile | 18m 29s | | the patch passed with JDK
Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10 |
| +1 :green_heart: | javac | 18m 29s | | the patch passed |
| +1 :green_heart: | blanks | 0m 0s | | The patch has no blanks
issues. |
| +1 :green_heart: | checkstyle | 3m 33s | | the patch passed |
| +1 :green_heart: | mvnsite | 3m 11s | | the patch passed |
| +1 :green_heart: | javadoc | 2m 17s | | the patch passed with JDK
Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04 |
| +1 :green_heart: | javadoc | 3m 28s | | the patch passed with JDK
Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10 |
| +1 :green_heart: | spotbugs | 6m 7s | | the patch passed |
| +1 :green_heart: | shadedclient | 22m 8s | | patch has no errors
when building and testing our client artifacts. |
_ Other Tests _ |
| +1 :green_heart: | unit | 17m 15s | | hadoop-common in the patch
passed. |
| +1 :green_heart: | unit | 225m 5s | | hadoop-hdfs in the patch
passed. |
| +1 :green_heart: | asflicense | 1m 10s | | The patch does not
generate ASF License warnings. |
| | | 449m 58s | | |
| Subsystem | Report/Notes |
|--:|:-|
| Docker | ClientAPI=1.41 ServerAPI=1.41 base:
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3494/1/artifact/out/Dockerfile
|
| GITHUB PR | https://github.com/apache/hadoop/pull/3494 |
| Optional Tests | dupname asflicense compile javac javadoc mvninstall
mvnsite unit shadedclient spotbugs checkstyle codespell |
| uname | Linux ec622a565571 4.15.0-156-generic #163-Ubuntu SMP Thu Aug 19
23:31:58 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/bin/hadoop.sh |
| git revision | trunk / 84f3331ed7d67d3310a1372451f93f96d57ceaad |
| Default Java | Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10 |
| Multi-JDK versions |
/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04
/usr/lib/jvm/java-8-openjdk-amd64:Private
Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10 |
| Test Results |
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3494/1/testReport/ |
| Max. process+thread count | 3106 (vs. ulimit of 5500) |
| modules | C: hadoop-common-project/hadoop-common
hadoop-hdfs-project/hadoop-hdfs U:
[jira] [Work logged] (HDFS-16238) Improve comments related to EncryptionZoneManager
[
https://issues.apache.org/jira/browse/HDFS-16238?focusedWorklogId=656279=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-656279
]
ASF GitHub Bot logged work on HDFS-16238:
-
Author: ASF GitHub Bot
Created on: 28/Sep/21 17:44
Start Date: 28/Sep/21 17:44
Worklog Time Spent: 10m
Work Description: virajjasani commented on pull request #3483:
URL: https://github.com/apache/hadoop/pull/3483#issuecomment-929483751
+1 (non-binding) from my side, thanks for the work @jianghuazhu. @ayushtkn
will be able to review this further.
Thanks
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 656279)
Time Spent: 1h 20m (was: 1h 10m)
> Improve comments related to EncryptionZoneManager
> -
>
> Key: HDFS-16238
> URL: https://issues.apache.org/jira/browse/HDFS-16238
> Project: Hadoop HDFS
> Issue Type: Improvement
> Components: documentation, encryption, namenode
>Reporter: JiangHua Zhu
>Assignee: JiangHua Zhu
>Priority: Minor
> Labels: pull-request-available
> Time Spent: 1h 20m
> Remaining Estimate: 0h
>
> In EncryptionZoneManager, there are some missing
> The description of the relevant comment. The purpose of this jira is to
> perfect them.
> E.g:
>/**
> * Re-encrypts the given encryption zone path. If the given path is not the
> * root of an encryption zone, an exception is thrown.
> * @param zoneIIP
> * @param keyVersionName
> * @throws IOException
> */
>List reencryptEncryptionZone(final INodesInPath zoneIIP,
>final String keyVersionName) throws IOException {
> ..
> }
> The description of zoneIIP and keyVersionName is missing here.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-15119) Allow expiration of cached locations in DFSInputStream
[ https://issues.apache.org/jira/browse/HDFS-15119?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17421490#comment-17421490 ] Bryan Beaudreault commented on HDFS-15119: -- Did anything come of the benchmarks [~ahussein]? Reading through, I agree it would be nice to have a mechanism for refreshing block locations. But in a low-latency usecase like HBase, ideally that would happen in the background, not in the critical path of a request. Alternatively, as mentioned above, one could refresh the locations only in response to certain exceptions. > Allow expiration of cached locations in DFSInputStream > -- > > Key: HDFS-15119 > URL: https://issues.apache.org/jira/browse/HDFS-15119 > Project: Hadoop HDFS > Issue Type: Improvement > Components: dfsclient >Reporter: Ahmed Hussein >Assignee: Ahmed Hussein >Priority: Minor > Fix For: 3.3.0, 3.1.4, 3.2.2 > > Attachments: HDFS-15119-branch-2.10.003.patch, HDFS-15119.001.patch, > HDFS-15119.002.patch, HDFS-15119.003.patch > > > Staleness and other transient conditions can affect reads for a long time > since the block locations may not be re-fetched. It makes sense to make > cached locations to expire. > For example, we may not take advantage of local-reads since the nodes are > blacklisted and have not been updated. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Work logged] (HDFS-16216) RBF: Wrong path when get mount point status
[
https://issues.apache.org/jira/browse/HDFS-16216?focusedWorklogId=656211=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-656211
]
ASF GitHub Bot logged work on HDFS-16216:
-
Author: ASF GitHub Bot
Created on: 28/Sep/21 15:35
Start Date: 28/Sep/21 15:35
Worklog Time Spent: 10m
Work Description: tomscut commented on a change in pull request #3415:
URL: https://github.com/apache/hadoop/pull/3415#discussion_r717704397
##
File path:
hadoop-hdfs-project/hadoop-hdfs-rbf/src/test/java/org/apache/hadoop/hdfs/server/federation/router/TestRouterMountTable.java
##
@@ -19,6 +19,7 @@
import static org.junit.Assert.assertEquals;
import static org.junit.Assert.assertFalse;
+import static org.junit.Assert.assertNull;
Review comment:
We should remove this unused import.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 656211)
Time Spent: 40m (was: 0.5h)
> RBF: Wrong path when get mount point status
> ---
>
> Key: HDFS-16216
> URL: https://issues.apache.org/jira/browse/HDFS-16216
> Project: Hadoop HDFS
> Issue Type: Bug
>Reporter: zhuobin zheng
>Assignee: zhuobin zheng
>Priority: Major
> Labels: pull-request-available
> Time Spent: 40m
> Remaining Estimate: 0h
>
> When we get mount point status, we will get wrong path in file status.
> {code:java}
> // prepare mount env
> hdfs dfsrouteradmin -add /test ns1 /test
> hdfs dfsrouteradmin -add /test/test1/test2/test3 ns1 /test/test1/test2/test3
> {code}
> {code:java}
> // java code
> public class TmpOp {
> public static void main(String[] args) throws Exception {
> Configuration conf = new HdfsConfiguration();
> FileSystem fs = FileSystem.get(conf);
> FileStatus test = fs.getFileStatus(new Path("/test"));
> FileStatus test1 = fs.getFileStatus(new Path("/test/test1"));
> FileStatus test2 = fs.getFileStatus(new Path("/test/test1/test2"));
> System.out.println(test.getPath());
> System.out.println(test1.getPath());
> System.out.println(test2.getPath());
> }
> }
> {code}
>
> {code:java}
> // result
> hdfs://ns1/test/test
> hdfs://ns1/test/test1/test1
> hdfs://ns1/test/test1/test2/test2
> {code}
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Updated] (HDFS-16243) The available disk space is less than the reserved space, and no log message is displayed
[ https://issues.apache.org/jira/browse/HDFS-16243?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hualong Zhang updated HDFS-16243: - Attachment: HDFS-16243.patch > The available disk space is less than the reserved space, and no log message > is displayed > - > > Key: HDFS-16243 > URL: https://issues.apache.org/jira/browse/HDFS-16243 > Project: Hadoop HDFS > Issue Type: Improvement > Components: datanode >Affects Versions: 2.7.2 >Reporter: Hualong Zhang >Priority: Major > Attachments: HDFS-16243.patch > > > When I submitted a task to the hadoop test cluster, it appeared "could only > be replicated to 0 nodes instead of minReplication (=1)" > I checked the namenode and datanode logs and did not find any error logs. It > was not until the use of dfsadmin -report that the available capacity was 0 > and I realized that it may be a configuration problem. > Checking the configuration found that the value of the > "dfs.datanode.du.reserved" configuration is greater than the available disk > space of HDFS, which caused this problem > It seems that there should be some warnings or errors in the log. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-12643) HDFS maintenance state behaviour is confusing and not well documented
[
https://issues.apache.org/jira/browse/HDFS-12643?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17421453#comment-17421453
]
Kihwal Lee commented on HDFS-12643:
---
Probably the missing information is that the cluster nodes need to be actively
managed using {{dfs.hosts}} in order to use the maintenance mode feature. It
was likely overlooked because most big organizations do use either the old or
new combined hosts file to manage cluster membership. For example,
decommissioning also requires the use of hosts file based cluster membership
management. At minimum, the documentation need to be updated.
> HDFS maintenance state behaviour is confusing and not well documented
> -
>
> Key: HDFS-12643
> URL: https://issues.apache.org/jira/browse/HDFS-12643
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: documentation, namenode
>Reporter: Andre Araujo
>Priority: Major
>
> The current implementation of the HDFS maintenance state feature is confusing
> and error-prone. The documentation is missing important information that's
> required for the correct use of the feature.
> For example, if the Hadoop admin wants to put a single node in maintenance
> state, he/she can add a single entry to the maintenance file with the
> contents:
> {code}
> {
>"hostName": "host-1.example.com",
>"adminState": "IN_MAINTENANCE",
>"maintenanceExpireTimeInMS": 1507663698000
> }
> {code}
> Let's say now that the actual maintenance finished well before the set
> expiration time and the Hadoop admin wants to bring the node back to NORMAL
> state. It would be natural to simply change the state of the node, as show
> below, and run another refresh:
> {code}
> {
>"hostName": "host-1.example.com",
>"adminState": "NORMAL"
> }
> {code}
> The configuration file above, though, not only take the node {{host-1}} out
> of maintenance state but it also *blacklists all the other DataNodes*. This
> behaviour seems inconsistent to me and is due to {{emptyInServiceNodeLists}}
> being set to {{false}}
> [here|https://github.com/apache/hadoop/blob/230b85d5865b7e08fb7aaeab45295b5b966011ef/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/blockmanagement/CombinedHostFileManager.java#L80]
> only when there is at least one node with {{adminState = NORMAL}} listed in
> the file.
> I believe that it would be more consistent, and less error prone, to simply
> implement the following:
> * If the dfs.hosts file is empty, all nodes are allowed and in normal state
> * If the file is not empty, any host *not* listed in the file is
> *blacklisted*, regardless of the state of the hosts listed in the file.
> Regardless of the implementation being changed or not, the documentation also
> needs to be updated to ensure the readers know of the caveats mentioned above.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Created] (HDFS-16243) The available disk space is less than the reserved space, and no log message is displayed
Hualong Zhang created HDFS-16243: Summary: The available disk space is less than the reserved space, and no log message is displayed Key: HDFS-16243 URL: https://issues.apache.org/jira/browse/HDFS-16243 Project: Hadoop HDFS Issue Type: Improvement Components: datanode Affects Versions: 2.7.2 Reporter: Hualong Zhang When I submitted a task to the hadoop test cluster, it appeared "could only be replicated to 0 nodes instead of minReplication (=1)" I checked the namenode and datanode logs and did not find any error logs. It was not until the use of dfsadmin -report that the available capacity was 0 and I realized that it may be a configuration problem. Checking the configuration found that the value of the "dfs.datanode.du.reserved" configuration is greater than the available disk space of HDFS, which caused this problem It seems that there should be some warnings or errors in the log. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Work logged] (HDFS-16155) Allow configurable exponential backoff in DFSInputStream refetchLocations
[ https://issues.apache.org/jira/browse/HDFS-16155?focusedWorklogId=656111=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-656111 ] ASF GitHub Bot logged work on HDFS-16155: - Author: ASF GitHub Bot Created on: 28/Sep/21 12:23 Start Date: 28/Sep/21 12:23 Worklog Time Spent: 10m Work Description: bbeaudreault commented on pull request #3271: URL: https://github.com/apache/hadoop/pull/3271#issuecomment-929150515 Any other comments on this patch? As tests demonstrate, it should have no impact on existing use-cases, aside from the requested backoff ceiling. It will enable operators to unlock faster retries if desired, and is much easier to read and test code. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 656111) Time Spent: 2.5h (was: 2h 20m) > Allow configurable exponential backoff in DFSInputStream refetchLocations > - > > Key: HDFS-16155 > URL: https://issues.apache.org/jira/browse/HDFS-16155 > Project: Hadoop HDFS > Issue Type: Improvement > Components: dfsclient >Reporter: Bryan Beaudreault >Assignee: Bryan Beaudreault >Priority: Minor > Labels: pull-request-available > Time Spent: 2.5h > Remaining Estimate: 0h > > The retry policy in > [DFSInputStream#refetchLocations|https://github.com/apache/hadoop/blob/trunk/hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/DFSInputStream.java#L1018-L1040] > was first written many years ago. It allows configuration of the base time > window, but subsequent retries double in an un-configurable way. This retry > strategy makes sense in some clusters as it's very conservative and will > avoid DDOSing the namenode in certain systemic failure modes – for example, > if a file is being read by a large hadoop job and the underlying blocks are > moved by the balancer. In this case, enough datanodes would be added to the > deadNodes list and all hadoop tasks would simultaneously try to refetch the > blocks. The 3s doubling with random factor helps break up that stampeding > herd. > However, not all cluster use-cases are created equal, so there are other > cases where a more aggressive initial backoff is preferred. For example in a > low-latency single reader scenario. In this case, if the balancer moves > enough blocks, the reader hits this 3s backoff which is way too long for a > low latency use-case. > One could configure the the window very low (10ms), but then you can hit > other systemic failure modes which would result in readers DDOSing the > namenode again. For example, if blocks went missing due to truly dead > datanodes. In this case, many readers might be refetching locations for > different files with retry backoffs like 10ms, 20ms, 40ms, etc. It takes a > while to backoff enough to avoid impacting the namenode with that strategy. > I suggest adding a configurable multiplier to the backoff strategy so that > operators can tune this as they see fit for their use-case. In the above low > latency case, one could set the base very low (say 2ms) and the multiplier > very high (say 50). This gives an aggressive first retry that very quickly > backs off. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Updated] (HDFS-16241) Standby close reconstruction thread
[ https://issues.apache.org/jira/browse/HDFS-16241?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated HDFS-16241: -- Labels: pull-request-available (was: ) > Standby close reconstruction thread > --- > > Key: HDFS-16241 > URL: https://issues.apache.org/jira/browse/HDFS-16241 > Project: Hadoop HDFS > Issue Type: Improvement >Reporter: zhanghuazong >Assignee: zhanghuazong >Priority: Major > Labels: pull-request-available > Attachments: HDFS-16241 > > Time Spent: 10m > Remaining Estimate: 0h > > When the "Reconstruction Queue Initializer" thread of the active namenode has > not stopped, switch to standby namenode. The "Reconstruction Queue > Initializer" thread should be closed -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Work logged] (HDFS-16241) Standby close reconstruction thread
[
https://issues.apache.org/jira/browse/HDFS-16241?focusedWorklogId=656108=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-656108
]
ASF GitHub Bot logged work on HDFS-16241:
-
Author: ASF GitHub Bot
Created on: 28/Sep/21 12:11
Start Date: 28/Sep/21 12:11
Worklog Time Spent: 10m
Work Description: hadoop-yetus commented on pull request #3493:
URL: https://github.com/apache/hadoop/pull/3493#issuecomment-929131317
:broken_heart: **-1 overall**
| Vote | Subsystem | Runtime | Logfile | Comment |
|::|--:|:|::|:---:|
| +0 :ok: | reexec | 0m 41s | | Docker mode activated. |
_ Prechecks _ |
| +1 :green_heart: | dupname | 0m 0s | | No case conflicting files
found. |
| +0 :ok: | codespell | 0m 1s | | codespell was not available. |
| +1 :green_heart: | @author | 0m 0s | | The patch does not contain
any @author tags. |
| -1 :x: | test4tests | 0m 0s | | The patch doesn't appear to include
any new or modified tests. Please justify why no new tests are needed for this
patch. Also please list what manual steps were performed to verify this patch.
|
_ trunk Compile Tests _ |
| +1 :green_heart: | mvninstall | 32m 11s | | trunk passed |
| +1 :green_heart: | compile | 1m 23s | | trunk passed with JDK
Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04 |
| +1 :green_heart: | compile | 1m 16s | | trunk passed with JDK
Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10 |
| +1 :green_heart: | checkstyle | 1m 2s | | trunk passed |
| +1 :green_heart: | mvnsite | 1m 22s | | trunk passed |
| +1 :green_heart: | javadoc | 0m 55s | | trunk passed with JDK
Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04 |
| +1 :green_heart: | javadoc | 1m 28s | | trunk passed with JDK
Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10 |
| +1 :green_heart: | spotbugs | 3m 6s | | trunk passed |
| +1 :green_heart: | shadedclient | 21m 17s | | branch has no errors
when building and testing our client artifacts. |
_ Patch Compile Tests _ |
| +1 :green_heart: | mvninstall | 1m 12s | | the patch passed |
| +1 :green_heart: | compile | 1m 13s | | the patch passed with JDK
Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04 |
| +1 :green_heart: | javac | 1m 13s | | the patch passed |
| +1 :green_heart: | compile | 1m 7s | | the patch passed with JDK
Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10 |
| +1 :green_heart: | javac | 1m 7s | | the patch passed |
| +1 :green_heart: | blanks | 0m 0s | | The patch has no blanks
issues. |
| +1 :green_heart: | checkstyle | 0m 52s | | the patch passed |
| +1 :green_heart: | mvnsite | 1m 17s | | the patch passed |
| +1 :green_heart: | javadoc | 0m 45s | | the patch passed with JDK
Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04 |
| +1 :green_heart: | javadoc | 1m 21s | | the patch passed with JDK
Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10 |
| +1 :green_heart: | spotbugs | 3m 8s | | the patch passed |
| +1 :green_heart: | shadedclient | 20m 47s | | patch has no errors
when building and testing our client artifacts. |
_ Other Tests _ |
| +1 :green_heart: | unit | 234m 13s | | hadoop-hdfs in the patch
passed. |
| +1 :green_heart: | asflicense | 0m 41s | | The patch does not
generate ASF License warnings. |
| | | 329m 23s | | |
| Subsystem | Report/Notes |
|--:|:-|
| Docker | ClientAPI=1.41 ServerAPI=1.41 base:
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3493/1/artifact/out/Dockerfile
|
| GITHUB PR | https://github.com/apache/hadoop/pull/3493 |
| Optional Tests | dupname asflicense compile javac javadoc mvninstall
mvnsite unit shadedclient spotbugs checkstyle codespell |
| uname | Linux 32505d960d16 4.15.0-156-generic #163-Ubuntu SMP Thu Aug 19
23:31:58 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/bin/hadoop.sh |
| git revision | trunk / 657d6e7b600a5361767eb9a8827a1a236823c788 |
| Default Java | Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10 |
| Multi-JDK versions |
/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04
/usr/lib/jvm/java-8-openjdk-amd64:Private
Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10 |
| Test Results |
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3493/1/testReport/ |
| Max. process+thread count | 3696 (vs. ulimit of 5500) |
| modules | C: hadoop-hdfs-project/hadoop-hdfs U:
hadoop-hdfs-project/hadoop-hdfs |
| Console output |
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3493/1/console |
| versions |
[jira] [Work logged] (HDFS-16242) JournalMetrics should add JournalId MetricTag to distinguish different nameservice journal metrics.
[
https://issues.apache.org/jira/browse/HDFS-16242?focusedWorklogId=656079=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-656079
]
ASF GitHub Bot logged work on HDFS-16242:
-
Author: ASF GitHub Bot
Created on: 28/Sep/21 10:33
Start Date: 28/Sep/21 10:33
Worklog Time Spent: 10m
Work Description: Neilxzn opened a new pull request #3494:
URL: https://github.com/apache/hadoop/pull/3494
### Description of PR
https://issues.apache.org/jira/browse/HDFS-16242
JournalMetrics should add JournalId MetricTag to distinguish different
nameservice journal metrics when JournalNode use ganglia sink or prometheus
sink.
JMX
Before this patch, journal' jmx :
// jmx json
{
"name" : "Hadoop:service=JournalNode,name=Journal-nntest1",
"Syncs60sNumOps" : 0,
...
},
{
"name" : "Hadoop:service=JournalNode,name=Journal-nntest2",
"Syncs60sNumOps" : 0,
...
}
After this patch, journal' jmx :
// jmx json
{
"name" : "Hadoop:service=JournalNode,name=Journal-nntest1",
"tag.JournalId" : "nntest1", // add this tag
"Syncs60sNumOps" : 0,
...
},
{
"name" : "Hadoop:service=JournalNode,name=Journal-nntest2",
"tag.JournalId" : "nntest2",
"Syncs60sNumOps" : 0,
...
}
PrometheusSink
Before this patch, journal' prometheus export :
journal_node_syncs60s_num_ops{context="dfs",hostname="host"} 2
After this patch, journal' prometheus export :
journal_node_syncs60s_num_ops{context="dfs",journalid="nntest2",hostname="host"}
2
journal_node_syncs60s_num_ops{context="dfs",journalid="nntest1",hostname="host"}
75
### How was this patch tested?
add test testJournalMetricTags
### For code changes:
add JournalMetrics$getJournalId
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 656079)
Remaining Estimate: 0h
Time Spent: 10m
> JournalMetrics should add JournalId MetricTag to distinguish different
> nameservice journal metrics.
>
>
> Key: HDFS-16242
> URL: https://issues.apache.org/jira/browse/HDFS-16242
> Project: Hadoop HDFS
> Issue Type: Improvement
> Components: journal-node
>Affects Versions: 3.4.0
>Reporter: Max Xie
>Priority: Minor
> Time Spent: 10m
> Remaining Estimate: 0h
>
> JournalMetrics should add JournalId MetricTag to distinguish different
> nameservice journal metrics when JournalNode use ganglia sink or prometheus
> sink.
> *JMX*
> Before this patch, journal' jmx :
> {code:java}
> // jmx json
> {
> "name" : "Hadoop:service=JournalNode,name=Journal-nntest1",
> "Syncs60sNumOps" : 0,
> ...
> },
> {
>"name" : "Hadoop:service=JournalNode,name=Journal-nntest2",
>"Syncs60sNumOps" : 0,
>...
>}
> {code}
>
> After this patch, journal' jmx :
>
> {code:java}
> // jmx json
> {
> "name" : "Hadoop:service=JournalNode,name=Journal-nntest1",
> "tag.JournalId" : "nntest1", // add this tag
> "Syncs60sNumOps" : 0,
>...
> },
> {
>"name" : "Hadoop:service=JournalNode,name=Journal-nntest2",
> "tag.JournalId" : "nntest2",
>"Syncs60sNumOps" : 0,
> ...
> }
> {code}
>
> *PrometheusSink*
> Before this patch, journal' prometheus export :
>
> {code:java}
> journal_node_syncs60s_num_ops{context="dfs",hostname="host"} 2
> {code}
> After this patch, journal' prometheus export :
> {code:java}
> journal_node_syncs60s_num_ops{context="dfs",journalid="nntest2",hostname="host"}
> 2
> journal_node_syncs60s_num_ops{context="dfs",journalid="nntest1",hostname="host"}
> 75
> {code}
>
>
>
>
>
>
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Updated] (HDFS-16242) JournalMetrics should add JournalId MetricTag to distinguish different nameservice journal metrics.
[
https://issues.apache.org/jira/browse/HDFS-16242?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
ASF GitHub Bot updated HDFS-16242:
--
Labels: pull-request-available (was: )
> JournalMetrics should add JournalId MetricTag to distinguish different
> nameservice journal metrics.
>
>
> Key: HDFS-16242
> URL: https://issues.apache.org/jira/browse/HDFS-16242
> Project: Hadoop HDFS
> Issue Type: Improvement
> Components: journal-node
>Affects Versions: 3.4.0
>Reporter: Max Xie
>Priority: Minor
> Labels: pull-request-available
> Time Spent: 10m
> Remaining Estimate: 0h
>
> JournalMetrics should add JournalId MetricTag to distinguish different
> nameservice journal metrics when JournalNode use ganglia sink or prometheus
> sink.
> *JMX*
> Before this patch, journal' jmx :
> {code:java}
> // jmx json
> {
> "name" : "Hadoop:service=JournalNode,name=Journal-nntest1",
> "Syncs60sNumOps" : 0,
> ...
> },
> {
>"name" : "Hadoop:service=JournalNode,name=Journal-nntest2",
>"Syncs60sNumOps" : 0,
>...
>}
> {code}
>
> After this patch, journal' jmx :
>
> {code:java}
> // jmx json
> {
> "name" : "Hadoop:service=JournalNode,name=Journal-nntest1",
> "tag.JournalId" : "nntest1", // add this tag
> "Syncs60sNumOps" : 0,
>...
> },
> {
>"name" : "Hadoop:service=JournalNode,name=Journal-nntest2",
> "tag.JournalId" : "nntest2",
>"Syncs60sNumOps" : 0,
> ...
> }
> {code}
>
> *PrometheusSink*
> Before this patch, journal' prometheus export :
>
> {code:java}
> journal_node_syncs60s_num_ops{context="dfs",hostname="host"} 2
> {code}
> After this patch, journal' prometheus export :
> {code:java}
> journal_node_syncs60s_num_ops{context="dfs",journalid="nntest2",hostname="host"}
> 2
> journal_node_syncs60s_num_ops{context="dfs",journalid="nntest1",hostname="host"}
> 75
> {code}
>
>
>
>
>
>
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Created] (HDFS-16242) JournalMetrics should add JournalId MetricTag to distinguish different nameservice journal metrics.
Max Xie created HDFS-16242:
---
Summary: JournalMetrics should add JournalId MetricTag to
distinguish different nameservice journal metrics.
Key: HDFS-16242
URL: https://issues.apache.org/jira/browse/HDFS-16242
Project: Hadoop HDFS
Issue Type: Improvement
Components: journal-node
Affects Versions: 3.4.0
Reporter: Max Xie
JournalMetrics should add JournalId MetricTag to distinguish different
nameservice journal metrics when JournalNode use ganglia sink or prometheus
sink.
*JMX*
Before this patch, journal' jmx :
{code:java}
// jmx json
{
"name" : "Hadoop:service=JournalNode,name=Journal-nntest1",
"Syncs60sNumOps" : 0,
...
},
{
"name" : "Hadoop:service=JournalNode,name=Journal-nntest2",
"Syncs60sNumOps" : 0,
...
}
{code}
After this patch, journal' jmx :
{code:java}
// jmx json
{
"name" : "Hadoop:service=JournalNode,name=Journal-nntest1",
"tag.JournalId" : "nntest1", // add this tag
"Syncs60sNumOps" : 0,
...
},
{
"name" : "Hadoop:service=JournalNode,name=Journal-nntest2",
"tag.JournalId" : "nntest2",
"Syncs60sNumOps" : 0,
...
}
{code}
*PrometheusSink*
Before this patch, journal' prometheus export :
{code:java}
journal_node_syncs60s_num_ops{context="dfs",hostname="host"} 2
{code}
After this patch, journal' prometheus export :
{code:java}
journal_node_syncs60s_num_ops{context="dfs",journalid="nntest2",hostname="host"}
2
journal_node_syncs60s_num_ops{context="dfs",journalid="nntest1",hostname="host"}
75
{code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-12643) HDFS maintenance state behaviour is confusing and not well documented
[
https://issues.apache.org/jira/browse/HDFS-12643?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17421255#comment-17421255
]
TisNotT commented on HDFS-12643:
it is confuse to me too.So I am finding whether there is an api to set special
datanodes to maintenance state or not. If not,maybe I will dev one for my
company.
> HDFS maintenance state behaviour is confusing and not well documented
> -
>
> Key: HDFS-12643
> URL: https://issues.apache.org/jira/browse/HDFS-12643
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: documentation, namenode
>Reporter: Andre Araujo
>Priority: Major
>
> The current implementation of the HDFS maintenance state feature is confusing
> and error-prone. The documentation is missing important information that's
> required for the correct use of the feature.
> For example, if the Hadoop admin wants to put a single node in maintenance
> state, he/she can add a single entry to the maintenance file with the
> contents:
> {code}
> {
>"hostName": "host-1.example.com",
>"adminState": "IN_MAINTENANCE",
>"maintenanceExpireTimeInMS": 1507663698000
> }
> {code}
> Let's say now that the actual maintenance finished well before the set
> expiration time and the Hadoop admin wants to bring the node back to NORMAL
> state. It would be natural to simply change the state of the node, as show
> below, and run another refresh:
> {code}
> {
>"hostName": "host-1.example.com",
>"adminState": "NORMAL"
> }
> {code}
> The configuration file above, though, not only take the node {{host-1}} out
> of maintenance state but it also *blacklists all the other DataNodes*. This
> behaviour seems inconsistent to me and is due to {{emptyInServiceNodeLists}}
> being set to {{false}}
> [here|https://github.com/apache/hadoop/blob/230b85d5865b7e08fb7aaeab45295b5b966011ef/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/blockmanagement/CombinedHostFileManager.java#L80]
> only when there is at least one node with {{adminState = NORMAL}} listed in
> the file.
> I believe that it would be more consistent, and less error prone, to simply
> implement the following:
> * If the dfs.hosts file is empty, all nodes are allowed and in normal state
> * If the file is not empty, any host *not* listed in the file is
> *blacklisted*, regardless of the state of the hosts listed in the file.
> Regardless of the implementation being changed or not, the documentation also
> needs to be updated to ensure the readers know of the caveats mentioned above.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-16233) Do not use exception handler to implement copy-on-write for EnumCounters
[ https://issues.apache.org/jira/browse/HDFS-16233?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17421220#comment-17421220 ] Akira Ajisaka commented on HDFS-16233: -- Backported to branch-3.2.3. > Do not use exception handler to implement copy-on-write for EnumCounters > > > Key: HDFS-16233 > URL: https://issues.apache.org/jira/browse/HDFS-16233 > Project: Hadoop HDFS > Issue Type: Bug > Components: namenode >Reporter: Wei-Chiu Chuang >Assignee: Wei-Chiu Chuang >Priority: Major > Labels: pull-request-available > Fix For: 3.4.0, 2.10.2, 3.2.3, 3.3.2, 3.1.5 > > Attachments: Screen Shot 2021-09-22 at 1.59.59 PM.png, > profile_c7_delete_asyncaudit.html > > Time Spent: 1h 10m > Remaining Estimate: 0h > > HDFS-14547 saves the NameNode heap space occupied by EnumCounters by > essentially implementing a copy-on-write strategy. > At beginning, all EnumCounters refers to the same ConstEnumCounters to save > heap space. When it is modified, an exception is thrown and the exception > handler converts ConstEnumCounters to EnumCounters object and updates it. > Using exception handler to perform anything more than occasional is bad for > performance. > Propose: use instanceof keyword to detect the type of object and do COW > accordingly. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Updated] (HDFS-16241) Standby close reconstruction thread
[ https://issues.apache.org/jira/browse/HDFS-16241?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] zhanghuazong updated HDFS-16241: Attachment: HDFS-16241 > Standby close reconstruction thread > --- > > Key: HDFS-16241 > URL: https://issues.apache.org/jira/browse/HDFS-16241 > Project: Hadoop HDFS > Issue Type: Improvement >Reporter: zhanghuazong >Assignee: zhanghuazong >Priority: Major > Attachments: HDFS-16241 > > > When the "Reconstruction Queue Initializer" thread of the active namenode has > not stopped, switch to standby namenode. The "Reconstruction Queue > Initializer" thread should be closed -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Updated] (HDFS-16241) Standby close reconstruction thread
[ https://issues.apache.org/jira/browse/HDFS-16241?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] zhanghuazong updated HDFS-16241: Summary: Standby close reconstruction thread (was: Standby closes reconstruction thread) > Standby close reconstruction thread > --- > > Key: HDFS-16241 > URL: https://issues.apache.org/jira/browse/HDFS-16241 > Project: Hadoop HDFS > Issue Type: Improvement >Reporter: zhanghuazong >Assignee: zhanghuazong >Priority: Major > > When the "Reconstruction Queue Initializer" thread of the active namenode has > not stopped, switch to standby namenode. The "Reconstruction Queue > Initializer" thread should be closed -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Updated] (HDFS-16241) Standby closes reconstruction thread
[ https://issues.apache.org/jira/browse/HDFS-16241?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] zhanghuazong updated HDFS-16241: Description: When the "Reconstruction Queue Initializer" thread of the active namenode has not stopped, switch to standby namenode. The "Reconstruction Queue Initializer" thread should be closed (was: Standby closes reconstruction thread) > Standby closes reconstruction thread > > > Key: HDFS-16241 > URL: https://issues.apache.org/jira/browse/HDFS-16241 > Project: Hadoop HDFS > Issue Type: Improvement >Reporter: zhanghuazong >Assignee: zhanghuazong >Priority: Major > > When the "Reconstruction Queue Initializer" thread of the active namenode has > not stopped, switch to standby namenode. The "Reconstruction Queue > Initializer" thread should be closed -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Created] (HDFS-16241) Standby closes reconstruction thread
zhanghuazong created HDFS-16241: --- Summary: Standby closes reconstruction thread Key: HDFS-16241 URL: https://issues.apache.org/jira/browse/HDFS-16241 Project: Hadoop HDFS Issue Type: Improvement Reporter: zhanghuazong Assignee: zhanghuazong Standby closes reconstruction thread -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
