[jira] [Commented] (HDFS-9901) Move disk IO out of the heartbeat thread

[ https://issues.apache.org/jira/browse/HDFS-9901?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15212107#comment-15212107 ] Hua Liu commented on HDFS-9901: --- Added comments as [~elgoiri] suggested. [~arpitagarwal], would you please

[jira] [Updated] (HDFS-9901) Move disk IO out of the heartbeat thread

[ https://issues.apache.org/jira/browse/HDFS-9901?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hua Liu updated HDFS-9901: -- Attachment: 0005-HDFS-9901-Move-diskIO-out-of-heartbeat-thread.patch > Move disk IO out of the heartbeat thread

[jira] [Updated] (HDFS-9901) Move disk IO out of the heartbeat thread

[ https://issues.apache.org/jira/browse/HDFS-9901?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hua Liu updated HDFS-9901: -- Status: Open (was: Patch Available) > Move disk IO out of the heartbeat thread >

[jira] [Updated] (HDFS-9901) Move disk IO out of the heartbeat thread

[ https://issues.apache.org/jira/browse/HDFS-9901?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hua Liu updated HDFS-9901: -- Status: Patch Available (was: Open) > Move disk IO out of the heartbeat thread >

[jira] [Updated] (HDFS-9901) Move disk IO out of the heartbeat thread

[ https://issues.apache.org/jira/browse/HDFS-9901?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hua Liu updated HDFS-9901: -- Attachment: 0004-HDFS-9901-move-diskIO-out-of-the-heartbeat-thread.patch > Move disk IO out of the heartbeat

[jira] [Commented] (HDFS-9901) Move disk IO out of the heartbeat thread

[ https://issues.apache.org/jira/browse/HDFS-9901?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15194414#comment-15194414 ] Hua Liu commented on HDFS-9901: --- Added comments for DFRefreshThread and DataCheckAndTransfer. > Move disk IO

[jira] [Updated] (HDFS-9901) Move disk IO out of the heartbeat thread

[ https://issues.apache.org/jira/browse/HDFS-9901?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hua Liu updated HDFS-9901: -- Status: Patch Available (was: Open) > Move disk IO out of the heartbeat thread >

[jira] [Updated] (HDFS-9901) Move disk IO out of the heartbeat thread

[ https://issues.apache.org/jira/browse/HDFS-9901?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hua Liu updated HDFS-9901: -- Status: Open (was: Patch Available) > Move disk IO out of the heartbeat thread >

[jira] [Updated] (HDFS-9901) Move disk IO out of the heartbeat thread

[ https://issues.apache.org/jira/browse/HDFS-9901?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hua Liu updated HDFS-9901: -- Status: Patch Available (was: Open) > Move disk IO out of the heartbeat thread >

[jira] [Updated] (HDFS-9901) Move disk IO out of the heartbeat thread

[ https://issues.apache.org/jira/browse/HDFS-9901?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hua Liu updated HDFS-9901: -- Attachment: 0003-HDFS-9901-Move-disk-IO-out-of-the-heartbeat-thread.patch > Move disk IO out of the heartbeat

[jira] [Updated] (HDFS-9901) Move disk IO out of the heartbeat thread

[ https://issues.apache.org/jira/browse/HDFS-9901?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hua Liu updated HDFS-9901: -- Status: Open (was: Patch Available) > Move disk IO out of the heartbeat thread >

[jira] [Updated] (HDFS-9901) Move disk IO out of the heartbeat thread

[ https://issues.apache.org/jira/browse/HDFS-9901?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hua Liu updated HDFS-9901: -- Summary: Move disk IO out of the heartbeat thread (was: Move block validation out of the heartbeat thread) >

[jira] [Updated] (HDFS-9901) Move block validation out of the heartbeat thread

[ https://issues.apache.org/jira/browse/HDFS-9901?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hua Liu updated HDFS-9901: -- Status: Patch Available (was: Open) > Move block validation out of the heartbeat thread >

[jira] [Updated] (HDFS-9901) Move block validation out of the heartbeat thread

[ https://issues.apache.org/jira/browse/HDFS-9901?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hua Liu updated HDFS-9901: -- Attachment: 0002-HDFS-9901-Move-block-validation-out-of-the-heartbeat.patch > Move block validation out of the

[jira] [Updated] (HDFS-9901) Move block validation out of the heartbeat thread

[ https://issues.apache.org/jira/browse/HDFS-9901?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hua Liu updated HDFS-9901: -- Status: Open (was: Patch Available) > Move block validation out of the heartbeat thread >

[jira] [Commented] (HDFS-9901) Move block validation out of the heartbeat thread

[ https://issues.apache.org/jira/browse/HDFS-9901?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15188664#comment-15188664 ] Hua Liu commented on HDFS-9901: --- Hi [~elgoiri], thanks for helping explain the approach. I've added it to the

[jira] [Commented] (HDFS-9901) Move block validation out of the heartbeat thread

[ https://issues.apache.org/jira/browse/HDFS-9901?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15188663#comment-15188663 ] Hua Liu commented on HDFS-9901: --- Hi [~arpiagariu], I extended the "Description" section with some detailed

[jira] [Updated] (HDFS-9901) Move block validation out of the heartbeat thread

[ https://issues.apache.org/jira/browse/HDFS-9901?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hua Liu updated HDFS-9901: -- Description: During heavy disk IO, we noticed hearbeat thread hangs on checkBlock method, which checks the

[jira] [Updated] (HDFS-9901) Move block validation out of the heartbeat thread

[ https://issues.apache.org/jira/browse/HDFS-9901?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hua Liu updated HDFS-9901: -- Description: During heavy disk IO, we noticed hearbeat thread hangs on checkBlock method, which checks the

[jira] [Updated] (HDFS-9901) Move block validation out of the heartbeat thread

[ https://issues.apache.org/jira/browse/HDFS-9901?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hua Liu updated HDFS-9901: -- Description: During heavy disk IO, we noticed hearbeat thread hangs on checkBlock method, which checks the

[jira] [Updated] (HDFS-9901) Move block validation out of the heartbeat thread

[ https://issues.apache.org/jira/browse/HDFS-9901?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hua Liu updated HDFS-9901: -- Status: Patch Available (was: In Progress) > Move block validation out of the heartbeat thread >

[jira] [Updated] (HDFS-9901) Move block validation out of the heartbeat thread

[ https://issues.apache.org/jira/browse/HDFS-9901?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hua Liu updated HDFS-9901: -- Attachment: 0001-HDFS-9901-Move-block-validation-out-of-the-heartbeat.patch > Move block validation out of the

[jira] [Updated] (HDFS-9901) Move block validation out of the heartbeat thread

[ https://issues.apache.org/jira/browse/HDFS-9901?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hua Liu updated HDFS-9901: -- Priority: Major (was: Minor) > Move block validation out of the heartbeat thread >

[jira] [Commented] (HDFS-9882) Add heartbeatsTotal in Datanode metrics

[ https://issues.apache.org/jira/browse/HDFS-9882?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15184312#comment-15184312 ] Hua Liu commented on HDFS-9882: --- Hi [~arpiagariu] I submitted the V4 patch a few hours ago but seems jenkins

[jira] [Updated] (HDFS-9882) Add heartbeatsTotal in Datanode metrics

[ https://issues.apache.org/jira/browse/HDFS-9882?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hua Liu updated HDFS-9882: -- Status: Patch Available (was: Open) > Add heartbeatsTotal in Datanode metrics >

[jira] [Updated] (HDFS-9882) Add heartbeatsTotal in Datanode metrics

[ https://issues.apache.org/jira/browse/HDFS-9882?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hua Liu updated HDFS-9882: -- Status: Open (was: Patch Available) > Add heartbeatsTotal in Datanode metrics >

[jira] [Updated] (HDFS-9882) Add heartbeatsTotal in Datanode metrics

[ https://issues.apache.org/jira/browse/HDFS-9882?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hua Liu updated HDFS-9882: -- Status: Patch Available (was: Open) > Add heartbeatsTotal in Datanode metrics >

[jira] [Updated] (HDFS-9882) Add heartbeatsTotal in Datanode metrics

[ https://issues.apache.org/jira/browse/HDFS-9882?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hua Liu updated HDFS-9882: -- Attachment: 0004-HDFS-9882.Add-heartbeatsTotal-in-Datanode-metrics.patch > Add heartbeatsTotal in Datanode

[jira] [Updated] (HDFS-9882) Add heartbeatsTotal in Datanode metrics

[ https://issues.apache.org/jira/browse/HDFS-9882?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hua Liu updated HDFS-9882: -- Status: Open (was: Patch Available) > Add heartbeatsTotal in Datanode metrics >

[jira] [Commented] (HDFS-9882) Add heartbeatsTotal in Datanode metrics

[ https://issues.apache.org/jira/browse/HDFS-9882?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15182559#comment-15182559 ] Hua Liu commented on HDFS-9882: --- Hi [~arpiagariu] Since NumOps and AvgTime are appended to the metric name,

[jira] [Updated] (HDFS-9882) Add heartbeatsTotal in Datanode metrics

[ https://issues.apache.org/jira/browse/HDFS-9882?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hua Liu updated HDFS-9882: -- Status: Patch Available (was: Open) > Add heartbeatsTotal in Datanode metrics >

[jira] [Updated] (HDFS-9882) Add heartbeatsTotal in Datanode metrics

[ https://issues.apache.org/jira/browse/HDFS-9882?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hua Liu updated HDFS-9882: -- Status: Open (was: Patch Available) > Add heartbeatsTotal in Datanode metrics >

[jira] [Updated] (HDFS-9882) Add heartbeatsTotal in Datanode metrics

[ https://issues.apache.org/jira/browse/HDFS-9882?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hua Liu updated HDFS-9882: -- Attachment: 0003-HDFS-9882.Add-heartbeatsTotal-in-Datanode-metrics.patch > Add heartbeatsTotal in Datanode

[jira] [Work started] (HDFS-9901) Move block validation out of the heartbeat thread

[ https://issues.apache.org/jira/browse/HDFS-9901?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Work on HDFS-9901 started by Hua Liu. - > Move block validation out of the heartbeat thread >

[jira] [Created] (HDFS-9901) Move block validation out of the heartbeat thread

Hua Liu created HDFS-9901: - Summary: Move block validation out of the heartbeat thread Key: HDFS-9901 URL: https://issues.apache.org/jira/browse/HDFS-9901 Project: Hadoop HDFS Issue Type:

[jira] [Commented] (HDFS-9882) Add heartbeatsTotal in Datanode metrics

[ https://issues.apache.org/jira/browse/HDFS-9882?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15179335#comment-15179335 ] Hua Liu commented on HDFS-9882: --- Hi [~arpiagariu] When a data node needs to transfer a block, it validates

[jira] [Updated] (HDFS-9882) Add heartbeatsTotal in Datanode metrics

[ https://issues.apache.org/jira/browse/HDFS-9882?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hua Liu updated HDFS-9882: -- Status: Patch Available (was: Open) > Add heartbeatsTotal in Datanode metrics >

[jira] [Work stopped] (HDFS-9882) Add heartbeatsTotal in Datanode metrics

[ https://issues.apache.org/jira/browse/HDFS-9882?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Work on HDFS-9882 stopped by Hua Liu. - > Add heartbeatsTotal in Datanode metrics > --- > >

[jira] [Updated] (HDFS-9882) Add heartbeatsTotal in Datanode metrics

[ https://issues.apache.org/jira/browse/HDFS-9882?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hua Liu updated HDFS-9882: -- Status: In Progress (was: Patch Available) > Add heartbeatsTotal in Datanode metrics >

[jira] [Updated] (HDFS-9882) Add heartbeatsTotal in Datanode metrics

[ https://issues.apache.org/jira/browse/HDFS-9882?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hua Liu updated HDFS-9882: -- Attachment: 0002-HDFS-9882.Add-heartbeatsTotal-in-Datanode-metrics.patch > Add heartbeatsTotal in Datanode

[jira] [Updated] (HDFS-9882) Add heartbeatsTotal in Datanode metrics

[ https://issues.apache.org/jira/browse/HDFS-9882?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hua Liu updated HDFS-9882: -- Status: Patch Available (was: Open) > Add heartbeatsTotal in Datanode metrics >

[jira] [Updated] (HDFS-9882) Add heartbeatsTotal in Datanode metrics

[ https://issues.apache.org/jira/browse/HDFS-9882?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hua Liu updated HDFS-9882: -- Status: Open (was: Patch Available) > Add heartbeatsTotal in Datanode metrics >

[jira] [Updated] (HDFS-9882) Add heartbeatsTotal in Datanode metrics

[ https://issues.apache.org/jira/browse/HDFS-9882?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hua Liu updated HDFS-9882: -- Status: Patch Available (was: Open) > Add heartbeatsTotal in Datanode metrics >

[jira] [Work stopped] (HDFS-9882) Add heartbeatsTotal in Datanode metrics

[ https://issues.apache.org/jira/browse/HDFS-9882?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Work on HDFS-9882 stopped by Hua Liu. - > Add heartbeatsTotal in Datanode metrics > --- > >

[jira] [Updated] (HDFS-9882) Add heartbeatsTotal in Datanode metrics

[ https://issues.apache.org/jira/browse/HDFS-9882?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hua Liu updated HDFS-9882: -- Attachment: 0001-HDFS-9882.Add-heartbeatsTotal-in-Datanode-metrics.patch > Add heartbeatsTotal in Datanode

[jira] [Updated] (HDFS-9882) Add heartbeatsTotal in Datanode metrics

[ https://issues.apache.org/jira/browse/HDFS-9882?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hua Liu updated HDFS-9882: -- Attachment: (was: 0001-Add-heartbeatsTotal-metric.patch) > Add heartbeatsTotal in Datanode metrics >

[jira] [Updated] (HDFS-9882) Add heartbeatsTotal in Datanode metrics

[ https://issues.apache.org/jira/browse/HDFS-9882?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hua Liu updated HDFS-9882: -- Status: Patch Available (was: Open) > Add heartbeatsTotal in Datanode metrics >

[jira] [Updated] (HDFS-9882) Add heartbeatsTotal in Datanode metrics

[ https://issues.apache.org/jira/browse/HDFS-9882?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hua Liu updated HDFS-9882: -- Status: Open (was: Patch Available) > Add heartbeatsTotal in Datanode metrics >

[jira] [Updated] (HDFS-9882) Add heartbeatsTotal in Datanode metrics

[ https://issues.apache.org/jira/browse/HDFS-9882?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hua Liu updated HDFS-9882: -- Status: Patch Available (was: In Progress) > Add heartbeatsTotal in Datanode metrics >

[jira] [Work started] (HDFS-9882) Add heartbeatsTotal in Datanode metrics

[ https://issues.apache.org/jira/browse/HDFS-9882?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Work on HDFS-9882 started by Hua Liu. - > Add heartbeatsTotal in Datanode metrics > --- > >

[jira] [Reopened] (HDFS-9882) Add heartbeatsTotal in Datanode metrics

[ https://issues.apache.org/jira/browse/HDFS-9882?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hua Liu reopened HDFS-9882: --- > Add heartbeatsTotal in Datanode metrics > --- > > Key:

[jira] [Resolved] (HDFS-9882) Add heartbeatsTotal in Datanode metrics

[ https://issues.apache.org/jira/browse/HDFS-9882?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hua Liu resolved HDFS-9882. --- Resolution: Fixed > Add heartbeatsTotal in Datanode metrics > --- > >

[jira] [Updated] (HDFS-9882) Add heartbeatsTotal in Datanode metrics

[ https://issues.apache.org/jira/browse/HDFS-9882?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hua Liu updated HDFS-9882: -- Attachment: 0001-Add-heartbeatsTotal-metric.patch > Add heartbeatsTotal in Datanode metrics >

[jira] [Updated] (HDFS-9882) Add heartbeatsTotal in Datanode metrics

[ https://issues.apache.org/jira/browse/HDFS-9882?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hua Liu updated HDFS-9882: -- Description: Heartbeat latency only reflects the time spent on generating reports and sending reports to NN.

[jira] [Updated] (HDFS-9882) Add heartbeatsTotal in Datanode metrics

[ https://issues.apache.org/jira/browse/HDFS-9882?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hua Liu updated HDFS-9882: -- Status: Open (was: Patch Available) > Add heartbeatsTotal in Datanode metrics >

[jira] [Updated] (HDFS-9882) Add heartbeatsTotal in Datanode metrics

[ https://issues.apache.org/jira/browse/HDFS-9882?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hua Liu updated HDFS-9882: -- Status: Patch Available (was: In Progress) > Add heartbeatsTotal in Datanode metrics >

[jira] [Updated] (HDFS-9882) Add heartbeatsTotal in Datanode metrics

[ https://issues.apache.org/jira/browse/HDFS-9882?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hua Liu updated HDFS-9882: -- Summary: Add heartbeatsTotal in Datanode metrics (was: Change the meaning of heartbeat latency in Datanode

[jira] [Work started] (HDFS-9882) Change the meaning of heartbeat latency

[ https://issues.apache.org/jira/browse/HDFS-9882?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Work on HDFS-9882 started by Hua Liu. - > Change the meaning of heartbeat latency > --- > >

[jira] [Created] (HDFS-9882) Change the meaning of heartbeat latency

Hua Liu created HDFS-9882: - Summary: Change the meaning of heartbeat latency Key: HDFS-9882 URL: https://issues.apache.org/jira/browse/HDFS-9882 Project: Hadoop HDFS Issue Type: Task