[jira] [Updated] (HDFS-10822) Log DataNodes in the write pipeline

2016-09-02 Thread Andrew Wang (JIRA) 

 [ 
https://issues.apache.org/jira/browse/HDFS-10822?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Andrew Wang updated HDFS-10822:
---
Fix Version/s: (was: 3.0.0-beta1)
   3.0.0-alpha2

> Log DataNodes in the write pipeline
> ---
>
> Key: HDFS-10822
> URL: https://issues.apache.org/jira/browse/HDFS-10822
> Project: Hadoop HDFS
>  Issue Type: Improvement
>  Components: hdfs-client
>Affects Versions: 2.6.0
>Reporter: John Zhuge
>Assignee: John Zhuge
>Priority: Trivial
>  Labels: supportability
> Fix For: 2.9.0, 3.0.0-alpha2
>
> Attachments: HDFS-10822.001.patch
>
>
> Trying to diagnose a slow HDFS flush, taking longer than 10 seconds, but did 
> not know which DNs were involved in the write pipeline. Of course, I could 
> search NN log for the list of DNs, but it might not be possible sometimes or 
> convenient.
> Propose to add a DEBUG trace to DataStreamer#setPipeline to print the list of 
> DNs in the pipeline.
> BTW, we do print the list of DNs during pipeline recovery.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org

[jira] [Updated] (HDFS-10822) Log DataNodes in the write pipeline

2016-09-02 Thread Lei (Eddy) Xu (JIRA) 

 [ 
https://issues.apache.org/jira/browse/HDFS-10822?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Lei (Eddy) Xu updated HDFS-10822:
-
   Resolution: Fixed
Fix Version/s: 3.0.0-beta1
   2.9.0
   Status: Resolved  (was: Patch Available)

> Log DataNodes in the write pipeline
> ---
>
> Key: HDFS-10822
> URL: https://issues.apache.org/jira/browse/HDFS-10822
> Project: Hadoop HDFS
>  Issue Type: Improvement
>  Components: hdfs-client
>Affects Versions: 2.6.0
>Reporter: John Zhuge
>Assignee: John Zhuge
>Priority: Trivial
>  Labels: supportability
> Fix For: 2.9.0, 3.0.0-beta1
>
> Attachments: HDFS-10822.001.patch
>
>
> Trying to diagnose a slow HDFS flush, taking longer than 10 seconds, but did 
> not know which DNs were involved in the write pipeline. Of course, I could 
> search NN log for the list of DNs, but it might not be possible sometimes or 
> convenient.
> Propose to add a DEBUG trace to DataStreamer#setPipeline to print the list of 
> DNs in the pipeline.
> BTW, we do print the list of DNs during pipeline recovery.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org

[jira] [Updated] (HDFS-10822) Log DataNodes in the write pipeline

2016-09-01 Thread John Zhuge (JIRA) 

 [ 
https://issues.apache.org/jira/browse/HDFS-10822?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

John Zhuge updated HDFS-10822:
--
Status: Patch Available  (was: Open)

> Log DataNodes in the write pipeline
> ---
>
> Key: HDFS-10822
> URL: https://issues.apache.org/jira/browse/HDFS-10822
> Project: Hadoop HDFS
>  Issue Type: Improvement
>  Components: hdfs-client
>Affects Versions: 2.6.0
>Reporter: John Zhuge
>Assignee: John Zhuge
>Priority: Trivial
>  Labels: supportability
> Attachments: HDFS-10822.001.patch
>
>
> Trying to diagnose a slow HDFS flush, taking longer than 10 seconds, but did 
> not know which DNs were involved in the write pipeline. Of course, I could 
> search NN log for the list of DNs, but it might not be possible sometimes or 
> convenient.
> Propose to add a DEBUG trace to DataStreamer#setPipeline to print the list of 
> DNs in the pipeline.
> BTW, we do print the list of DNs during pipeline recovery.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org

[jira] [Updated] (HDFS-10822) Log DataNodes in the write pipeline

2016-09-01 Thread John Zhuge (JIRA) 

 [ 
https://issues.apache.org/jira/browse/HDFS-10822?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

John Zhuge updated HDFS-10822:
--
Attachment: HDFS-10822.001.patch

Patch 001:
* {{initDataStreaming}} is a better place to add DEBUG msg because the log msg 
will have {{file}}, {{block}}, and {{pipeline}} all together.
* No unit test because the patch adds DEBUG msg only.

{{TestDataStream}} log that shows the new DEBUG msg:
{noformat}
2016-09-01 11:54:18,470 [DataStreamer for file /file1 block 
BP-1484500550-172.16.1.255-1472756056162:blk_1073741825_1001] DEBUG 
hdfs.DataStreamer (DataStreamer.java:initDataStreaming(512)) - nodes 
[DatanodeInfoWithStorage[127.0.0.1:61765,DS-124a1018-6033-4f81-bc43-cba740bd9538,DISK]]
 storageTypes [DISK] storageIDs [DS-124a1018-6033-4f81-bc43-cba740bd9538]
{noformat}

> Log DataNodes in the write pipeline
> ---
>
> Key: HDFS-10822
> URL: https://issues.apache.org/jira/browse/HDFS-10822
> Project: Hadoop HDFS
>  Issue Type: Improvement
>  Components: hdfs-client
>Affects Versions: 2.6.0
>Reporter: John Zhuge
>Assignee: John Zhuge
>Priority: Trivial
>  Labels: supportability
> Attachments: HDFS-10822.001.patch
>
>
> Trying to diagnose a slow HDFS flush, taking longer than 10 seconds, but did 
> not know which DNs were involved in the write pipeline. Of course, I could 
> search NN log for the list of DNs, but it might not be possible sometimes or 
> convenient.
> Propose to add a DEBUG trace to DataStreamer#setPipeline to print the list of 
> DNs in the pipeline.
> BTW, we do print the list of DNs during pipeline recovery.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org