[jira] [Work started] (HDFS-15719) [Hadoop 3] Both NameNodes can crash simultaneously due to the short JN socket timeout

2021-01-04 Thread Wei-Chiu Chuang (Jira)
[ https://issues.apache.org/jira/browse/HDFS-15719?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Work on HDFS-15719 started by Wei-Chiu Chuang. -- > [Hadoop 3] Both NameNodes can crash simultaneously due to the short JN

[jira] [Work logged] (HDFS-15624) Fix the SetQuotaByStorageTypeOp problem after updating hadoop

2021-01-04 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HDFS-15624?focusedWorklogId=531041=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-531041 ] ASF GitHub Bot logged work on HDFS-15624: - Author: ASF GitHub Bot

[jira] [Work logged] (HDFS-15624) Fix the SetQuotaByStorageTypeOp problem after updating hadoop

2021-01-04 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HDFS-15624?focusedWorklogId=531040=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-531040 ] ASF GitHub Bot logged work on HDFS-15624: - Author: ASF GitHub Bot

[jira] [Work logged] (HDFS-15624) Fix the SetQuotaByStorageTypeOp problem after updating hadoop

2021-01-04 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HDFS-15624?focusedWorklogId=531063=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-531063 ] ASF GitHub Bot logged work on HDFS-15624: - Author: ASF GitHub Bot

[jira] [Work logged] (HDFS-15624) Fix the SetQuotaByStorageTypeOp problem after updating hadoop

2021-01-04 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HDFS-15624?focusedWorklogId=531046=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-531046 ] ASF GitHub Bot logged work on HDFS-15624: - Author: ASF GitHub Bot

[jira] [Work logged] (HDFS-15719) [Hadoop 3] Both NameNodes can crash simultaneously due to the short JN socket timeout

2021-01-04 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HDFS-15719?focusedWorklogId=531016=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-531016 ] ASF GitHub Bot logged work on HDFS-15719: - Author: ASF GitHub Bot

[jira] [Resolved] (HDFS-15719) [Hadoop 3] Both NameNodes can crash simultaneously due to the short JN socket timeout

2021-01-04 Thread Wei-Chiu Chuang (Jira)
[ https://issues.apache.org/jira/browse/HDFS-15719?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wei-Chiu Chuang resolved HDFS-15719. Fix Version/s: 3.2.3 3.1.5 3.4.0

[jira] [Work logged] (HDFS-15719) [Hadoop 3] Both NameNodes can crash simultaneously due to the short JN socket timeout

2021-01-04 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HDFS-15719?focusedWorklogId=531017=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-531017 ] ASF GitHub Bot logged work on HDFS-15719: - Author: ASF GitHub Bot

[jira] [Work logged] (HDFS-15624) Fix the SetQuotaByStorageTypeOp problem after updating hadoop

2021-01-04 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HDFS-15624?focusedWorklogId=530579=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-530579 ] ASF GitHub Bot logged work on HDFS-15624: - Author: ASF GitHub Bot

[jira] [Work logged] (HDFS-15549) Improve DISK/ARCHIVE movement if they are on same filesystem

2021-01-04 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HDFS-15549?focusedWorklogId=530583=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-530583 ] ASF GitHub Bot logged work on HDFS-15549: - Author: ASF GitHub Bot

[jira] [Commented] (HDFS-15735) NameNode memory Leak on frequent execution of fsck

2021-01-04 Thread Ayush Saxena (Jira)
[ https://issues.apache.org/jira/browse/HDFS-15735?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17258131#comment-17258131 ] Ayush Saxena commented on HDFS-15735: - Tracer is a {{private}} variable, Not used anywhere, Tracer

[jira] [Work logged] (HDFS-15754) Create packet metrics for DataNode

2021-01-04 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HDFS-15754?focusedWorklogId=530640=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-530640 ] ASF GitHub Bot logged work on HDFS-15754: - Author: ASF GitHub Bot

[jira] [Commented] (HDFS-15735) NameNode memory Leak on frequent execution of fsck

2021-01-04 Thread Brahma Reddy Battula (Jira)
[ https://issues.apache.org/jira/browse/HDFS-15735?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17258055#comment-17258055 ] Brahma Reddy Battula commented on HDFS-15735: - {quote}I am not sure about it, why it being

[jira] [Work logged] (HDFS-15759) EC: Verify EC reconstruction correctness on DataNode

2021-01-04 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HDFS-15759?focusedWorklogId=530681=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-530681 ] ASF GitHub Bot logged work on HDFS-15759: - Author: ASF GitHub Bot

[jira] [Updated] (HDFS-15759) EC: Verify EC reconstruction correctness on DataNode

2021-01-04 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HDFS-15759?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated HDFS-15759: -- Labels: pull-request-available (was: ) > EC: Verify EC reconstruction correctness on DataNode

[jira] [Created] (HDFS-15759) EC: Verify EC reconstruction correctness on DataNode

2021-01-04 Thread Toshihiko Uchida (Jira)
Toshihiko Uchida created HDFS-15759: --- Summary: EC: Verify EC reconstruction correctness on DataNode Key: HDFS-15759 URL: https://issues.apache.org/jira/browse/HDFS-15759 Project: Hadoop HDFS

[jira] [Commented] (HDFS-15751) Add documentation for msync() API to filesystem.md

2021-01-04 Thread Steve Loughran (Jira)
[ https://issues.apache.org/jira/browse/HDFS-15751?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17258259#comment-17258259 ] Steve Loughran commented on HDFS-15751: --- LGTM, though the doc reference to HDFS should be relative

[jira] [Assigned] (HDFS-15759) EC: Verify EC reconstruction correctness on DataNode

2021-01-04 Thread Toshihiko Uchida (Jira)
[ https://issues.apache.org/jira/browse/HDFS-15759?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Toshihiko Uchida reassigned HDFS-15759: --- Assignee: Toshihiko Uchida > EC: Verify EC reconstruction correctness on DataNode >

[jira] [Commented] (HDFS-15757) RBF: Improving Router Connection Management

2021-01-04 Thread Jira
[ https://issues.apache.org/jira/browse/HDFS-15757?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17258364#comment-17258364 ] Íñigo Goiri commented on HDFS-15757: Thank you [~fengnanli] for the proposal. The connection manager

[jira] [Created] (HDFS-15760) Validate the target indices in ErasureCoding worker in reconstruction process

2021-01-04 Thread Uma Maheswara Rao G (Jira)
Uma Maheswara Rao G created HDFS-15760: -- Summary: Validate the target indices in ErasureCoding worker in reconstruction process Key: HDFS-15760 URL: https://issues.apache.org/jira/browse/HDFS-15760

[jira] [Commented] (HDFS-15748) RBF: Move the router related part from hadoop-federation-balance module to hadoop-hdfs-rbf.

2021-01-04 Thread Jira
[ https://issues.apache.org/jira/browse/HDFS-15748?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17258366#comment-17258366 ] Íñigo Goiri commented on HDFS-15748: +1 on [^HDFS-15748.004.patch]. > RBF: Move the router related

[jira] [Commented] (HDFS-15757) RBF: Improving Router Connection Management

2021-01-04 Thread Fengnan Li (Jira)
[ https://issues.apache.org/jira/browse/HDFS-15757?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17258376#comment-17258376 ] Fengnan Li commented on HDFS-15757: --- Thanks for the review [~elgoiri] There are two metrics we will try

[jira] [Updated] (HDFS-15748) RBF: Move the router related part from hadoop-federation-balance module to hadoop-hdfs-rbf.

2021-01-04 Thread Ayush Saxena (Jira)
[ https://issues.apache.org/jira/browse/HDFS-15748?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ayush Saxena updated HDFS-15748: Hadoop Flags: Reviewed Resolution: Fixed Status: Resolved (was: Patch Available)

[jira] [Comment Edited] (HDFS-15761) Dead NORMAL DN shouldn't transit to DECOMMISSIONED immediately

2021-01-04 Thread Ye Ni (Jira)
[ https://issues.apache.org/jira/browse/HDFS-15761?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17258452#comment-17258452 ] Ye Ni edited comment on HDFS-15761 at 1/4/21, 7:45 PM: --- cc [~mingma],

[jira] [Updated] (HDFS-15761) Dead NORMAL DN shouldn't transit to DECOMMISSIONED immediately

2021-01-04 Thread Ye Ni (Jira)
[ https://issues.apache.org/jira/browse/HDFS-15761?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ye Ni updated HDFS-15761: - Description: To decommission a dead DN, the complete logic should be Dead, NORMAL -> Dead,

[jira] [Commented] (HDFS-15748) RBF: Move the router related part from hadoop-federation-balance module to hadoop-hdfs-rbf.

2021-01-04 Thread Ayush Saxena (Jira)
[ https://issues.apache.org/jira/browse/HDFS-15748?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17258410#comment-17258410 ] Ayush Saxena commented on HDFS-15748: - Committed to trunk. Thanx [~LiJinglun] for the contribution

[jira] [Work logged] (HDFS-15549) Improve DISK/ARCHIVE movement if they are on same filesystem

2021-01-04 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HDFS-15549?focusedWorklogId=530857=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-530857 ] ASF GitHub Bot logged work on HDFS-15549: - Author: ASF GitHub Bot

[jira] [Work logged] (HDFS-15759) EC: Verify EC reconstruction correctness on DataNode

2021-01-04 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HDFS-15759?focusedWorklogId=530820=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-530820 ] ASF GitHub Bot logged work on HDFS-15759: - Author: ASF GitHub Bot

[jira] [Created] (HDFS-15761) Dead NORMAL DN shouldn't transit to DECOMMISSIONED immediately

2021-01-04 Thread Ye Ni (Jira)
Ye Ni created HDFS-15761: Summary: Dead NORMAL DN shouldn't transit to DECOMMISSIONED immediately Key: HDFS-15761 URL: https://issues.apache.org/jira/browse/HDFS-15761 Project: Hadoop HDFS Issue

[jira] [Work logged] (HDFS-15761) Dead NORMAL DN shouldn't transit to DECOMMISSIONED immediately

2021-01-04 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HDFS-15761?focusedWorklogId=530835=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-530835 ] ASF GitHub Bot logged work on HDFS-15761: - Author: ASF GitHub Bot

[jira] [Updated] (HDFS-15761) Dead NORMAL DN shouldn't transit to DECOMMISSIONED immediately

2021-01-04 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HDFS-15761?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated HDFS-15761: -- Labels: pull-request-available (was: ) > Dead NORMAL DN shouldn't transit to DECOMMISSIONED

[jira] [Comment Edited] (HDFS-15761) Dead NORMAL DN shouldn't transit to DECOMMISSIONED immediately

2021-01-04 Thread Ye Ni (Jira)
[ https://issues.apache.org/jira/browse/HDFS-15761?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17258452#comment-17258452 ] Ye Ni edited comment on HDFS-15761 at 1/4/21, 7:46 PM: --- cc [~mingma],

[jira] [Updated] (HDFS-15761) Dead NORMAL DN shouldn't transit to DECOMMISSIONED immediately

2021-01-04 Thread Ye Ni (Jira)
[ https://issues.apache.org/jira/browse/HDFS-15761?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ye Ni updated HDFS-15761: - Description: To decommission a dead DN, the complete logic should be Dead, NORMAL -> Dead,

[jira] [Updated] (HDFS-15761) Dead NORMAL DN shouldn't transit to DECOMMISSIONED immediately

2021-01-04 Thread Ye Ni (Jira)
[ https://issues.apache.org/jira/browse/HDFS-15761?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ye Ni updated HDFS-15761: - Description: To decommission a dead DN, the complete logic should be Dead, NORMAL -> Dead,

[jira] [Commented] (HDFS-15761) Dead NORMAL DN shouldn't transit to DECOMMISSIONED immediately

2021-01-04 Thread Ye Ni (Jira)
[ https://issues.apache.org/jira/browse/HDFS-15761?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17258452#comment-17258452 ] Ye Ni commented on HDFS-15761: -- cc [~mingma], [~andrew.wang], [~aiden_zhang], [~inigoiri] > Dead NORMAL DN

[jira] [Updated] (HDFS-15761) Dead NORMAL DN shouldn't transit to DECOMMISSIONED immediately

2021-01-04 Thread Ye Ni (Jira)
[ https://issues.apache.org/jira/browse/HDFS-15761?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ye Ni updated HDFS-15761: - Description: To decommission a dead DN, the complete logic should be Dead, NORMAL -> Dead,

[jira] [Updated] (HDFS-15761) Dead NORMAL DN shouldn't transit to DECOMMISSIONED immediately

2021-01-04 Thread Ye Ni (Jira)
[ https://issues.apache.org/jira/browse/HDFS-15761?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ye Ni updated HDFS-15761: - Description: To decommission a dead DN, the complete logic should be Dead, NORMAL -> Dead,

[jira] [Updated] (HDFS-15761) Dead NORMAL DN shouldn't transit to DECOMMISSIONED immediately

2021-01-04 Thread Ye Ni (Jira)
[ https://issues.apache.org/jira/browse/HDFS-15761?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ye Ni updated HDFS-15761: - Description: To decommission a dead DN, the complete logic should be Dead, NORMAL -> Dead,

[jira] [Commented] (HDFS-15732) EC client will not retry get block token when block token expired in kerberized cluster

2021-01-04 Thread Wei-Chiu Chuang (Jira)
[ https://issues.apache.org/jira/browse/HDFS-15732?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17258598#comment-17258598 ] Wei-Chiu Chuang commented on HDFS-15732: Probably similar to HDFS-10609 and HDFS-11741 where we

[jira] [Commented] (HDFS-15757) RBF: Improving Router Connection Management

2021-01-04 Thread Fengnan Li (Jira)
[ https://issues.apache.org/jira/browse/HDFS-15757?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17258611#comment-17258611 ] Fengnan Li commented on HDFS-15757: --- Uploaded v2 with more metrics and some changes. I will start some

[jira] [Updated] (HDFS-15757) RBF: Improving Router Connection Management

2021-01-04 Thread Fengnan Li (Jira)
[ https://issues.apache.org/jira/browse/HDFS-15757?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Fengnan Li updated HDFS-15757: -- Attachment: RBF_ Improving Router Connection Management_v2.pdf > RBF: Improving Router Connection

[jira] [Updated] (HDFS-15757) RBF: Improving Router Connection Management

2021-01-04 Thread Fengnan Li (Jira)
[ https://issues.apache.org/jira/browse/HDFS-15757?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Fengnan Li updated HDFS-15757: -- Attachment: RBF_ Improving Router Connection Management_v2.pdf > RBF: Improving Router Connection

[jira] [Commented] (HDFS-15732) EC client will not retry get block token when block token expired in kerberized cluster

2021-01-04 Thread Wei-Chiu Chuang (Jira)
[ https://issues.apache.org/jira/browse/HDFS-15732?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17258600#comment-17258600 ] Wei-Chiu Chuang commented on HDFS-15732: [~lalapala] would you like to submit a PR? I see that

[jira] [Assigned] (HDFS-15732) EC client will not retry get block token when block token expired in kerberized cluster

2021-01-04 Thread Wei-Chiu Chuang (Jira)
[ https://issues.apache.org/jira/browse/HDFS-15732?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wei-Chiu Chuang reassigned HDFS-15732: -- Assignee: gaozhan ding > EC client will not retry get block token when block token

[jira] [Work logged] (HDFS-15761) Dead NORMAL DN shouldn't transit to DECOMMISSIONED immediately

2021-01-04 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/HDFS-15761?focusedWorklogId=530986=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-530986 ] ASF GitHub Bot logged work on HDFS-15761: - Author: ASF GitHub Bot

[jira] [Updated] (HDFS-15761) Dead NORMAL DN shouldn't transit to DECOMMISSIONED immediately

2021-01-04 Thread Ye Ni (Jira)
[ https://issues.apache.org/jira/browse/HDFS-15761?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ye Ni updated HDFS-15761: - Description: To decommission a dead DN, the complete logic should be Dead, NORMAL -> Dead,

[jira] [Updated] (HDFS-15757) RBF: Improving Router Connection Management

2021-01-04 Thread Fengnan Li (Jira)
[ https://issues.apache.org/jira/browse/HDFS-15757?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Fengnan Li updated HDFS-15757: -- Attachment: (was: RBF_ Improving Router Connection Management_v2.pdf) > RBF: Improving Router

[jira] [Updated] (HDFS-15757) RBF: Improving Router Connection Management

2021-01-04 Thread Fengnan Li (Jira)
[ https://issues.apache.org/jira/browse/HDFS-15757?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Fengnan Li updated HDFS-15757: -- Attachment: RBF_ Improving Router Connection Management_v2.pdf > RBF: Improving Router Connection

[jira] [Updated] (HDFS-15757) RBF: Improving Router Connection Management

2021-01-04 Thread Fengnan Li (Jira)
[ https://issues.apache.org/jira/browse/HDFS-15757?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Fengnan Li updated HDFS-15757: -- Attachment: (was: RBF_ Improving Router Connection Management_v2.pdf) > RBF: Improving Router