[jira] [Assigned] (HDFS-15998) Fix NullPointException In listOpenFiles

2021-05-11 Thread Haiyang Hu (Jira)
[ https://issues.apache.org/jira/browse/HDFS-15998?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Haiyang Hu reassigned HDFS-15998: - Assignee: Haiyang Hu > Fix NullPointException In listOpenFiles >

[jira] [Commented] (HDFS-15998) Fix NullPointException In listOpenFiles

2021-05-20 Thread Haiyang Hu (Jira)
[ https://issues.apache.org/jira/browse/HDFS-15998?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17348163#comment-17348163 ] Haiyang Hu commented on HDFS-15998: --- [~hexiaoqiao] Thanks for comments. The current log attached above

[jira] [Created] (HDFS-15998) Fix NullPointException In listOpenFiles

2021-04-26 Thread Haiyang Hu (Jira)
Haiyang Hu created HDFS-15998: - Summary: Fix NullPointException In listOpenFiles Key: HDFS-15998 URL: https://issues.apache.org/jira/browse/HDFS-15998 Project: Hadoop HDFS Issue Type: Bug

[jira] [Updated] (HDFS-15998) Fix NullPointException In listOpenFiles

2021-04-26 Thread Haiyang Hu (Jira)
[ https://issues.apache.org/jira/browse/HDFS-15998?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Haiyang Hu updated HDFS-15998: -- Description: Use the Hadoop 3.2.0 client execute the following command: occasionally appear NPE.

[jira] [Updated] (HDFS-15998) Fix NullPointException In listOpenFiles

2021-04-26 Thread Haiyang Hu (Jira)
[ https://issues.apache.org/jira/browse/HDFS-15998?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Haiyang Hu updated HDFS-15998: -- Description: Use the Hadoop 3.2.0 client execute the following command: occasionally throw NPE. hdfs

[jira] [Issue Comment Deleted] (HDFS-15998) Fix NullPointException In listOpenFiles

2021-04-26 Thread Haiyang Hu (Jira)
[ https://issues.apache.org/jira/browse/HDFS-15998?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Haiyang Hu updated HDFS-15998: -- Comment: was deleted (was: {code:java}LeaseManager.java public BatchedListEntries

[jira] [Commented] (HDFS-15998) Fix NullPointException In listOpenFiles

2021-04-26 Thread Haiyang Hu (Jira)
[ https://issues.apache.org/jira/browse/HDFS-15998?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17332916#comment-17332916 ] Haiyang Hu commented on HDFS-15998: --- Later,upload PR > Fix NullPointException In listOpenFiles >

[jira] [Updated] (HDFS-15998) Fix NullPointException In listOpenFiles

2021-04-26 Thread Haiyang Hu (Jira)
[ https://issues.apache.org/jira/browse/HDFS-15998?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Haiyang Hu updated HDFS-15998: -- Description: Use the Hadoop 3.2.0 client execute the following command: occasionally throw NPE. hdfs

[jira] [Comment Edited] (HDFS-15998) Fix NullPointException In listOpenFiles

2021-04-26 Thread Haiyang Hu (Jira)
[ https://issues.apache.org/jira/browse/HDFS-15998?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17332916#comment-17332916 ] Haiyang Hu edited comment on HDFS-15998 at 4/27/21, 4:12 AM: -

[jira] [Comment Edited] (HDFS-15998) Fix NullPointException In listOpenFiles

2021-04-26 Thread Haiyang Hu (Jira)
[ https://issues.apache.org/jira/browse/HDFS-15998?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17332916#comment-17332916 ] Haiyang Hu edited comment on HDFS-15998 at 4/27/21, 4:10 AM: -

[jira] [Updated] (HDFS-15998) Fix NullPointException In listOpenFiles

2021-04-26 Thread Haiyang Hu (Jira)
[ https://issues.apache.org/jira/browse/HDFS-15998?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Haiyang Hu updated HDFS-15998: -- Description: Use the Hadoop 3.2.0 client execute the following command: occasionally appear NPE.

[jira] [Comment Edited] (HDFS-15998) Fix NullPointException In listOpenFiles

2021-04-26 Thread Haiyang Hu (Jira)
[ https://issues.apache.org/jira/browse/HDFS-15998?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17332914#comment-17332914 ] Haiyang Hu edited comment on HDFS-15998 at 4/27/21, 4:03 AM: -

[jira] [Commented] (HDFS-15998) Fix NullPointException In listOpenFiles

2021-04-26 Thread Haiyang Hu (Jira)
[ https://issues.apache.org/jira/browse/HDFS-15998?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17332914#comment-17332914 ] Haiyang Hu commented on HDFS-15998: --- {code:java}LeaseManager.java public BatchedListEntries

[jira] [Updated] (HDFS-15998) Fix NullPointException In listOpenFiles

2021-04-26 Thread Haiyang Hu (Jira)
[ https://issues.apache.org/jira/browse/HDFS-15998?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Haiyang Hu updated HDFS-15998: -- Description: Use the Hadoop 3.2.0 client execute the following command: occasionally appear NPE.

[jira] [Updated] (HDFS-15998) Fix NullPointException In listOpenFiles

2021-04-26 Thread Haiyang Hu (Jira)
[ https://issues.apache.org/jira/browse/HDFS-15998?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Haiyang Hu updated HDFS-15998: -- Description: Use the Hadoop 3.2.0 client execute the following command: occasionally throw NPE. hdfs

[jira] [Updated] (HDFS-15998) Fix NullPointException In listOpenFiles

2021-04-26 Thread Haiyang Hu (Jira)
[ https://issues.apache.org/jira/browse/HDFS-15998?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Haiyang Hu updated HDFS-15998: -- Description: Use the Hadoop 3.2.0 client execute the following command: occasionally throw NPE. hdfs

[jira] [Comment Edited] (HDFS-15998) Fix NullPointException In listOpenFiles

2021-04-26 Thread Haiyang Hu (Jira)
[ https://issues.apache.org/jira/browse/HDFS-15998?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17332916#comment-17332916 ] Haiyang Hu edited comment on HDFS-15998 at 4/27/21, 4:09 AM: -

[jira] [Updated] (HDFS-15889) Erasure Coding: Limit the number of concurrent EC Reconstruct blocks in NameNode

2021-03-12 Thread Haiyang Hu (Jira)
[ https://issues.apache.org/jira/browse/HDFS-15889?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Haiyang Hu updated HDFS-15889: -- Description: Consider limit the number of concurrent EC Reconstruct blocks in NameNode to avoid a

[jira] [Created] (HDFS-15889) Erasure Coding: Limit the number of concurrent EC Reconstruct blocks in NameNode

2021-03-12 Thread Haiyang Hu (Jira)
Haiyang Hu created HDFS-15889: - Summary: Erasure Coding: Limit the number of concurrent EC Reconstruct blocks in NameNode Key: HDFS-15889 URL: https://issues.apache.org/jira/browse/HDFS-15889 Project:

[jira] [Created] (HDFS-15877) BlockReconstructionWork should resetTargets() before BlockManager#validateReconstructionWork return false

2021-03-05 Thread Haiyang Hu (Jira)
Haiyang Hu created HDFS-15877: - Summary: BlockReconstructionWork should resetTargets() before BlockManager#validateReconstructionWork return false Key: HDFS-15877 URL: https://issues.apache.org/jira/browse/HDFS-15877

[jira] [Updated] (HDFS-15877) BlockReconstructionWork should resetTargets() before BlockManager#validateReconstructionWork return false

2021-03-05 Thread Haiyang Hu (Jira)
[ https://issues.apache.org/jira/browse/HDFS-15877?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Haiyang Hu updated HDFS-15877: -- Description: the related code is in BlockRecoveryWorker.java {code}  {code} private boolean

[jira] [Updated] (HDFS-15877) BlockReconstructionWork should resetTargets() before BlockManager#validateReconstructionWork return false

2021-03-05 Thread Haiyang Hu (Jira)
[ https://issues.apache.org/jira/browse/HDFS-15877?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Haiyang Hu updated HDFS-15877: -- Description: the related code is in BlockManager.java {code:java}  {code} private boolean

[jira] [Updated] (HDFS-15877) BlockReconstructionWork should resetTargets() before BlockManager#validateReconstructionWork return false

2021-03-05 Thread Haiyang Hu (Jira)
[ https://issues.apache.org/jira/browse/HDFS-15877?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Haiyang Hu updated HDFS-15877: -- Description: the related code is in BlockManager.java {code:java} // code placeholder {code} private

[jira] [Updated] (HDFS-15877) BlockReconstructionWork should resetTargets() before BlockManager#validateReconstructionWork return false

2021-03-05 Thread Haiyang Hu (Jira)
[ https://issues.apache.org/jira/browse/HDFS-15877?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Haiyang Hu updated HDFS-15877: -- Description: the related code is in BlockManager.java {code:java} // code placeholder {code} private

[jira] [Updated] (HDFS-15877) BlockReconstructionWork should resetTargets() before BlockManager#validateReconstructionWork return false

2021-03-05 Thread Haiyang Hu (Jira)
[ https://issues.apache.org/jira/browse/HDFS-15877?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Haiyang Hu updated HDFS-15877: -- Description: the related code is in BlockManager.java {code:java} // code placeholder private boolean

[jira] [Updated] (HDFS-15877) BlockReconstructionWork should resetTargets() before BlockManager#validateReconstructionWork return false

2021-03-05 Thread Haiyang Hu (Jira)
[ https://issues.apache.org/jira/browse/HDFS-15877?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Haiyang Hu updated HDFS-15877: -- Description: the related code is in BlockManager.java   {code:java} // code placeholder private

[jira] [Updated] (HDFS-15877) BlockReconstructionWork should resetTargets() before BlockManager#validateReconstructionWork return false

2021-03-05 Thread Haiyang Hu (Jira)
[ https://issues.apache.org/jira/browse/HDFS-15877?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Haiyang Hu updated HDFS-15877: -- Description: the related code is in BlockManager.java   {code:java} // code placeholder private

[jira] [Updated] (HDFS-15877) BlockReconstructionWork should resetTargets() before BlockManager#validateReconstructionWork return false

2021-03-05 Thread Haiyang Hu (Jira)
[ https://issues.apache.org/jira/browse/HDFS-15877?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Haiyang Hu updated HDFS-15877: -- Description: BlockReconstructionWork should resetTargets() before

[jira] [Updated] (HDFS-15877) BlockReconstructionWork should resetTargets() before BlockManager#validateReconstructionWork return false

2021-03-05 Thread Haiyang Hu (Jira)
[ https://issues.apache.org/jira/browse/HDFS-15877?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Haiyang Hu updated HDFS-15877: -- Description: BlockReconstructionWork should resetTargets() before

[jira] [Updated] (HDFS-15877) BlockReconstructionWork should resetTargets() before BlockManager#validateReconstructionWork return false

2021-03-05 Thread Haiyang Hu (Jira)
[ https://issues.apache.org/jira/browse/HDFS-15877?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Haiyang Hu updated HDFS-15877: -- Description: BlockReconstructionWork should resetTargets() before

[jira] [Assigned] (HDFS-15877) BlockReconstructionWork should resetTargets() before BlockManager#validateReconstructionWork return false

2021-03-05 Thread Haiyang Hu (Jira)
[ https://issues.apache.org/jira/browse/HDFS-15877?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Haiyang Hu reassigned HDFS-15877: - Assignee: Haiyang Hu > BlockReconstructionWork should resetTargets() before >

[jira] [Updated] (HDFS-16231) Fix TestDataNodeMetrics#testReceivePacketSlowMetrics

2021-09-18 Thread Haiyang Hu (Jira)
[ https://issues.apache.org/jira/browse/HDFS-16231?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Haiyang Hu updated HDFS-16231: -- Description: TestDataNodeMetrics#testReceivePacketSlowMetrics fails with stacktrace:

[jira] [Updated] (HDFS-16231) Fix TestDataNodeMetrics#testReceivePacketSlowMetrics

2021-09-18 Thread Haiyang Hu (Jira)
[ https://issues.apache.org/jira/browse/HDFS-16231?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Haiyang Hu updated HDFS-16231: -- Parent: HDFS-15646 Issue Type: Sub-task (was: Bug) > Fix

[jira] [Updated] (HDFS-16231) Fix TestDataNodeMetrics#testReceivePacketSlowMetrics

2021-09-18 Thread Haiyang Hu (Jira)
[ https://issues.apache.org/jira/browse/HDFS-16231?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Haiyang Hu updated HDFS-16231: -- Description: TestMover#testMoverWithStripedFile fails intermittently with stacktrace: > Fix

[jira] [Created] (HDFS-16231) Fix TestDataNodeMetrics#testReceivePacketSlowMetrics

2021-09-18 Thread Haiyang Hu (Jira)
Haiyang Hu created HDFS-16231: - Summary: Fix TestDataNodeMetrics#testReceivePacketSlowMetrics Key: HDFS-16231 URL: https://issues.apache.org/jira/browse/HDFS-16231 Project: Hadoop HDFS Issue

[jira] [Updated] (HDFS-16231) Fix TestDataNodeMetrics#testReceivePacketSlowMetrics

2021-09-18 Thread Haiyang Hu (Jira)
[ https://issues.apache.org/jira/browse/HDFS-16231?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Haiyang Hu updated HDFS-16231: -- Issue Type: Bug (was: Task) > Fix TestDataNodeMetrics#testReceivePacketSlowMetrics >

[jira] [Updated] (HDFS-16231) Fix TestDataNodeMetrics#testReceivePacketSlowMetrics

2021-09-18 Thread Haiyang Hu (Jira)
[ https://issues.apache.org/jira/browse/HDFS-16231?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Haiyang Hu updated HDFS-16231: -- Description: TestDataNodeMetrics#testReceivePacketSlowMetrics fails with stacktrace: {code:java}

[jira] [Updated] (HDFS-16231) Fix TestDataNodeMetrics#testReceivePacketSlowMetrics

2021-09-18 Thread Haiyang Hu (Jira)
[ https://issues.apache.org/jira/browse/HDFS-16231?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Haiyang Hu updated HDFS-16231: -- Description: TestDataNodeMetrics#testReceivePacketSlowMetrics fails with stacktrace: {code:java}

[jira] [Commented] (HDFS-16231) Fix TestDataNodeMetrics#testReceivePacketSlowMetrics

2021-09-30 Thread Haiyang Hu (Jira)
[ https://issues.apache.org/jira/browse/HDFS-16231?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17422605#comment-17422605 ] Haiyang Hu commented on HDFS-16231: --- [~ferhui] Check branch-3.3 and there is no

[jira] [Created] (HDFS-16247) RBF: Fix the ProcessingAvgTime and ProxyAvgTime code comments and document metrics describe ms unit

2021-10-01 Thread Haiyang Hu (Jira)
Haiyang Hu created HDFS-16247: - Summary: RBF: Fix the ProcessingAvgTime and ProxyAvgTime code comments and document metrics describe ms unit Key: HDFS-16247 URL: https://issues.apache.org/jira/browse/HDFS-16247

[jira] [Updated] (HDFS-16247) RBF: Fix the ProcessingAvgTime and ProxyAvgTime code comments and document metrics describe ms unit

2021-10-01 Thread Haiyang Hu (Jira)
[ https://issues.apache.org/jira/browse/HDFS-16247?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Haiyang Hu updated HDFS-16247: -- Description: In the FederationRPCMetrics, proxy time and processing time ms units ,but code comments

[jira] [Updated] (HDFS-16247) RBF: Fix the ProcessingAvgTime and ProxyAvgTime code comments and document metrics describe ms unit

2021-10-01 Thread Haiyang Hu (Jira)
[ https://issues.apache.org/jira/browse/HDFS-16247?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Haiyang Hu updated HDFS-16247: -- Description: In the FederationRPCMetrics, proxy time and processing time ms units ,but code comments

[jira] [Created] (HDFS-16314) Support to makedfs.namenode.block-placement-policy.exclude-slow-nodes.enabled reconfigurable

2021-11-10 Thread Haiyang Hu (Jira)
Haiyang Hu created HDFS-16314: - Summary: Support to makedfs.namenode.block-placement-policy.exclude-slow-nodes.enabled reconfigurable Key: HDFS-16314 URL: https://issues.apache.org/jira/browse/HDFS-16314

[jira] [Updated] (HDFS-16314) Support to make dfs.namenode.block-placement-policy.exclude-slow-nodes.enabled reconfigurable

2021-11-10 Thread Haiyang Hu (Jira)
[ https://issues.apache.org/jira/browse/HDFS-16314?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Haiyang Hu updated HDFS-16314: -- Description: Consider that make dfs.namenode.block-placement-policy.exclude-slow-nodes.enabled

[jira] [Updated] (HDFS-16314) Support to makedfs.namenode.block-placement-policy.exclude-slow-nodes.enabled reconfigurable

2021-11-10 Thread Haiyang Hu (Jira)
[ https://issues.apache.org/jira/browse/HDFS-16314?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Haiyang Hu updated HDFS-16314: -- Description: Consider that make dfs.namenode.avoid.read.slow.datanode reconfigurable and rapid

[jira] [Updated] (HDFS-16314) Support to make dfs.namenode.block-placement-policy.exclude-slow-nodes.enabled reconfigurable

2021-11-10 Thread Haiyang Hu (Jira)
[ https://issues.apache.org/jira/browse/HDFS-16314?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Haiyang Hu updated HDFS-16314: -- Summary: Support to make dfs.namenode.block-placement-policy.exclude-slow-nodes.enabled reconfigurable

[jira] [Assigned] (HDFS-16314) Support to make dfs.namenode.block-placement-policy.exclude-slow-nodes.enabled reconfigurable

2021-11-10 Thread Haiyang Hu (Jira)
[ https://issues.apache.org/jira/browse/HDFS-16314?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Haiyang Hu reassigned HDFS-16314: - Assignee: Haiyang Hu > Support to make >

[jira] [Commented] (HDFS-16287) Support to make dfs.namenode.avoid.read.slow.datanode and dfs.namenode.block-placement-policy.exclude-slow-nodes.enabled reconfigurable

2021-10-28 Thread Haiyang Hu (Jira)
[ https://issues.apache.org/jira/browse/HDFS-16287?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17435271#comment-17435271 ] Haiyang Hu commented on HDFS-16287: --- [~tomscut] Thank you for reply!PR have submitted. > Support to

[jira] [Updated] (HDFS-16287) Support to make dfs.namenode.avoid.read.slow.datanode reconfigurable

2021-10-29 Thread Haiyang Hu (Jira)
[ https://issues.apache.org/jira/browse/HDFS-16287?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Haiyang Hu updated HDFS-16287: -- Description: 1. Consider that make dfs.namenode.avoid.read.slow.datanode reconfigurable and rapid

[jira] [Updated] (HDFS-16287) Support to make dfs.namenode.avoid.read.slow.datanode reconfigurable

2021-10-29 Thread Haiyang Hu (Jira)
[ https://issues.apache.org/jira/browse/HDFS-16287?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Haiyang Hu updated HDFS-16287: -- Summary: Support to make dfs.namenode.avoid.read.slow.datanode reconfigurable (was: Support to make

[jira] [Created] (HDFS-16287) Support to make dfs.namenode.avoid.read.slow.datanode and dfs.namenode.block-placement-policy.exclude-slow-nodes.enabled reconfigurable

2021-10-27 Thread Haiyang Hu (Jira)
Haiyang Hu created HDFS-16287: - Summary: Support to make dfs.namenode.avoid.read.slow.datanode and dfs.namenode.block-placement-policy.exclude-slow-nodes.enabled reconfigurable Key: HDFS-16287 URL:

[jira] [Updated] (HDFS-16287) Support to make dfs.namenode.avoid.read.slow.datanode and dfs.namenode.block-placement-policy.exclude-slow-nodes.enabled reconfigurable

2021-10-27 Thread Haiyang Hu (Jira)
[ https://issues.apache.org/jira/browse/HDFS-16287?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Haiyang Hu updated HDFS-16287: -- Description: 1. Consider that make dfs.namenode.avoid.read.slow.datanode and

[jira] [Updated] (HDFS-16287) Support to make dfs.namenode.avoid.read.slow.datanode and dfs.namenode.block-placement-policy.exclude-slow-nodes.enabled reconfigurable

2021-10-27 Thread Haiyang Hu (Jira)
[ https://issues.apache.org/jira/browse/HDFS-16287?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Haiyang Hu updated HDFS-16287: -- Description: 1. Consider that make dfs.namenode.avoid.read.slow.datanode and

[jira] [Updated] (HDFS-16287) Support to make dfs.namenode.avoid.read.slow.datanode and dfs.namenode.block-placement-policy.exclude-slow-nodes.enabled reconfigurable

2021-10-27 Thread Haiyang Hu (Jira)
[ https://issues.apache.org/jira/browse/HDFS-16287?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Haiyang Hu updated HDFS-16287: -- Description: 1. Consider that make dfs.namenode.avoid.read.slow.datanode and

[jira] [Updated] (HDFS-16287) Support to make dfs.namenode.avoid.read.slow.datanode and dfs.namenode.block-placement-policy.exclude-slow-nodes.enabled reconfigurable

2021-10-27 Thread Haiyang Hu (Jira)
[ https://issues.apache.org/jira/browse/HDFS-16287?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Haiyang Hu updated HDFS-16287: -- Description: 1. Consider that make dfs.namenode.avoid.read.slow.datanode and

[jira] [Updated] (HDFS-16287) Support to make dfs.namenode.avoid.read.slow.datanode and dfs.namenode.block-placement-policy.exclude-slow-nodes.enabled reconfigurable

2021-10-27 Thread Haiyang Hu (Jira)
[ https://issues.apache.org/jira/browse/HDFS-16287?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Haiyang Hu updated HDFS-16287: -- Description: 1. Consider that make dfs.namenode.avoid.read.slow.datanode and

[jira] [Updated] (HDFS-16287) Support to make dfs.namenode.avoid.read.slow.datanode and dfs.namenode.block-placement-policy.exclude-slow-nodes.enabled reconfigurable

2021-10-27 Thread Haiyang Hu (Jira)
[ https://issues.apache.org/jira/browse/HDFS-16287?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Haiyang Hu updated HDFS-16287: -- Description: 1. Consider that make dfs.namenode.avoid.read.slow.datanode and

[jira] [Updated] (HDFS-16287) Support to make dfs.namenode.avoid.read.slow.datanode and dfs.namenode.block-placement-policy.exclude-slow-nodes.enabled reconfigurable

2021-10-27 Thread Haiyang Hu (Jira)
[ https://issues.apache.org/jira/browse/HDFS-16287?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Haiyang Hu updated HDFS-16287: -- Description: 1. Consider that make dfs.namenode.avoid.read.slow.datanode and

[jira] [Work stopped] (HDFS-16306) Stale record should be remove when DataNodePeerMetrics#dumpSendPacketDownstreamAvgInfoAsJson

2021-11-08 Thread Haiyang Hu (Jira)
[ https://issues.apache.org/jira/browse/HDFS-16306?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Work on HDFS-16306 stopped by Haiyang Hu. - > Stale record should be remove when >

[jira] [Work started] (HDFS-16306) Stale record should be remove when DataNodePeerMetrics#dumpSendPacketDownstreamAvgInfoAsJson

2021-11-08 Thread Haiyang Hu (Jira)
[ https://issues.apache.org/jira/browse/HDFS-16306?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Work on HDFS-16306 started by Haiyang Hu. - > Stale record should be remove when >

[jira] [Created] (HDFS-16306) Stale record should be remove when DataNodePeerMetrics#dumpSendPacketDownstreamAvgInfoAsJson

2021-11-08 Thread Haiyang Hu (Jira)
Haiyang Hu created HDFS-16306: - Summary: Stale record should be remove when DataNodePeerMetrics#dumpSendPacketDownstreamAvgInfoAsJson Key: HDFS-16306 URL: https://issues.apache.org/jira/browse/HDFS-16306

[jira] [Assigned] (HDFS-16306) Stale record should be remove when DataNodePeerMetrics#dumpSendPacketDownstreamAvgInfoAsJson

2021-11-08 Thread Haiyang Hu (Jira)
[ https://issues.apache.org/jira/browse/HDFS-16306?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Haiyang Hu reassigned HDFS-16306: - Assignee: Haiyang Hu > Stale record should be remove when >

[jira] [Updated] (HDFS-17177) ErasureCodingWork reconstruct ignore the block length is Long.MAX_VALUE.

2023-09-03 Thread Haiyang Hu (Jira)
[ https://issues.apache.org/jira/browse/HDFS-17177?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Haiyang Hu updated HDFS-17177: -- Description: As noted in https://issues.apache.org/jira/browse/HDFS-14720, ErasureCodingWork

[jira] [Updated] (HDFS-17177) ErasureCodingWork reconstruct ignore the block length is Long.MAX_VALUE.

2023-09-03 Thread Haiyang Hu (Jira)
[ https://issues.apache.org/jira/browse/HDFS-17177?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Haiyang Hu updated HDFS-17177: -- Description: As noted in https://issues.apache.org/jira/browse/HDFS-14720, ErasureCodingWork

[jira] [Created] (HDFS-17177) ErasureCodingWork reconstruct ignore the block length is Long.MAX_VALUE.

2023-09-03 Thread Haiyang Hu (Jira)
Haiyang Hu created HDFS-17177: - Summary: ErasureCodingWork reconstruct ignore the block length is Long.MAX_VALUE. Key: HDFS-17177 URL: https://issues.apache.org/jira/browse/HDFS-17177 Project: Hadoop

[jira] [Created] (HDFS-17194) Enhance the log message for striped block recovery

2023-09-16 Thread Haiyang Hu (Jira)
Haiyang Hu created HDFS-17194: - Summary: Enhance the log message for striped block recovery Key: HDFS-17194 URL: https://issues.apache.org/jira/browse/HDFS-17194 Project: Hadoop HDFS Issue Type:

[jira] [Updated] (HDFS-17194) Enhance the log message for striped block recovery

2023-09-16 Thread Haiyang Hu (Jira)
[ https://issues.apache.org/jira/browse/HDFS-17194?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Haiyang Hu updated HDFS-17194: -- Description: For RecoveryTaskStriped#recover log message add internalBlk info,convenient to locate

[jira] [Updated] (HDFS-17194) Enhance the log message for striped block recovery

2023-09-16 Thread Haiyang Hu (Jira)
[ https://issues.apache.org/jira/browse/HDFS-17194?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Haiyang Hu updated HDFS-17194: -- Description: In order to convenient troubleshoot problems, consider add internalBlk information to

[jira] [Updated] (HDFS-17184) In BlockReceiver constructor method need throw DiskOutOfSpaceException

2023-09-09 Thread Haiyang Hu (Jira)
[ https://issues.apache.org/jira/browse/HDFS-17184?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Haiyang Hu updated HDFS-17184: -- Summary: In BlockReceiver constructor method need throw DiskOutOfSpaceException (was: In

[jira] [Updated] (HDFS-17184) In BlockReceiver constructor method need throw DiskOutOfSpaceException

2023-09-09 Thread Haiyang Hu (Jira)
[ https://issues.apache.org/jira/browse/HDFS-17184?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Haiyang Hu updated HDFS-17184: -- Description: BlockReceiver class will receives a block and writes to its disk, in the constructor

[jira] [Updated] (HDFS-17184) In BlockReceiver constructor method need throw DiskOutOfSpaceException

2023-09-09 Thread Haiyang Hu (Jira)
[ https://issues.apache.org/jira/browse/HDFS-17184?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Haiyang Hu updated HDFS-17184: -- Description: BlockReceiver class will receives a block and writes to its disk, in the constructor

[jira] [Assigned] (HDFS-17184) In BlockReceiver method need throw DiskOutOfSpaceException

2023-09-09 Thread Haiyang Hu (Jira)
[ https://issues.apache.org/jira/browse/HDFS-17184?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Haiyang Hu reassigned HDFS-17184: - Assignee: Haiyang Hu > In BlockReceiver method need throw DiskOutOfSpaceException >

[jira] [Created] (HDFS-17184) In BlockReceiver method need throw DiskOutOfSpaceException

2023-09-09 Thread Haiyang Hu (Jira)
Haiyang Hu created HDFS-17184: - Summary: In BlockReceiver method need throw DiskOutOfSpaceException Key: HDFS-17184 URL: https://issues.apache.org/jira/browse/HDFS-17184 Project: Hadoop HDFS

[jira] [Updated] (HDFS-17247) Improve AvailableSpaceRackFaultTolerantBlockPlacementPolicy logic

2023-11-01 Thread Haiyang Hu (Jira)
[ https://issues.apache.org/jira/browse/HDFS-17247?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Haiyang Hu updated HDFS-17247: -- Description: The AvailableSpaceBlockPlacementPolicy has made some optimizations such as

[jira] [Updated] (HDFS-17247) Improve AvailableSpaceRackFaultTolerantBlockPlacementPolicy logic

2023-11-01 Thread Haiyang Hu (Jira)
[ https://issues.apache.org/jira/browse/HDFS-17247?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Haiyang Hu updated HDFS-17247: -- Description: The AvailableSpaceBlockPlacementPolicy has made some optimizations such as

[jira] [Updated] (HDFS-17247) Improve AvailableSpaceRackFaultTolerantBlockPlacementPolicy logic

2023-11-01 Thread Haiyang Hu (Jira)
[ https://issues.apache.org/jira/browse/HDFS-17247?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Haiyang Hu updated HDFS-17247: -- Description: The AvailableSpaceBlockPlacementPolicy has made some optimizations such as

[jira] [Updated] (HDFS-17247) Improve AvailableSpaceRackFaultTolerantBlockPlacementPolicy logic

2023-11-01 Thread Haiyang Hu (Jira)
[ https://issues.apache.org/jira/browse/HDFS-17247?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Haiyang Hu updated HDFS-17247: -- Description: The AvailableSpaceBlockPlacementPolicy has made some optimizations such as

[jira] [Created] (HDFS-17247) Improve AvailableSpaceRackFaultTolerantBlockPlacementPolicy logic

2023-11-01 Thread Haiyang Hu (Jira)
Haiyang Hu created HDFS-17247: - Summary: Improve AvailableSpaceRackFaultTolerantBlockPlacementPolicy logic Key: HDFS-17247 URL: https://issues.apache.org/jira/browse/HDFS-17247 Project: Hadoop HDFS

[jira] [Updated] (HDFS-17247) Improve AvailableSpaceRackFaultTolerantBlockPlacementPolicy logic

2023-11-01 Thread Haiyang Hu (Jira)
[ https://issues.apache.org/jira/browse/HDFS-17247?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Haiyang Hu updated HDFS-17247: -- Description: The AvailableSpaceBlockPlacementPolicy has made some optimizations such as

[jira] [Updated] (HDFS-17247) Improve AvailableSpaceRackFaultTolerantBlockPlacementPolicy logic

2023-11-02 Thread Haiyang Hu (Jira)
[ https://issues.apache.org/jira/browse/HDFS-17247?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Haiyang Hu updated HDFS-17247: -- Description: The AvailableSpaceBlockPlacementPolicy has made some optimizations such as

[jira] [Created] (HDFS-17243) Add the parameter storage type for getBlocks method

2023-10-29 Thread Haiyang Hu (Jira)
Haiyang Hu created HDFS-17243: - Summary: Add the parameter storage type for getBlocks method Key: HDFS-17243 URL: https://issues.apache.org/jira/browse/HDFS-17243 Project: Hadoop HDFS Issue

[jira] [Updated] (HDFS-17247) Improve AvailableSpaceRackFaultTolerantBlockPlacementPolicy logic

2023-11-04 Thread Haiyang Hu (Jira)
[ https://issues.apache.org/jira/browse/HDFS-17247?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Haiyang Hu updated HDFS-17247: -- Description: The AvailableSpaceBlockPlacementPolicy has made some optimizations such as HDFS-14578

[jira] [Updated] (HDFS-17247) Improve AvailableSpaceRackFaultTolerantBlockPlacementPolicy logic

2023-11-04 Thread Haiyang Hu (Jira)
[ https://issues.apache.org/jira/browse/HDFS-17247?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Haiyang Hu updated HDFS-17247: -- Description: * The AvailableSpaceBlockPlacementPolicy has made some optimizations such as HDFS-14578

[jira] [Updated] (HDFS-17247) Improve AvailableSpaceRackFaultTolerantBlockPlacementPolicy logic

2023-11-04 Thread Haiyang Hu (Jira)
[ https://issues.apache.org/jira/browse/HDFS-17247?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Haiyang Hu updated HDFS-17247: -- Description: * The AvailableSpaceBlockPlacementPolicy has made some optimizations such as HDFS-14578

[jira] [Created] (HDFS-17205) HdfsServerConstants.MIN_BLOCKS_FOR_WRITE should be configurable

2023-09-22 Thread Haiyang Hu (Jira)
Haiyang Hu created HDFS-17205: - Summary: HdfsServerConstants.MIN_BLOCKS_FOR_WRITE should be configurable Key: HDFS-17205 URL: https://issues.apache.org/jira/browse/HDFS-17205 Project: Hadoop HDFS

[jira] [Updated] (HDFS-17205) HdfsServerConstants.MIN_BLOCKS_FOR_WRITE should be configurable

2023-09-23 Thread Haiyang Hu (Jira)
[ https://issues.apache.org/jira/browse/HDFS-17205?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Haiyang Hu updated HDFS-17205: -- Description: Current allocate new block , the NameNode will choose datanode and choose a good storage

[jira] [Updated] (HDFS-17205) HdfsServerConstants.MIN_BLOCKS_FOR_WRITE should be configurable

2023-09-23 Thread Haiyang Hu (Jira)
[ https://issues.apache.org/jira/browse/HDFS-17205?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Haiyang Hu updated HDFS-17205: -- Description: Current allocate new block , the NameNode will choose datanode and choose a good storage

[jira] [Created] (HDFS-17208) Add the metrics PendingAsyncDiskOperations in datanode

2023-09-23 Thread Haiyang Hu (Jira)
Haiyang Hu created HDFS-17208: - Summary: Add the metrics PendingAsyncDiskOperations in datanode Key: HDFS-17208 URL: https://issues.apache.org/jira/browse/HDFS-17208 Project: Hadoop HDFS Issue

[jira] [Updated] (HDFS-17208) Add the metrics PendingAsyncDiskOperations in datanode

2023-09-23 Thread Haiyang Hu (Jira)
[ https://issues.apache.org/jira/browse/HDFS-17208?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Haiyang Hu updated HDFS-17208: -- Description: Consider should add the metrics `PendingAsyncDiskOperations` to be able to track if we

[jira] [Commented] (HDFS-17218) NameNode should remove its excess blocks from the ExcessRedundancyMap When a DN registers

2023-10-11 Thread Haiyang Hu (Jira)
[ https://issues.apache.org/jira/browse/HDFS-17218?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17774302#comment-17774302 ] Haiyang Hu commented on HDFS-17218: --- Hi [~hexiaoqiao] Sir do you have any other comments or suggestions

[jira] [Commented] (HDFS-17218) NameNode should remove its excess blocks from the ExcessRedundancyMap When a DN registers

2023-10-12 Thread Haiyang Hu (Jira)
[ https://issues.apache.org/jira/browse/HDFS-17218?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17774492#comment-17774492 ] Haiyang Hu commented on HDFS-17218: --- Thanks [~zhangshuyan] for you comment. {quote} I think the root

[jira] [Commented] (HDFS-17218) NameNode should remove its excess blocks from the ExcessRedundancyMap When a DN registers

2023-10-09 Thread Haiyang Hu (Jira)
[ https://issues.apache.org/jira/browse/HDFS-17218?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17773348#comment-17773348 ] Haiyang Hu commented on HDFS-17218: --- Hi Sir [~hexiaoqiao] thanks for you comment. The problem currently

[jira] [Updated] (HDFS-17235) Fix javadoc errors in BlockManager

2023-10-22 Thread Haiyang Hu (Jira)
[ https://issues.apache.org/jira/browse/HDFS-17235?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Haiyang Hu updated HDFS-17235: -- Description: There are 2 errors in BlockManager.java

[jira] [Created] (HDFS-17235) Fix javadoc errors in BlockManager

2023-10-22 Thread Haiyang Hu (Jira)
Haiyang Hu created HDFS-17235: - Summary: Fix javadoc errors in BlockManager Key: HDFS-17235 URL: https://issues.apache.org/jira/browse/HDFS-17235 Project: Hadoop HDFS Issue Type: Improvement

[jira] [Comment Edited] (HDFS-17218) NameNode should remove its excess blocks from the ExcessRedundancyMap When a DN registers

2023-10-09 Thread Haiyang Hu (Jira)
[ https://issues.apache.org/jira/browse/HDFS-17218?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17773525#comment-17773525 ] Haiyang Hu edited comment on HDFS-17218 at 10/10/23 3:59 AM: - Sure, attach

[jira] [Created] (HDFS-17217) Add lifeline RPC start up log when NameNode#startCommonServices

2023-10-06 Thread Haiyang Hu (Jira)
Haiyang Hu created HDFS-17217: - Summary: Add lifeline RPC start up log when NameNode#startCommonServices Key: HDFS-17217 URL: https://issues.apache.org/jira/browse/HDFS-17217 Project: Hadoop HDFS

[jira] [Comment Edited] (HDFS-17218) NameNode should remove its excess blocks from the ExcessRedundancyMap When a DN registers

2023-10-10 Thread Haiyang Hu (Jira)
[ https://issues.apache.org/jira/browse/HDFS-17218?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17773563#comment-17773563 ] Haiyang Hu edited comment on HDFS-17218 at 10/10/23 6:48 AM: - Sure, attach

[jira] [Commented] (HDFS-17218) NameNode should remove its excess blocks from the ExcessRedundancyMap When a DN registers

2023-10-10 Thread Haiyang Hu (Jira)
[ https://issues.apache.org/jira/browse/HDFS-17218?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17773563#comment-17773563 ] Haiyang Hu commented on HDFS-17218: --- Sure, attach code snippet that shows the current implements :

[jira] [Commented] (HDFS-17218) NameNode should remove its excess blocks from the ExcessRedundancyMap When a DN registers

2023-10-10 Thread Haiyang Hu (Jira)
[ https://issues.apache.org/jira/browse/HDFS-17218?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17773564#comment-17773564 ] Haiyang Hu commented on HDFS-17218: --- {quote} 7. so the block of dn1 will always exist in

[jira] [Commented] (HDFS-17218) NameNode should remove its excess blocks from the ExcessRedundancyMap When a DN registers

2023-10-10 Thread Haiyang Hu (Jira)
[ https://issues.apache.org/jira/browse/HDFS-17218?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17773573#comment-17773573 ] Haiyang Hu commented on HDFS-17218: --- This test UT maybe can reproduce this case. {code:java}

  1   2   3   4   >