[jira] [Updated] (HDFS-16241) Standby close reconstruction thread
[ https://issues.apache.org/jira/browse/HDFS-16241?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] zhanghuazong updated HDFS-16241: Attachment: HDFS-16241 > Standby close reconstruction thread > --- > > Key: HDFS-16241 > URL: https://issues.apache.org/jira/browse/HDFS-16241 > Project: Hadoop HDFS > Issue Type: Improvement >Reporter: zhanghuazong >Assignee: zhanghuazong >Priority: Major > Attachments: HDFS-16241 > > > When the "Reconstruction Queue Initializer" thread of the active namenode has > not stopped, switch to standby namenode. The "Reconstruction Queue > Initializer" thread should be closed -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Updated] (HDFS-16241) Standby close reconstruction thread
[ https://issues.apache.org/jira/browse/HDFS-16241?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] zhanghuazong updated HDFS-16241: Summary: Standby close reconstruction thread (was: Standby closes reconstruction thread) > Standby close reconstruction thread > --- > > Key: HDFS-16241 > URL: https://issues.apache.org/jira/browse/HDFS-16241 > Project: Hadoop HDFS > Issue Type: Improvement >Reporter: zhanghuazong >Assignee: zhanghuazong >Priority: Major > > When the "Reconstruction Queue Initializer" thread of the active namenode has > not stopped, switch to standby namenode. The "Reconstruction Queue > Initializer" thread should be closed -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Updated] (HDFS-16241) Standby closes reconstruction thread
[ https://issues.apache.org/jira/browse/HDFS-16241?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] zhanghuazong updated HDFS-16241: Description: When the "Reconstruction Queue Initializer" thread of the active namenode has not stopped, switch to standby namenode. The "Reconstruction Queue Initializer" thread should be closed (was: Standby closes reconstruction thread) > Standby closes reconstruction thread > > > Key: HDFS-16241 > URL: https://issues.apache.org/jira/browse/HDFS-16241 > Project: Hadoop HDFS > Issue Type: Improvement >Reporter: zhanghuazong >Assignee: zhanghuazong >Priority: Major > > When the "Reconstruction Queue Initializer" thread of the active namenode has > not stopped, switch to standby namenode. The "Reconstruction Queue > Initializer" thread should be closed -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Created] (HDFS-16241) Standby closes reconstruction thread
zhanghuazong created HDFS-16241: --- Summary: Standby closes reconstruction thread Key: HDFS-16241 URL: https://issues.apache.org/jira/browse/HDFS-16241 Project: Hadoop HDFS Issue Type: Improvement Reporter: zhanghuazong Assignee: zhanghuazong Standby closes reconstruction thread -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-15993) INodesInPath#toString() will throw AssertionError.
[ https://issues.apache.org/jira/browse/HDFS-15993?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17355450#comment-17355450 ] zhanghuazong commented on HDFS-15993: - [~ferhui] Dear HuiFei, this patch can help review this patch, and there is another patch (https://issues.apache.org/jira/browse/HDFS-15981) > INodesInPath#toString() will throw AssertionError. > -- > > Key: HDFS-15993 > URL: https://issues.apache.org/jira/browse/HDFS-15993 > Project: Hadoop HDFS > Issue Type: Improvement > Components: hdfs >Reporter: zhanghuazong >Assignee: zhanghuazong >Priority: Major > Labels: pull-request-available > Attachments: HDFS-15993.patch > > Time Spent: 0.5h > Remaining Estimate: 0h > > In the case of a snapshot, INodesInpath#toString() will throw an > AssertionError -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-15966) Empty the statistical parameters when emptying the redundant queue
[ https://issues.apache.org/jira/browse/HDFS-15966?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17355440#comment-17355440 ] zhanghuazong commented on HDFS-15966: - [~ferhui]Thank you very much Hui Fei, I have re-operated according to your suggestion. > Empty the statistical parameters when emptying the redundant queue > -- > > Key: HDFS-15966 > URL: https://issues.apache.org/jira/browse/HDFS-15966 > Project: Hadoop HDFS > Issue Type: Improvement > Components: hdfs >Reporter: zhanghuazong >Assignee: zhanghuazong >Priority: Minor > Labels: pull-request-available > Attachments: HDFS-15966.patch > > Time Spent: 40m > Remaining Estimate: 0h > > Clear the two indicators highestPriorityLowRedundancyReplicatedBlocks and > highestPriorityLowRedundancyReplicatedBlocks when emptying the redundant > queue. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-15966) Empty the statistical parameters when emptying the redundant queue
[ https://issues.apache.org/jira/browse/HDFS-15966?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17354940#comment-17354940 ] zhanghuazong commented on HDFS-15966: - [~ferhui] Dear HuiFei, can you please review. > Empty the statistical parameters when emptying the redundant queue > -- > > Key: HDFS-15966 > URL: https://issues.apache.org/jira/browse/HDFS-15966 > Project: Hadoop HDFS > Issue Type: Improvement > Components: hdfs >Reporter: zhanghuazong >Assignee: zhanghuazong >Priority: Minor > Labels: pull-request-available > Attachments: HDFS-15966.patch > > Time Spent: 10m > Remaining Estimate: 0h > > Clear the two indicators highestPriorityLowRedundancyReplicatedBlocks and > highestPriorityLowRedundancyReplicatedBlocks when emptying the redundant > queue. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Reopened] (HDFS-15981) Removing redundant block queues will slow down block reporting
[ https://issues.apache.org/jira/browse/HDFS-15981?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] zhanghuazong reopened HDFS-15981: - > Removing redundant block queues will slow down block reporting > -- > > Key: HDFS-15981 > URL: https://issues.apache.org/jira/browse/HDFS-15981 > Project: Hadoop HDFS > Issue Type: Improvement > Components: hdfs >Reporter: zhanghuazong >Assignee: zhanghuazong >Priority: Major > Labels: pull-request-available > Attachments: HDFS-15981.patch > > Time Spent: 20m > Remaining Estimate: 0h > > When the block report satisfies the block distribution strategy, the block is > removed from the lowredundancyBlocks. But removing the block from the > lowredundancyBlocks is a redundant operation. > First, in the patch queue, the block removal operation will be performed in > the method chooseSourceDatanodes and validateReconstructionWork. > second, the removal of the block report will only be at the > QUEUE_REPLICAS_BADLY_DISTRIBUTED level, which is not an accurate operation. > Finally, when there is a large amount of data in the > QUEUE_REPLICAS_BADLY_DISTRIBUTED queue, the processing efficiency of the > block report will be reduced -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Reopened] (HDFS-15993) INodesInPath#toString() will throw AssertionError.
[ https://issues.apache.org/jira/browse/HDFS-15993?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] zhanghuazong reopened HDFS-15993: - > INodesInPath#toString() will throw AssertionError. > -- > > Key: HDFS-15993 > URL: https://issues.apache.org/jira/browse/HDFS-15993 > Project: Hadoop HDFS > Issue Type: Improvement > Components: hdfs >Reporter: zhanghuazong >Assignee: zhanghuazong >Priority: Major > Labels: pull-request-available > Attachments: HDFS-15993.patch > > Time Spent: 20m > Remaining Estimate: 0h > > In the case of a snapshot, INodesInpath#toString() will throw an > AssertionError -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Reopened] (HDFS-15966) Empty the statistical parameters when emptying the redundant queue
[ https://issues.apache.org/jira/browse/HDFS-15966?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] zhanghuazong reopened HDFS-15966: - > Empty the statistical parameters when emptying the redundant queue > -- > > Key: HDFS-15966 > URL: https://issues.apache.org/jira/browse/HDFS-15966 > Project: Hadoop HDFS > Issue Type: Improvement > Components: hdfs >Reporter: zhanghuazong >Assignee: zhanghuazong >Priority: Minor > Labels: pull-request-available > Attachments: HDFS-15966.patch > > Time Spent: 10m > Remaining Estimate: 0h > > Clear the two indicators highestPriorityLowRedundancyReplicatedBlocks and > highestPriorityLowRedundancyReplicatedBlocks when emptying the redundant > queue. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Resolved] (HDFS-15993) INodesInPath#toString() will throw AssertionError.
[ https://issues.apache.org/jira/browse/HDFS-15993?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] zhanghuazong resolved HDFS-15993. - Resolution: Fixed > INodesInPath#toString() will throw AssertionError. > -- > > Key: HDFS-15993 > URL: https://issues.apache.org/jira/browse/HDFS-15993 > Project: Hadoop HDFS > Issue Type: Improvement > Components: hdfs >Reporter: zhanghuazong >Assignee: zhanghuazong >Priority: Major > Labels: pull-request-available > Attachments: HDFS-15993.patch > > Time Spent: 10m > Remaining Estimate: 0h > > In the case of a snapshot, INodesInpath#toString() will throw an > AssertionError -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Updated] (HDFS-15993) INodesInPath#toString() will throw AssertionError.
[ https://issues.apache.org/jira/browse/HDFS-15993?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] zhanghuazong updated HDFS-15993: Attachment: HDFS-15993.patch > INodesInPath#toString() will throw AssertionError. > -- > > Key: HDFS-15993 > URL: https://issues.apache.org/jira/browse/HDFS-15993 > Project: Hadoop HDFS > Issue Type: Improvement > Components: hdfs >Reporter: zhanghuazong >Assignee: zhanghuazong >Priority: Major > Labels: pull-request-available > Attachments: HDFS-15993.patch > > Time Spent: 10m > Remaining Estimate: 0h > > In the case of a snapshot, INodesInpath#toString() will throw an > AssertionError -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Work started] (HDFS-15993) INodesInPath#toString() will throw AssertionError.
[ https://issues.apache.org/jira/browse/HDFS-15993?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Work on HDFS-15993 started by zhanghuazong. --- > INodesInPath#toString() will throw AssertionError. > -- > > Key: HDFS-15993 > URL: https://issues.apache.org/jira/browse/HDFS-15993 > Project: Hadoop HDFS > Issue Type: Improvement > Components: hdfs >Reporter: zhanghuazong >Assignee: zhanghuazong >Priority: Major > > In the case of a snapshot, INodesInpath#toString() will throw an > AssertionError -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Created] (HDFS-15993) INodesInPath#toString() will throw AssertionError.
zhanghuazong created HDFS-15993: --- Summary: INodesInPath#toString() will throw AssertionError. Key: HDFS-15993 URL: https://issues.apache.org/jira/browse/HDFS-15993 Project: Hadoop HDFS Issue Type: Improvement Components: hdfs Reporter: zhanghuazong Assignee: zhanghuazong In the case of a snapshot, INodesInpath#toString() will throw an AssertionError -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Resolved] (HDFS-15981) Removing redundant block queues will slow down block reporting
[ https://issues.apache.org/jira/browse/HDFS-15981?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] zhanghuazong resolved HDFS-15981. - Resolution: Fixed > Removing redundant block queues will slow down block reporting > -- > > Key: HDFS-15981 > URL: https://issues.apache.org/jira/browse/HDFS-15981 > Project: Hadoop HDFS > Issue Type: Improvement > Components: hdfs >Reporter: zhanghuazong >Assignee: zhanghuazong >Priority: Major > Labels: pull-request-available > Attachments: HDFS-15981.patch > > Time Spent: 10m > Remaining Estimate: 0h > > When the block report satisfies the block distribution strategy, the block is > removed from the lowredundancyBlocks. But removing the block from the > lowredundancyBlocks is a redundant operation. > First, in the patch queue, the block removal operation will be performed in > the method chooseSourceDatanodes and validateReconstructionWork. > second, the removal of the block report will only be at the > QUEUE_REPLICAS_BADLY_DISTRIBUTED level, which is not an accurate operation. > Finally, when there is a large amount of data in the > QUEUE_REPLICAS_BADLY_DISTRIBUTED queue, the processing efficiency of the > block report will be reduced -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Updated] (HDFS-15981) Removing redundant block queues will slow down block reporting
[ https://issues.apache.org/jira/browse/HDFS-15981?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] zhanghuazong updated HDFS-15981: Attachment: (was: HDFS-15966.patch) > Removing redundant block queues will slow down block reporting > -- > > Key: HDFS-15981 > URL: https://issues.apache.org/jira/browse/HDFS-15981 > Project: Hadoop HDFS > Issue Type: Improvement > Components: hdfs >Reporter: zhanghuazong >Assignee: zhanghuazong >Priority: Major > Labels: pull-request-available > Attachments: HDFS-15981.patch > > Time Spent: 10m > Remaining Estimate: 0h > > When the block report satisfies the block distribution strategy, the block is > removed from the lowredundancyBlocks. But removing the block from the > lowredundancyBlocks is a redundant operation. > First, in the patch queue, the block removal operation will be performed in > the method chooseSourceDatanodes and validateReconstructionWork. > second, the removal of the block report will only be at the > QUEUE_REPLICAS_BADLY_DISTRIBUTED level, which is not an accurate operation. > Finally, when there is a large amount of data in the > QUEUE_REPLICAS_BADLY_DISTRIBUTED queue, the processing efficiency of the > block report will be reduced -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Updated] (HDFS-15981) Removing redundant block queues will slow down block reporting
[ https://issues.apache.org/jira/browse/HDFS-15981?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] zhanghuazong updated HDFS-15981: Attachment: HDFS-15981.patch > Removing redundant block queues will slow down block reporting > -- > > Key: HDFS-15981 > URL: https://issues.apache.org/jira/browse/HDFS-15981 > Project: Hadoop HDFS > Issue Type: Improvement > Components: hdfs >Reporter: zhanghuazong >Assignee: zhanghuazong >Priority: Major > Labels: pull-request-available > Attachments: HDFS-15981.patch > > Time Spent: 10m > Remaining Estimate: 0h > > When the block report satisfies the block distribution strategy, the block is > removed from the lowredundancyBlocks. But removing the block from the > lowredundancyBlocks is a redundant operation. > First, in the patch queue, the block removal operation will be performed in > the method chooseSourceDatanodes and validateReconstructionWork. > second, the removal of the block report will only be at the > QUEUE_REPLICAS_BADLY_DISTRIBUTED level, which is not an accurate operation. > Finally, when there is a large amount of data in the > QUEUE_REPLICAS_BADLY_DISTRIBUTED queue, the processing efficiency of the > block report will be reduced -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Updated] (HDFS-15981) Removing redundant block queues will slow down block reporting
[ https://issues.apache.org/jira/browse/HDFS-15981?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] zhanghuazong updated HDFS-15981: Attachment: HDFS-15966.patch > Removing redundant block queues will slow down block reporting > -- > > Key: HDFS-15981 > URL: https://issues.apache.org/jira/browse/HDFS-15981 > Project: Hadoop HDFS > Issue Type: Improvement > Components: hdfs >Reporter: zhanghuazong >Assignee: zhanghuazong >Priority: Major > Labels: pull-request-available > Time Spent: 10m > Remaining Estimate: 0h > > When the block report satisfies the block distribution strategy, the block is > removed from the lowredundancyBlocks. But removing the block from the > lowredundancyBlocks is a redundant operation. > First, in the patch queue, the block removal operation will be performed in > the method chooseSourceDatanodes and validateReconstructionWork. > second, the removal of the block report will only be at the > QUEUE_REPLICAS_BADLY_DISTRIBUTED level, which is not an accurate operation. > Finally, when there is a large amount of data in the > QUEUE_REPLICAS_BADLY_DISTRIBUTED queue, the processing efficiency of the > block report will be reduced -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Work stopped] (HDFS-15981) Removing redundant block queues will slow down block reporting
[ https://issues.apache.org/jira/browse/HDFS-15981?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Work on HDFS-15981 stopped by zhanghuazong. --- > Removing redundant block queues will slow down block reporting > -- > > Key: HDFS-15981 > URL: https://issues.apache.org/jira/browse/HDFS-15981 > Project: Hadoop HDFS > Issue Type: Improvement > Components: hdfs >Reporter: zhanghuazong >Assignee: zhanghuazong >Priority: Major > > When the block report satisfies the block distribution strategy, the block is > removed from the lowredundancyBlocks. But removing the block from the > lowredundancyBlocks is a redundant operation. > First, in the patch queue, the block removal operation will be performed in > the method chooseSourceDatanodes and validateReconstructionWork. > second, the removal of the block report will only be at the > QUEUE_REPLICAS_BADLY_DISTRIBUTED level, which is not an accurate operation. > Finally, when there is a large amount of data in the > QUEUE_REPLICAS_BADLY_DISTRIBUTED queue, the processing efficiency of the > block report will be reduced -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Work started] (HDFS-15981) Removing redundant block queues will slow down block reporting
[ https://issues.apache.org/jira/browse/HDFS-15981?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Work on HDFS-15981 started by zhanghuazong. --- > Removing redundant block queues will slow down block reporting > -- > > Key: HDFS-15981 > URL: https://issues.apache.org/jira/browse/HDFS-15981 > Project: Hadoop HDFS > Issue Type: Improvement > Components: hdfs >Reporter: zhanghuazong >Assignee: zhanghuazong >Priority: Major > > When the block report satisfies the block distribution strategy, the block is > removed from the lowredundancyBlocks. But removing the block from the > lowredundancyBlocks is a redundant operation. > First, in the patch queue, the block removal operation will be performed in > the method chooseSourceDatanodes and validateReconstructionWork. > second, the removal of the block report will only be at the > QUEUE_REPLICAS_BADLY_DISTRIBUTED level, which is not an accurate operation. > Finally, when there is a large amount of data in the > QUEUE_REPLICAS_BADLY_DISTRIBUTED queue, the processing efficiency of the > block report will be reduced -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Updated] (HDFS-15981) Removing redundant block queues will slow down block reporting
[ https://issues.apache.org/jira/browse/HDFS-15981?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] zhanghuazong updated HDFS-15981: Description: When the block report satisfies the block distribution strategy, the block is removed from the lowredundancyBlocks. But removing the block from the lowredundancyBlocks is a redundant operation. First, in the patch queue, the block removal operation will be performed in the method chooseSourceDatanodes and validateReconstructionWork. second, the removal of the block report will only be at the QUEUE_REPLICAS_BADLY_DISTRIBUTED level, which is not an accurate operation. Finally, when there is a large amount of data in the QUEUE_REPLICAS_BADLY_DISTRIBUTED queue, the processing efficiency of the block report will be reduced was: When the block report satisfies the block distribution strategy, the block is removed from the lowredundancyBlocks. But removing the block from the lowredundancyBlocks is a redundant operation. First, in the patch queue, the block removal operation will be performed in the method chooseSourceDatanodes and validateReconstructionWork. second, the removal of the block report will only be at the QUEUE_REPLICAS_BADLY_DISTRIBUTED level, which is not an accurate operation. > Removing redundant block queues will slow down block reporting > -- > > Key: HDFS-15981 > URL: https://issues.apache.org/jira/browse/HDFS-15981 > Project: Hadoop HDFS > Issue Type: Improvement > Components: hdfs >Reporter: zhanghuazong >Assignee: zhanghuazong >Priority: Major > > When the block report satisfies the block distribution strategy, the block is > removed from the lowredundancyBlocks. But removing the block from the > lowredundancyBlocks is a redundant operation. > First, in the patch queue, the block removal operation will be performed in > the method chooseSourceDatanodes and validateReconstructionWork. > second, the removal of the block report will only be at the > QUEUE_REPLICAS_BADLY_DISTRIBUTED level, which is not an accurate operation. > Finally, when there is a large amount of data in the > QUEUE_REPLICAS_BADLY_DISTRIBUTED queue, the processing efficiency of the > block report will be reduced -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Updated] (HDFS-15981) Removing redundant block queues will slow down block reporting
[ https://issues.apache.org/jira/browse/HDFS-15981?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] zhanghuazong updated HDFS-15981: Description: When the block report satisfies the block distribution strategy, the block is removed from the lowredundancyBlocks. But removing the block from the lowredundancyBlocks is a redundant operation. First, in the patch queue, the block removal operation will be performed in the method chooseSourceDatanodes and validateReconstructionWork. second, the removal of the block report will only be at the QUEUE_REPLICAS_BADLY_DISTRIBUTED level, which is not an accurate operation. > Removing redundant block queues will slow down block reporting > -- > > Key: HDFS-15981 > URL: https://issues.apache.org/jira/browse/HDFS-15981 > Project: Hadoop HDFS > Issue Type: Improvement > Components: hdfs >Reporter: zhanghuazong >Assignee: zhanghuazong >Priority: Major > > When the block report satisfies the block distribution strategy, the block is > removed from the lowredundancyBlocks. But removing the block from the > lowredundancyBlocks is a redundant operation. > First, in the patch queue, the block removal operation will be performed in > the method chooseSourceDatanodes and validateReconstructionWork. > second, the removal of the block report will only be at the > QUEUE_REPLICAS_BADLY_DISTRIBUTED level, which is not an accurate operation. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Updated] (HDFS-15966) Empty the statistical parameters when emptying the redundant queue
[ https://issues.apache.org/jira/browse/HDFS-15966?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] zhanghuazong updated HDFS-15966: Attachment: HDFS-15966.patch > Empty the statistical parameters when emptying the redundant queue > -- > > Key: HDFS-15966 > URL: https://issues.apache.org/jira/browse/HDFS-15966 > Project: Hadoop HDFS > Issue Type: Improvement > Components: hdfs >Reporter: zhanghuazong >Assignee: zhanghuazong >Priority: Minor > Labels: pull-request-available > Attachments: HDFS-15966.patch > > Time Spent: 10m > Remaining Estimate: 0h > > Clear the two indicators highestPriorityLowRedundancyReplicatedBlocks and > highestPriorityLowRedundancyReplicatedBlocks when emptying the redundant > queue. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Created] (HDFS-15981) Removing redundant block queues will slow down block reporting
zhanghuazong created HDFS-15981: --- Summary: Removing redundant block queues will slow down block reporting Key: HDFS-15981 URL: https://issues.apache.org/jira/browse/HDFS-15981 Project: Hadoop HDFS Issue Type: Improvement Components: hdfs Reporter: zhanghuazong Assignee: zhanghuazong -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Resolved] (HDFS-15966) Empty the statistical parameters when emptying the redundant queue
[ https://issues.apache.org/jira/browse/HDFS-15966?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] zhanghuazong resolved HDFS-15966. - Resolution: Fixed > Empty the statistical parameters when emptying the redundant queue > -- > > Key: HDFS-15966 > URL: https://issues.apache.org/jira/browse/HDFS-15966 > Project: Hadoop HDFS > Issue Type: Improvement > Components: hdfs >Reporter: zhanghuazong >Assignee: zhanghuazong >Priority: Minor > Labels: pull-request-available > Time Spent: 10m > Remaining Estimate: 0h > > Clear the two indicators highestPriorityLowRedundancyReplicatedBlocks and > highestPriorityLowRedundancyReplicatedBlocks when emptying the redundant > queue. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Assigned] (HDFS-15966) Empty the statistical parameters when emptying the redundant queue
[ https://issues.apache.org/jira/browse/HDFS-15966?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] zhanghuazong reassigned HDFS-15966: --- Assignee: zhanghuazong > Empty the statistical parameters when emptying the redundant queue > -- > > Key: HDFS-15966 > URL: https://issues.apache.org/jira/browse/HDFS-15966 > Project: Hadoop HDFS > Issue Type: Improvement > Components: hdfs >Reporter: zhanghuazong >Assignee: zhanghuazong >Priority: Minor > Labels: pull-request-available > Time Spent: 10m > Remaining Estimate: 0h > > Clear the two indicators highestPriorityLowRedundancyReplicatedBlocks and > highestPriorityLowRedundancyReplicatedBlocks when emptying the redundant > queue. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Updated] (HDFS-15966) Empty the statistical parameters when emptying the redundant queue
[ https://issues.apache.org/jira/browse/HDFS-15966?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] zhanghuazong updated HDFS-15966: Description: Clear the two indicators highestPriorityLowRedundancyReplicatedBlocks and highestPriorityLowRedundancyReplicatedBlocks when emptying the redundant queue. (was: Clear the two indicators highestPriorityLowRedundancyReplicatedBlocks and highestPriorityLowRedundancyReplicatedBlocks when emptying the redundant queue,) > Empty the statistical parameters when emptying the redundant queue > -- > > Key: HDFS-15966 > URL: https://issues.apache.org/jira/browse/HDFS-15966 > Project: Hadoop HDFS > Issue Type: Improvement > Components: hdfs >Reporter: zhanghuazong >Priority: Minor > > Clear the two indicators highestPriorityLowRedundancyReplicatedBlocks and > highestPriorityLowRedundancyReplicatedBlocks when emptying the redundant > queue. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Updated] (HDFS-15966) Empty the statistical parameters when emptying the redundant queue
[ https://issues.apache.org/jira/browse/HDFS-15966?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] zhanghuazong updated HDFS-15966: Summary: Empty the statistical parameters when emptying the redundant queue (was: Empty the statistical parameters when emptying the redundant queue, ) > Empty the statistical parameters when emptying the redundant queue > -- > > Key: HDFS-15966 > URL: https://issues.apache.org/jira/browse/HDFS-15966 > Project: Hadoop HDFS > Issue Type: Improvement > Components: hdfs >Reporter: zhanghuazong >Priority: Minor > > Clear the two indicators highestPriorityLowRedundancyReplicatedBlocks and > highestPriorityLowRedundancyReplicatedBlocks when emptying the redundant > queue, -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Created] (HDFS-15966) Empty the statistical parameters when emptying the redundant queue,
zhanghuazong created HDFS-15966: --- Summary: Empty the statistical parameters when emptying the redundant queue, Key: HDFS-15966 URL: https://issues.apache.org/jira/browse/HDFS-15966 Project: Hadoop HDFS Issue Type: Improvement Components: hdfs Reporter: zhanghuazong Clear the two indicators highestPriorityLowRedundancyReplicatedBlocks and highestPriorityLowRedundancyReplicatedBlocks when emptying the redundant queue, -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Updated] (HDFS-15739) Missing Javadoc for a param in DFSNetworkTopology
[ https://issues.apache.org/jira/browse/HDFS-15739?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] zhanghuazong updated HDFS-15739: Attachment: (was: HDFS-15739.0.patch) > Missing Javadoc for a param in DFSNetworkTopology > - > > Key: HDFS-15739 > URL: https://issues.apache.org/jira/browse/HDFS-15739 > Project: Hadoop HDFS > Issue Type: Bug > Components: hdfs >Reporter: zhanghuazong >Priority: Minor > Labels: pull-request-available > Attachments: HDFS-15739.0.patch > > Time Spent: 50m > Remaining Estimate: 0h > > Only add missing Javadoc for a param in method chooseRandomWithStorageType > of DFSNetworkTopology.java. > -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Updated] (HDFS-15739) Missing Javadoc for a param in DFSNetworkTopology
[ https://issues.apache.org/jira/browse/HDFS-15739?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] zhanghuazong updated HDFS-15739: Attachment: HDFS-15739.0.patch > Missing Javadoc for a param in DFSNetworkTopology > - > > Key: HDFS-15739 > URL: https://issues.apache.org/jira/browse/HDFS-15739 > Project: Hadoop HDFS > Issue Type: Bug > Components: hdfs >Reporter: zhanghuazong >Priority: Minor > Labels: pull-request-available > Attachments: HDFS-15739.0.patch > > Time Spent: 50m > Remaining Estimate: 0h > > Only add missing Javadoc for a param in method chooseRandomWithStorageType > of DFSNetworkTopology.java. > -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Updated] (HDFS-15739) Missing Javadoc for a param in DFSNetworkTopology
[ https://issues.apache.org/jira/browse/HDFS-15739?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] zhanghuazong updated HDFS-15739: Description: Only add missing Javadoc for a param in method chooseRandomWithStorageType of DFSNetworkTopology.java. was: Only add missing Javadoc for a param in method chooseRandomWithStorageType of AppSchedulingInfo.java. > Missing Javadoc for a param in DFSNetworkTopology > - > > Key: HDFS-15739 > URL: https://issues.apache.org/jira/browse/HDFS-15739 > Project: Hadoop HDFS > Issue Type: Bug > Components: hdfs >Reporter: zhanghuazong >Priority: Minor > Labels: pull-request-available > Attachments: HDFS-15739.0.patch > > Time Spent: 0.5h > Remaining Estimate: 0h > > Only add missing Javadoc for a param in method chooseRandomWithStorageType > of DFSNetworkTopology.java. > -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Reopened] (HDFS-15739) Missing Javadoc for a param in DFSNetworkTopology
[ https://issues.apache.org/jira/browse/HDFS-15739?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] zhanghuazong reopened HDFS-15739: - > Missing Javadoc for a param in DFSNetworkTopology > - > > Key: HDFS-15739 > URL: https://issues.apache.org/jira/browse/HDFS-15739 > Project: Hadoop HDFS > Issue Type: Bug > Components: hdfs >Reporter: zhanghuazong >Priority: Minor > Labels: pull-request-available > Attachments: HDFS-15739.0.patch > > Time Spent: 10m > Remaining Estimate: 0h > > Only add missing Javadoc for a param in method chooseRandomWithStorageType > of AppSchedulingInfo.java. > -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Updated] (HDFS-15739) Missing Javadoc for a param in DFSNetworkTopology
[ https://issues.apache.org/jira/browse/HDFS-15739?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] zhanghuazong updated HDFS-15739: Summary: Missing Javadoc for a param in DFSNetworkTopology (was: Missing Javadoc for a param in org.apache.hadoop.hdfs.net.DFSNetworkTopology) > Missing Javadoc for a param in DFSNetworkTopology > - > > Key: HDFS-15739 > URL: https://issues.apache.org/jira/browse/HDFS-15739 > Project: Hadoop HDFS > Issue Type: Bug > Components: hdfs >Reporter: zhanghuazong >Priority: Minor > Labels: pull-request-available > Attachments: HDFS-15739.0.patch > > Time Spent: 10m > Remaining Estimate: 0h > > Only add missing Javadoc for a param in method chooseRandomWithStorageType > of AppSchedulingInfo.java. > -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Resolved] (HDFS-15739) Missing Javadoc for a param in org.apache.hadoop.hdfs.net.DFSNetworkTopology
[ https://issues.apache.org/jira/browse/HDFS-15739?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] zhanghuazong resolved HDFS-15739. - Resolution: Fixed > Missing Javadoc for a param in org.apache.hadoop.hdfs.net.DFSNetworkTopology > > > Key: HDFS-15739 > URL: https://issues.apache.org/jira/browse/HDFS-15739 > Project: Hadoop HDFS > Issue Type: Bug > Components: hdfs >Reporter: zhanghuazong >Priority: Minor > Labels: pull-request-available > Attachments: HDFS-15739.0.patch > > Time Spent: 10m > Remaining Estimate: 0h > > Only add missing Javadoc for a param in method chooseRandomWithStorageType > of AppSchedulingInfo.java. > -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Updated] (HDFS-15739) Missing Javadoc for a param in org.apache.hadoop.hdfs.net.DFSNetworkTopology
[ https://issues.apache.org/jira/browse/HDFS-15739?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] zhanghuazong updated HDFS-15739: Attachment: HDFS-15739.0.patch > Missing Javadoc for a param in org.apache.hadoop.hdfs.net.DFSNetworkTopology > > > Key: HDFS-15739 > URL: https://issues.apache.org/jira/browse/HDFS-15739 > Project: Hadoop HDFS > Issue Type: Bug > Components: hdfs >Reporter: zhanghuazong >Priority: Minor > Labels: pull-request-available > Attachments: HDFS-15739.0.patch > > Time Spent: 10m > Remaining Estimate: 0h > > Only add missing Javadoc for a param in method chooseRandomWithStorageType > of AppSchedulingInfo.java. > -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Updated] (HDFS-15739) Missing Javadoc for a param in org.apache.hadoop.hdfs.net.DFSNetworkTopology
[ https://issues.apache.org/jira/browse/HDFS-15739?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] zhanghuazong updated HDFS-15739: Summary: Missing Javadoc for a param in org.apache.hadoop.hdfs.net.DFSNetworkTopology (was: Fix comment in org.apache.hadoop.hdfs.net.DFSNetworkTopology) > Missing Javadoc for a param in org.apache.hadoop.hdfs.net.DFSNetworkTopology > > > Key: HDFS-15739 > URL: https://issues.apache.org/jira/browse/HDFS-15739 > Project: Hadoop HDFS > Issue Type: Bug > Components: hdfs >Reporter: zhanghuazong >Priority: Minor > > Only add missing Javadoc for a param in method chooseRandomWithStorageType > of AppSchedulingInfo.java. > -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Created] (HDFS-15739) Fix comment in org.apache.hadoop.hdfs.net.DFSNetworkTopology
zhanghuazong created HDFS-15739: --- Summary: Fix comment in org.apache.hadoop.hdfs.net.DFSNetworkTopology Key: HDFS-15739 URL: https://issues.apache.org/jira/browse/HDFS-15739 Project: Hadoop HDFS Issue Type: Bug Components: hdfs Reporter: zhanghuazong Only add missing Javadoc for a param in method chooseRandomWithStorageType of AppSchedulingInfo.java. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
