[jira] [Comment Edited] (HDFS-15237) Get checksum of EC file failed, when some block is missing or corrupt

2020-03-26 Thread guojh (Jira)
[ https://issues.apache.org/jira/browse/HDFS-15237?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17068263#comment-17068263 ] guojh edited comment on HDFS-15237 at 3/27/20, 4:42 AM: [~zhengch

[jira] [Commented] (HDFS-15237) Get checksum of EC file failed, when some block is missing or corrupt

2020-03-26 Thread guojh (Jira)
[ https://issues.apache.org/jira/browse/HDFS-15237?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17068263#comment-17068263 ] guojh commented on HDFS-15237: -- [~zhengchenyu] Have you try this patch (https://issues.apac

[jira] [Commented] (HDFS-15240) Erasure Coding: dirty buffer causes reconstruction block error

2020-03-26 Thread guojh (Jira)
[ https://issues.apache.org/jira/browse/HDFS-15240?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17067559#comment-17067559 ] guojh commented on HDFS-15240: -- [~surendrasingh]  We have the same problem,please take a loo

[jira] [Commented] (HDFS-15186) Erasure Coding: Decommission may generate the parity block's content with all 0 in some case

2020-02-23 Thread guojh (Jira)
[ https://issues.apache.org/jira/browse/HDFS-15186?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17043125#comment-17043125 ] guojh commented on HDFS-15186: -- [~ayushtkn] This problem can fixed in namenode side, I have

[jira] [Commented] (HDFS-14768) EC : Busy DN replica should be consider in live replica check.

2019-11-03 Thread guojh (Jira)
[ https://issues.apache.org/jira/browse/HDFS-14768?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16966312#comment-16966312 ] guojh commented on HDFS-14768: -- [~surendrasingh] I will add patch for branch3.1 and 3.2 soon

[jira] [Comment Edited] (HDFS-14768) EC : Busy DN replica should be consider in live replica check.

2019-11-03 Thread guojh (Jira)
[ https://issues.apache.org/jira/browse/HDFS-14768?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16966312#comment-16966312 ] guojh edited comment on HDFS-14768 at 11/3/19 11:53 PM: [~surendr

[jira] [Commented] (HDFS-14768) EC : Busy DN replica should be consider in live replica check.

2019-11-01 Thread guojh (Jira)
[ https://issues.apache.org/jira/browse/HDFS-14768?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16964767#comment-16964767 ] guojh commented on HDFS-14768: -- [~surendrasingh]  [~ferhui]  I'm sorry for I lost my patienc

[jira] [Updated] (HDFS-14768) EC : Busy DN replica should be consider in live replica check.

2019-11-01 Thread guojh (Jira)
[ https://issues.apache.org/jira/browse/HDFS-14768?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] guojh updated HDFS-14768: - Attachment: HDFS-14768.011.patch > EC : Busy DN replica should be consider in live replica check. > -

[jira] [Resolved] (HDFS-14768) EC : Busy DN replica should be consider in live replica check.

2019-10-31 Thread guojh (Jira)
[ https://issues.apache.org/jira/browse/HDFS-14768?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] guojh resolved HDFS-14768. -- Resolution: Not A Problem > EC : Busy DN replica should be consider in live replica check. > --

[jira] [Reopened] (HDFS-14768) EC : Busy DN replica should be consider in live replica check.

2019-10-31 Thread guojh (Jira)
[ https://issues.apache.org/jira/browse/HDFS-14768?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] guojh reopened HDFS-14768: -- > EC : Busy DN replica should be consider in live replica check. >

[jira] [Resolved] (HDFS-14768) EC : Busy DN replica should be consider in live replica check.

2019-10-31 Thread guojh (Jira)
[ https://issues.apache.org/jira/browse/HDFS-14768?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] guojh resolved HDFS-14768. -- Resolution: Abandoned > EC : Busy DN replica should be consider in live replica check. > --

[jira] [Updated] (HDFS-14768) EC : Busy DN replica should be consider in live replica check.

2019-10-31 Thread guojh (Jira)
[ https://issues.apache.org/jira/browse/HDFS-14768?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] guojh updated HDFS-14768: - Status: Open (was: Patch Available) > EC : Busy DN replica should be consider in live replica check. > -

[jira] [Commented] (HDFS-14768) EC : Busy DN replica should be consider in live replica check.

2019-10-28 Thread guojh (Jira)
[ https://issues.apache.org/jira/browse/HDFS-14768?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16960850#comment-16960850 ] guojh commented on HDFS-14768: -- [~surendrasingh] Thanks for you UT, I update the code and fi

[jira] [Commented] (HDFS-14920) Erasure Coding: Decommission may hang If one or more datanodes are out of service during decommission

2019-10-28 Thread guojh (Jira)
[ https://issues.apache.org/jira/browse/HDFS-14920?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16960817#comment-16960817 ] guojh commented on HDFS-14920: -- Thanks  [~ferhui] I am clear now. You just want to correct t

[jira] [Updated] (HDFS-14768) EC : Busy DN replica should be consider in live replica check.

2019-10-27 Thread guojh (Jira)
[ https://issues.apache.org/jira/browse/HDFS-14768?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] guojh updated HDFS-14768: - Attachment: HDFS-14768.010.patch > EC : Busy DN replica should be consider in live replica check. > -

[jira] [Commented] (HDFS-14920) Erasure Coding: Decommission may hang If one or more datanodes are out of service during decommission

2019-10-27 Thread guojh (Jira)
[ https://issues.apache.org/jira/browse/HDFS-14920?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16960772#comment-16960772 ] guojh commented on HDFS-14920: -- [~ferhui] If the block group not has all internal blocks,  t

[jira] [Commented] (HDFS-14920) Erasure Coding: Decommission may hang If one or more datanodes are out of service during decommission

2019-10-27 Thread guojh (Jira)
[ https://issues.apache.org/jira/browse/HDFS-14920?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16960745#comment-16960745 ] guojh commented on HDFS-14920: -- [~ferhui] As you description,this is a bug to compute the a

[jira] [Commented] (HDFS-14768) EC : Busy DN replica should be consider in live replica check.

2019-10-27 Thread guojh (Jira)
[ https://issues.apache.org/jira/browse/HDFS-14768?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16960728#comment-16960728 ] guojh commented on HDFS-14768: -- [~ferhui] If datanode is out of service, the block in it sho

[jira] [Commented] (HDFS-14768) EC : Busy DN replica should be consider in live replica check.

2019-10-22 Thread guojh (Jira)
[ https://issues.apache.org/jira/browse/HDFS-14768?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16956793#comment-16956793 ] guojh commented on HDFS-14768: -- [~surendrasingh] Ok > EC : Busy DN replica should be consid

[jira] [Comment Edited] (HDFS-14768) EC : Busy DN replica should be consider in live replica check.

2019-10-22 Thread guojh (Jira)
[ https://issues.apache.org/jira/browse/HDFS-14768?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16956746#comment-16956746 ] guojh edited comment on HDFS-14768 at 10/22/19 8:01 AM: [~surendr

[jira] [Updated] (HDFS-14768) EC : Busy DN replica should be consider in live replica check.

2019-10-22 Thread guojh (Jira)
[ https://issues.apache.org/jira/browse/HDFS-14768?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] guojh updated HDFS-14768: - Attachment: HDFS-14768.008.patch > EC : Busy DN replica should be consider in live replica check. > -

[jira] [Commented] (HDFS-14768) EC : Busy DN replica should be consider in live replica check.

2019-10-22 Thread guojh (Jira)
[ https://issues.apache.org/jira/browse/HDFS-14768?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16956746#comment-16956746 ] guojh commented on HDFS-14768: -- [~surendrasingh] Do you have any ideas about the UT?    >

[jira] [Commented] (HDFS-14768) EC : Busy DN replica should be consider in live replica check.

2019-10-21 Thread guojh (Jira)
[ https://issues.apache.org/jira/browse/HDFS-14768?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16956578#comment-16956578 ] guojh commented on HDFS-14768: -- [~surendrasingh] Thanks. > EC : Busy DN replica should be c

[jira] [Comment Edited] (HDFS-14768) EC : Busy DN replica should be consider in live replica check.

2019-10-21 Thread guojh (Jira)
[ https://issues.apache.org/jira/browse/HDFS-14768?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16956094#comment-16956094 ] guojh edited comment on HDFS-14768 at 10/21/19 1:44 PM: [~surendr

[jira] [Commented] (HDFS-14768) EC : Busy DN replica should be consider in live replica check.

2019-10-21 Thread guojh (Jira)
[ https://issues.apache.org/jira/browse/HDFS-14768?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16956094#comment-16956094 ] guojh commented on HDFS-14768: -- [~surendrasingh] You can run the UT more times, the UT passe

[jira] [Commented] (HDFS-14768) EC : Busy DN replica should be consider in live replica check.

2019-10-20 Thread guojh (Jira)
[ https://issues.apache.org/jira/browse/HDFS-14768?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16955660#comment-16955660 ] guojh commented on HDFS-14768: -- [~surendrasingh] Please review it. > EC : Busy DN replica s

[jira] [Commented] (HDFS-14768) EC : Busy DN replica should be consider in live replica check.

2019-10-20 Thread guojh (Jira)
[ https://issues.apache.org/jira/browse/HDFS-14768?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16955446#comment-16955446 ] guojh commented on HDFS-14768: -- rebase > EC : Busy DN replica should be consider in live re

[jira] [Updated] (HDFS-14768) EC : Busy DN replica should be consider in live replica check.

2019-10-20 Thread guojh (Jira)
[ https://issues.apache.org/jira/browse/HDFS-14768?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] guojh updated HDFS-14768: - Attachment: HDFS-14768.007.patch > EC : Busy DN replica should be consider in live replica check. > -

[jira] [Commented] (HDFS-14768) In some cases, erasure blocks are corruption when they are reconstruct.

2019-10-17 Thread guojh (Jira)
[ https://issues.apache.org/jira/browse/HDFS-14768?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16954296#comment-16954296 ] guojh commented on HDFS-14768: -- rebase trunk > In some cases, erasure blocks are corruption

[jira] [Updated] (HDFS-14768) In some cases, erasure blocks are corruption when they are reconstruct.

2019-10-17 Thread guojh (Jira)
[ https://issues.apache.org/jira/browse/HDFS-14768?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] guojh updated HDFS-14768: - Attachment: HDFS-14768.006.patch > In some cases, erasure blocks are corruption when they are reconstruct. > ---

[jira] [Commented] (HDFS-14768) In some cases, erasure blocks are corruption when they are reconstruct.

2019-10-17 Thread guojh (Jira)
[ https://issues.apache.org/jira/browse/HDFS-14768?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16954292#comment-16954292 ] guojh commented on HDFS-14768: -- [~hadoopqa] test > In some cases, erasure blocks are corrup

[jira] [Commented] (HDFS-14768) In some cases, erasure blocks are corruption when they are reconstruct.

2019-10-17 Thread guojh (Jira)
[ https://issues.apache.org/jira/browse/HDFS-14768?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16954257#comment-16954257 ] guojh commented on HDFS-14768: -- [~surendrasingh] Sorry about my thoughtless and thanks for y

[jira] [Updated] (HDFS-14768) In some cases, erasure blocks are corruption when they are reconstruct.

2019-10-17 Thread guojh (Jira)
[ https://issues.apache.org/jira/browse/HDFS-14768?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] guojh updated HDFS-14768: - Attachment: HDFS-14768.005.patch > In some cases, erasure blocks are corruption when they are reconstruct. > ---

[jira] [Updated] (HDFS-14768) In some cases, erasure blocks are corruption when they are reconstruct.

2019-10-17 Thread guojh (Jira)
[ https://issues.apache.org/jira/browse/HDFS-14768?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] guojh updated HDFS-14768: - Attachment: (was: HDFS-14768.006.patch) > In some cases, erasure blocks are corruption when they are reconst

[jira] [Updated] (HDFS-14768) In some cases, erasure blocks are corruption when they are reconstruct.

2019-10-17 Thread guojh (Jira)
[ https://issues.apache.org/jira/browse/HDFS-14768?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] guojh updated HDFS-14768: - Attachment: (was: HDFS-14768.005.patch) > In some cases, erasure blocks are corruption when they are reconst

[jira] [Updated] (HDFS-14768) In some cases, erasure blocks are corruption when they are reconstruct.

2019-10-17 Thread guojh (Jira)
[ https://issues.apache.org/jira/browse/HDFS-14768?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] guojh updated HDFS-14768: - Attachment: HDFS-14768.006.patch > In some cases, erasure blocks are corruption when they are reconstruct. > ---

[jira] [Updated] (HDFS-14768) In some cases, erasure blocks are corruption when they are reconstruct.

2019-10-17 Thread guojh (Jira)
[ https://issues.apache.org/jira/browse/HDFS-14768?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] guojh updated HDFS-14768: - Attachment: (was: HDFS-14768.006.patch) > In some cases, erasure blocks are corruption when they are reconst

[jira] [Commented] (HDFS-14699) Erasure Coding: Storage not considered in live replica when replication streams hard limit reached to threshold

2019-10-17 Thread guojh (Jira)
[ https://issues.apache.org/jira/browse/HDFS-14699?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16954245#comment-16954245 ] guojh commented on HDFS-14699: -- [~surendrasingh] The UT testChooseSrcDatanodesWithDupEC in c

[jira] [Updated] (HDFS-14768) In some cases, erasure blocks are corruption when they are reconstruct.

2019-10-17 Thread guojh (Jira)
[ https://issues.apache.org/jira/browse/HDFS-14768?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] guojh updated HDFS-14768: - Attachment: HDFS-14768.006.patch > In some cases, erasure blocks are corruption when they are reconstruct. > ---

[jira] [Updated] (HDFS-14768) In some cases, erasure blocks are corruption when they are reconstruct.

2019-10-17 Thread guojh (Jira)
[ https://issues.apache.org/jira/browse/HDFS-14768?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] guojh updated HDFS-14768: - Attachment: (was: HDFS-14768.006.patch) > In some cases, erasure blocks are corruption when they are reconst

[jira] [Commented] (HDFS-14768) In some cases, erasure blocks are corruption when they are reconstruct.

2019-10-17 Thread guojh (Jira)
[ https://issues.apache.org/jira/browse/HDFS-14768?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16954202#comment-16954202 ] guojh commented on HDFS-14768: -- [~surendrasingh] I put a new path06, please review it. > In

[jira] [Updated] (HDFS-14768) In some cases, erasure blocks are corruption when they are reconstruct.

2019-10-17 Thread guojh (Jira)
[ https://issues.apache.org/jira/browse/HDFS-14768?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] guojh updated HDFS-14768: - Attachment: HDFS-14768.006.patch > In some cases, erasure blocks are corruption when they are reconstruct. > ---

[jira] [Commented] (HDFS-14699) Erasure Coding: Storage not considered in live replica when replication streams hard limit reached to threshold

2019-10-17 Thread guojh (Jira)
[ https://issues.apache.org/jira/browse/HDFS-14699?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16954193#comment-16954193 ] guojh commented on HDFS-14699: -- [~zhaoyim] [~surendrasingh] In you test case, why you set th

[jira] [Updated] (HDFS-14768) In some cases, erasure blocks are corruption when they are reconstruct.

2019-10-17 Thread guojh (Jira)
[ https://issues.apache.org/jira/browse/HDFS-14768?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] guojh updated HDFS-14768: - Attachment: HDFS-14768.005.patch > In some cases, erasure blocks are corruption when they are reconstruct. > ---

[jira] [Updated] (HDFS-14768) In some cases, erasure blocks are corruption when they are reconstruct.

2019-10-17 Thread guojh (Jira)
[ https://issues.apache.org/jira/browse/HDFS-14768?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] guojh updated HDFS-14768: - Attachment: (was: HDFS-14768.004.patch) > In some cases, erasure blocks are corruption when they are reconst

[jira] [Updated] (HDFS-14768) In some cases, erasure blocks are corruption when they are reconstruct.

2019-10-17 Thread guojh (Jira)
[ https://issues.apache.org/jira/browse/HDFS-14768?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] guojh updated HDFS-14768: - Attachment: HDFS-14768.004.patch > In some cases, erasure blocks are corruption when they are reconstruct. > ---

[jira] [Commented] (HDFS-14768) In some cases, erasure blocks are corruption when they are reconstruct.

2019-10-17 Thread guojh (Jira)
[ https://issues.apache.org/jira/browse/HDFS-14768?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16954183#comment-16954183 ] guojh commented on HDFS-14768: -- [~surendrasingh] Thanks for your review. I don't understand

[jira] [Commented] (HDFS-14768) In some cases, erasure blocks are corruption when they are reconstruct.

2019-10-17 Thread guojh (Jira)
[ https://issues.apache.org/jira/browse/HDFS-14768?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16953456#comment-16953456 ] guojh commented on HDFS-14768: -- [~surendrasingh] I put a new patch. Please review it. > In

[jira] [Updated] (HDFS-14768) In some cases, erasure blocks are corruption when they are reconstruct.

2019-10-17 Thread guojh (Jira)
[ https://issues.apache.org/jira/browse/HDFS-14768?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] guojh updated HDFS-14768: - Attachment: HDFS-14768.004.patch > In some cases, erasure blocks are corruption when they are reconstruct. > ---

[jira] [Commented] (HDFS-14768) In some cases, erasure blocks are corruption when they are reconstruct.

2019-10-16 Thread guojh (Jira)
[ https://issues.apache.org/jira/browse/HDFS-14768?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16953447#comment-16953447 ] guojh commented on HDFS-14768: -- [~ayushtkn] [~surendrasingh] Could you help review the code?

[jira] [Commented] (HDFS-14768) In some cases, erasure blocks are corruption when they are reconstruct.

2019-10-16 Thread guojh (Jira)
[ https://issues.apache.org/jira/browse/HDFS-14768?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16953443#comment-16953443 ] guojh commented on HDFS-14768: -- [~ayushtkn] Thanks for you reply, This patch try to fix two

[jira] [Commented] (HDFS-14768) In some cases, erasure blocks are corruption when they are reconstruct.

2019-10-16 Thread guojh (Jira)
[ https://issues.apache.org/jira/browse/HDFS-14768?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16953287#comment-16953287 ] guojh commented on HDFS-14768: -- [~surendrasingh] If I decommission two datanode that have th

[jira] [Updated] (HDFS-14768) In some cases, erasure blocks are corruption when they are reconstruct.

2019-10-09 Thread guojh (Jira)
[ https://issues.apache.org/jira/browse/HDFS-14768?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] guojh updated HDFS-14768: - Attachment: HDFS-14768.003.patch > In some cases, erasure blocks are corruption when they are reconstruct. > ---

[jira] [Updated] (HDFS-14768) In some cases, erasure blocks are corruption when they are reconstruct.

2019-10-09 Thread guojh (Jira)
[ https://issues.apache.org/jira/browse/HDFS-14768?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] guojh updated HDFS-14768: - Attachment: (was: HDFS-14768.003.patch) > In some cases, erasure blocks are corruption when they are reconst

[jira] [Updated] (HDFS-14768) In some cases, erasure blocks are corruption when they are reconstruct.

2019-10-09 Thread guojh (Jira)
[ https://issues.apache.org/jira/browse/HDFS-14768?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] guojh updated HDFS-14768: - Attachment: HDFS-14768.003.patch > In some cases, erasure blocks are corruption when they are reconstruct. > ---

[jira] [Commented] (HDFS-14768) In some cases, erasure blocks are corruption when they are reconstruct.

2019-10-09 Thread guojh (Jira)
[ https://issues.apache.org/jira/browse/HDFS-14768?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16947422#comment-16947422 ] guojh commented on HDFS-14768: -- [~surendrasingh]  Thanks for you reply, the incrementPendin

[jira] [Updated] (HDFS-14768) In some cases, erasure blocks are corruption when they are reconstruct.

2019-10-09 Thread guojh (Jira)
[ https://issues.apache.org/jira/browse/HDFS-14768?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] guojh updated HDFS-14768: - Attachment: (was: HDFS-14768.003.patch) > In some cases, erasure blocks are corruption when they are reconst

[jira] [Updated] (HDFS-14768) In some cases, erasure blocks are corruption when they are reconstruct.

2019-10-09 Thread guojh (Jira)
[ https://issues.apache.org/jira/browse/HDFS-14768?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] guojh updated HDFS-14768: - Attachment: HDFS-14768.003.patch > In some cases, erasure blocks are corruption when they are reconstruct. > ---

[jira] [Commented] (HDFS-14768) In some cases, erasure blocks are corruption when they are reconstruct.

2019-09-18 Thread guojh (Jira)
[ https://issues.apache.org/jira/browse/HDFS-14768?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16932196#comment-16932196 ] guojh commented on HDFS-14768: -- [~zhaoyim] Yes, I encounter the same problem. > In some cas

[jira] [Commented] (HDFS-14768) In some cases, erasure blocks are corruption when they are reconstruct.

2019-09-17 Thread guojh (Jira)
[ https://issues.apache.org/jira/browse/HDFS-14768?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16931985#comment-16931985 ] guojh commented on HDFS-14768: -- [~zhaoyim] I am confused too. [~surendrasingh] Could you hel

[jira] [Issue Comment Deleted] (HDFS-14768) In some cases, erasure blocks are corruption when they are reconstruct.

2019-09-17 Thread guojh (Jira)
[ https://issues.apache.org/jira/browse/HDFS-14768?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] guojh updated HDFS-14768: - Comment: was deleted (was: [~zhaoyim] I am confused too. If you add break point in the check parity like the fol

[jira] [Commented] (HDFS-14768) In some cases, erasure blocks are corruption when they are reconstruct.

2019-09-17 Thread guojh (Jira)
[ https://issues.apache.org/jira/browse/HDFS-14768?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16931965#comment-16931965 ] guojh commented on HDFS-14768: -- [~zhaoyim] I am confused too. If you add break point in the

[jira] [Updated] (HDFS-14768) In some cases, erasure blocks are corruption when they are reconstruct.

2019-09-17 Thread guojh (Jira)
[ https://issues.apache.org/jira/browse/HDFS-14768?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] guojh updated HDFS-14768: - Attachment: 1568771471942.jpg > In some cases, erasure blocks are corruption when they are reconstruct. > --

[jira] [Commented] (HDFS-14768) In some cases, erasure blocks are corruption when they are reconstruct.

2019-09-17 Thread guojh (Jira)
[ https://issues.apache.org/jira/browse/HDFS-14768?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16931153#comment-16931153 ] guojh commented on HDFS-14768: -- [~zhaoyim] Try the new patch. > In some cases, erasure bloc

[jira] [Updated] (HDFS-14768) In some cases, erasure blocks are corruption when they are reconstruct.

2019-09-17 Thread guojh (Jira)
[ https://issues.apache.org/jira/browse/HDFS-14768?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] guojh updated HDFS-14768: - Attachment: HDFS-14768.002.patch > In some cases, erasure blocks are corruption when they are reconstruct. > ---

[jira] [Updated] (HDFS-14768) In some cases, erasure blocks are corruption when they are reconstruct.

2019-09-16 Thread guojh (Jira)
[ https://issues.apache.org/jira/browse/HDFS-14768?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] guojh updated HDFS-14768: - Attachment: (was: HDFS-14768.0001.patch) > In some cases, erasure blocks are corruption when they are recons

[jira] [Updated] (HDFS-14768) In some cases, erasure blocks are corruption when they are reconstruct.

2019-09-16 Thread guojh (Jira)
[ https://issues.apache.org/jira/browse/HDFS-14768?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] guojh updated HDFS-14768: - Attachment: HDFS-14768.001.patch > In some cases, erasure blocks are corruption when they are reconstruct. > ---

[jira] [Comment Edited] (HDFS-14768) In some cases, erasure blocks are corruption when they are reconstruct.

2019-09-16 Thread guojh (Jira)
[ https://issues.apache.org/jira/browse/HDFS-14768?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16931021#comment-16931021 ] guojh edited comment on HDFS-14768 at 9/17/19 2:34 AM: --- [~surendras

[jira] [Commented] (HDFS-14768) In some cases, erasure blocks are corruption when they are reconstruct.

2019-09-16 Thread guojh (Jira)
[ https://issues.apache.org/jira/browse/HDFS-14768?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16931021#comment-16931021 ] guojh commented on HDFS-14768: -- [~surendrasingh] I put a new patch based on the latest code.

[jira] [Updated] (HDFS-14768) In some cases, erasure blocks are corruption when they are reconstruct.

2019-09-16 Thread guojh (Jira)
[ https://issues.apache.org/jira/browse/HDFS-14768?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] guojh updated HDFS-14768: - Attachment: (was: HDFS-14768.0001.patch) > In some cases, erasure blocks are corruption when they are recons

[jira] [Updated] (HDFS-14768) In some cases, erasure blocks are corruption when they are reconstruct.

2019-09-16 Thread guojh (Jira)
[ https://issues.apache.org/jira/browse/HDFS-14768?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] guojh updated HDFS-14768: - Attachment: HDFS-14768.0001.patch > In some cases, erasure blocks are corruption when they are reconstruct. > --

[jira] [Updated] (HDFS-14768) In some cases, erasure blocks are corruption when they are reconstruct.

2019-09-16 Thread guojh (Jira)
[ https://issues.apache.org/jira/browse/HDFS-14768?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] guojh updated HDFS-14768: - Attachment: HDFS-14768.0001.patch > In some cases, erasure blocks are corruption when they are reconstruct. > --

[jira] [Commented] (HDFS-14768) In some cases, erasure blocks are corruption when they are reconstruct.

2019-09-16 Thread guojh (Jira)
[ https://issues.apache.org/jira/browse/HDFS-14768?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16931008#comment-16931008 ] guojh commented on HDFS-14768: -- [~surendrasingh] [~zhaoyim] Used the latest code in branch t

[jira] [Commented] (HDFS-14768) In some cases, erasure blocks are corruption when they are reconstruct.

2019-09-16 Thread guojh (Jira)
[ https://issues.apache.org/jira/browse/HDFS-14768?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16930340#comment-16930340 ] guojh commented on HDFS-14768: -- [~zhaoyim] Before checkData I sleep 3min still have the chec

[jira] [Commented] (HDFS-14768) In some cases, erasure blocks are corruption when they are reconstruct.

2019-09-15 Thread guojh (Jira)
[ https://issues.apache.org/jira/browse/HDFS-14768?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16930259#comment-16930259 ] guojh commented on HDFS-14768: -- [~zhaoyim] The index 6 in the busy DN is correct, another in

[jira] [Commented] (HDFS-14768) In some cases, erasure blocks are corruption when they are reconstruct.

2019-09-12 Thread guojh (Jira)
[ https://issues.apache.org/jira/browse/HDFS-14768?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16928962#comment-16928962 ] guojh commented on HDFS-14768: -- [~zhaoyim] Yes, after you sleep 3 min, NN will reconstruct a

[jira] [Commented] (HDFS-14768) In some cases, erasure blocks are corruption when they are reconstruct.

2019-09-12 Thread guojh (Jira)
[ https://issues.apache.org/jira/browse/HDFS-14768?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16928424#comment-16928424 ] guojh commented on HDFS-14768: -- [~janick] set more numDNS, for example: numDNs = dataBlocks

[jira] [Comment Edited] (HDFS-14768) In some cases, erasure blocks are corruption when they are reconstruct.

2019-09-12 Thread guojh (Jira)
[ https://issues.apache.org/jira/browse/HDFS-14768?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16928348#comment-16928348 ] guojh edited comment on HDFS-14768 at 9/12/19 8:19 AM: --- @[~surendra

[jira] [Updated] (HDFS-14768) In some cases, erasure blocks are corruption when they are reconstruct.

2019-09-12 Thread guojh (Jira)
[ https://issues.apache.org/jira/browse/HDFS-14768?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] guojh updated HDFS-14768: - Attachment: 1568275810244.jpg > In some cases, erasure blocks are corruption when they are reconstruct. > --

[jira] [Commented] (HDFS-14768) In some cases, erasure blocks are corruption when they are reconstruct.

2019-09-12 Thread guojh (Jira)
[ https://issues.apache.org/jira/browse/HDFS-14768?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16928348#comment-16928348 ] guojh commented on HDFS-14768: -- @[~surendrasingh] I update the UT in Description to reproduc

[jira] [Commented] (HDFS-14768) In some cases, erasure blocks are corruption when they are reconstruct.

2019-09-12 Thread guojh (Jira)
[ https://issues.apache.org/jira/browse/HDFS-14768?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16928344#comment-16928344 ] guojh commented on HDFS-14768: -- [~zhaoyim] The key point about this issue is make one datano

[jira] [Updated] (HDFS-14768) In some cases, erasure blocks are corruption when they are reconstruct.

2019-09-12 Thread guojh (Jira)
[ https://issues.apache.org/jira/browse/HDFS-14768?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] guojh updated HDFS-14768: - Description: Policy is RS-6-3-1024K, version is hadoop 3.0.2; We suppose a file's block Index is [0,1,2,3,4,5,6,

[jira] [Commented] (HDFS-14768) In some cases, erasure blocks are corruption when they are reconstruct.

2019-09-11 Thread guojh (Jira)
[ https://issues.apache.org/jira/browse/HDFS-14768?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16928146#comment-16928146 ] guojh commented on HDFS-14768: -- [~zhaoyim] My UT is not good, you need to check the block ma

[jira] [Commented] (HDFS-14768) In some cases, erasure blocks are corruption when they are reconstruct.

2019-09-11 Thread guojh (Jira)
[ https://issues.apache.org/jira/browse/HDFS-14768?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16928145#comment-16928145 ] guojh commented on HDFS-14768: -- [~surendrasingh] Sorry about my UT. I will update a new UT t

[jira] [Comment Edited] (HDFS-14768) In some cases, erasure blocks are corruption when they are reconstruct.

2019-09-08 Thread guojh (Jira)
[ https://issues.apache.org/jira/browse/HDFS-14768?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16925298#comment-16925298 ] guojh edited comment on HDFS-14768 at 9/9/19 2:16 AM: -- [~surendrasin

[jira] [Comment Edited] (HDFS-14768) In some cases, erasure blocks are corruption when they are reconstruct.

2019-09-08 Thread guojh (Jira)
[ https://issues.apache.org/jira/browse/HDFS-14768?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16925298#comment-16925298 ] guojh edited comment on HDFS-14768 at 9/9/19 2:14 AM: -- [~surendrasin

[jira] [Comment Edited] (HDFS-14768) In some cases, erasure blocks are corruption when they are reconstruct.

2019-09-08 Thread guojh (Jira)
[ https://issues.apache.org/jira/browse/HDFS-14768?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16925298#comment-16925298 ] guojh edited comment on HDFS-14768 at 9/9/19 2:12 AM: -- [~surendrasin

[jira] [Comment Edited] (HDFS-14768) In some cases, erasure blocks are corruption when they are reconstruct.

2019-09-08 Thread guojh (Jira)
[ https://issues.apache.org/jira/browse/HDFS-14768?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16925298#comment-16925298 ] guojh edited comment on HDFS-14768 at 9/9/19 2:12 AM: -- [~surendrasin

[jira] [Commented] (HDFS-14768) In some cases, erasure blocks are corruption when they are reconstruct.

2019-09-08 Thread guojh (Jira)
[ https://issues.apache.org/jira/browse/HDFS-14768?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16925298#comment-16925298 ] guojh commented on HDFS-14768: -- [~surendrasingh] Thanks for you replay. you need run the cod

[jira] [Updated] (HDFS-14768) In some cases, erasure blocks are corruption when they are reconstruct.

2019-08-27 Thread guojh (Jira)
[ https://issues.apache.org/jira/browse/HDFS-14768?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] guojh updated HDFS-14768: - Attachment: HDFS-14768.000.patch > In some cases, erasure blocks are corruption when they are reconstruct. > ---

[jira] [Updated] (HDFS-14768) In some cases, erasure blocks are corruption when they are reconstruct.

2019-08-26 Thread guojh (Jira)
[ https://issues.apache.org/jira/browse/HDFS-14768?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] guojh updated HDFS-14768: - Attachment: (was: HDFS-14768.001.patch) > In some cases, erasure blocks are corruption when they are reconst

[jira] [Updated] (HDFS-14768) In some cases, erasure blocks are corruption when they are reconstruct.

2019-08-26 Thread guojh (Jira)
[ https://issues.apache.org/jira/browse/HDFS-14768?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] guojh updated HDFS-14768: - Attachment: (was: HDFS-14768.000.patch) > In some cases, erasure blocks are corruption when they are reconst

[jira] [Updated] (HDFS-14768) In some cases, erasure blocks are corruption when they are reconstruct.

2019-08-26 Thread guojh (Jira)
[ https://issues.apache.org/jira/browse/HDFS-14768?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] guojh updated HDFS-14768: - Attachment: HDFS-14768.001.patch > In some cases, erasure blocks are corruption when they are reconstruct. > ---

[jira] [Updated] (HDFS-14768) In some cases, erasure blocks are corruption when they are reconstruct.

2019-08-25 Thread guojh (Jira)
[ https://issues.apache.org/jira/browse/HDFS-14768?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] guojh updated HDFS-14768: - Summary: In some cases, erasure blocks are corruption when they are reconstruct. (was: In some cases, erasure b

[jira] [Commented] (HDFS-14768) In some cases, erasure blocks are corruption when they are rebuilt.

2019-08-25 Thread guojh (Jira)
[ https://issues.apache.org/jira/browse/HDFS-14768?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16915395#comment-16915395 ] guojh commented on HDFS-14768: -- [~drankye] Can we talk about this issue? > In some cases, e

[jira] [Updated] (HDFS-14768) In some cases, erasure blocks are corruption when they are rebuilt.

2019-08-25 Thread guojh (Jira)
[ https://issues.apache.org/jira/browse/HDFS-14768?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] guojh updated HDFS-14768: - Attachment: (was: HDFS-14768.000.patch) > In some cases, erasure blocks are corruption when they are rebuilt

[jira] [Updated] (HDFS-14768) In some cases, erasure blocks are corruption when they are rebuilt.

2019-08-25 Thread guojh (Jira)
[ https://issues.apache.org/jira/browse/HDFS-14768?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] guojh updated HDFS-14768: - Attachment: HDFS-14768.000.patch Fix Version/s: 3.3.0 Target Version/s: 3.3.0 Label

[jira] [Updated] (HDFS-14768) In some cases, erasure blocks are corruption when they are rebuilt.

2019-08-23 Thread guojh (Jira)
[ https://issues.apache.org/jira/browse/HDFS-14768?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] guojh updated HDFS-14768: - Attachment: HDFS-14768.000.patch > In some cases, erasure blocks are corruption when they are rebuilt. > ---

[jira] [Commented] (HDFS-14768) In some cases, erasure blocks are corruption when they are rebuilt.

2019-08-22 Thread guojh (Jira)
[ https://issues.apache.org/jira/browse/HDFS-14768?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16913961#comment-16913961 ] guojh commented on HDFS-14768: -- [~ayushtkn] yes, we have solved the bug. But I still need yo

[jira] [Commented] (HDFS-14768) In some cases, erasure blocks are corruption when they are rebuilt.

2019-08-22 Thread guojh (Jira)
[ https://issues.apache.org/jira/browse/HDFS-14768?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16913954#comment-16913954 ] guojh commented on HDFS-14768: -- [~hemanthboyina] Thanks for you attention. I post the detail

  1   2   >