[jira] [Commented] (HDFS-14247) Repeat adding node description into network topology

2019-02-26 Thread HuangTao (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-14247?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=1697#comment-1697 ] HuangTao commented on HDFS-14247: - I tried `mvn -Dtest="hadoop.hdfs.server.datanode.TestBPOfferService"

[jira] [Comment Edited] (HDFS-14247) Repeat adding node description into network topology

2019-02-26 Thread HuangTao (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-14247?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16778789#comment-16778789 ] HuangTao edited comment on HDFS-14247 at 2/27/19 2:18 AM: -- HDFS-14245 failed on 

[jira] [Commented] (HDFS-14247) Repeat adding node description into network topology

2019-02-26 Thread HuangTao (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-14247?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16778789#comment-16778789 ] HuangTao commented on HDFS-14247: - #HDFS-14245 failed on TestBPOfferService too. > Repeat adding node

[jira] [Commented] (HDFS-14247) Repeat adding node description into network topology

2019-03-01 Thread HuangTao (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-14247?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16781636#comment-16781636 ] HuangTao commented on HDFS-14247: - Can this patch be merged? [~elgoiri] > Repeat adding node description

[jira] [Updated] (HDFS-14247) Repeat adding node description into network topology

2019-01-30 Thread HuangTao (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-14247?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] HuangTao updated HDFS-14247: Attachment: HDFS-14247.001.patch > Repeat adding node description into network topology >

[jira] [Created] (HDFS-14247) Repeat adding node description into network topology

2019-01-30 Thread HuangTao (JIRA)
HuangTao created HDFS-14247: --- Summary: Repeat adding node description into network topology Key: HDFS-14247 URL: https://issues.apache.org/jira/browse/HDFS-14247 Project: Hadoop HDFS Issue Type:

[jira] [Updated] (HDFS-14247) Repeat adding node description into network topology

2019-01-30 Thread HuangTao (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-14247?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] HuangTao updated HDFS-14247: Description: I find there is a duplicate code to add nodeDescr to networktopology in the 

[jira] [Updated] (HDFS-14247) Repeat adding node description into network topology

2019-01-30 Thread HuangTao (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-14247?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] HuangTao updated HDFS-14247: Description: I find there is a duplicate code to add nodeDescr to networktopology in the 

[jira] [Commented] (HDFS-14247) Repeat adding node description into network topology

2019-01-30 Thread HuangTao (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-14247?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16756851#comment-16756851 ] HuangTao commented on HDFS-14247: - Added the initial patch. Please check it once. > Repeat adding node

[jira] [Commented] (HDFS-14419) Avoid repeated calls to the listOpenFiles function

2019-04-09 Thread HuangTao (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-14419?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16813995#comment-16813995 ] HuangTao commented on HDFS-14419: - Thanks [~daryn] for quick response. I have submitted another patch. >

[jira] [Updated] (HDFS-14419) Avoid repeated calls to the listOpenFiles function

2019-04-09 Thread HuangTao (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-14419?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] HuangTao updated HDFS-14419: Status: Patch Available (was: Open) > Avoid repeated calls to the listOpenFiles function >

[jira] [Updated] (HDFS-14419) Avoid repeated calls to the listOpenFiles function

2019-04-09 Thread HuangTao (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-14419?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] HuangTao updated HDFS-14419: Attachment: HDFS-14419.002.patch > Avoid repeated calls to the listOpenFiles function >

[jira] [Updated] (HDFS-14446) No enough internal blocks causes NN infinitely loop on releasing lease

2019-04-20 Thread HuangTao (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-14446?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] HuangTao updated HDFS-14446: Description: {code:java} [2019-04-20T18:59:16.564+08:00] [INFO]

[jira] [Created] (HDFS-14446) No enough internal blocks causes NN infinitely loop on releasing lease

2019-04-20 Thread HuangTao (JIRA)
HuangTao created HDFS-14446: --- Summary: No enough internal blocks causes NN infinitely loop on releasing lease Key: HDFS-14446 URL: https://issues.apache.org/jira/browse/HDFS-14446 Project: Hadoop HDFS

[jira] [Updated] (HDFS-14419) Avoid repeated calls to the listOpenFiles function

2019-04-09 Thread HuangTao (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-14419?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] HuangTao updated HDFS-14419: Description: `hdfs dfsadmin -listOpenFiles -path /any/path` will request all opened files. In the 

[jira] [Updated] (HDFS-14419) Avoid repeated calls to the listOpenFiles function

2019-04-09 Thread HuangTao (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-14419?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] HuangTao updated HDFS-14419: Attachment: HDFS-14419.001.patch > Avoid repeated calls to the listOpenFiles function >

[jira] [Created] (HDFS-14419) Avoid repeated calls to the listOpenFiles function

2019-04-09 Thread HuangTao (JIRA)
HuangTao created HDFS-14419: --- Summary: Avoid repeated calls to the listOpenFiles function Key: HDFS-14419 URL: https://issues.apache.org/jira/browse/HDFS-14419 Project: Hadoop HDFS Issue Type:

[jira] [Commented] (HDFS-14419) Avoid repeated calls to the listOpenFiles function

2019-04-09 Thread HuangTao (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-14419?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16813238#comment-16813238 ] HuangTao commented on HDFS-14419: - Add patch HDFS-14419.001.patch to resolve this issue. > Avoid

[jira] [Commented] (HDFS-14770) Erasure Coding: reconstructed internal block is error

2019-08-27 Thread HuangTao (Jira)
[ https://issues.apache.org/jira/browse/HDFS-14770?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16916552#comment-16916552 ] HuangTao commented on HDFS-14770: - {code:java} [2019-08-13T18:51:09.587+08:00] [INFO] [stripedRead-2122]

[jira] [Comment Edited] (HDFS-14770) Erasure Coding: reconstructed internal block is error

2019-08-27 Thread HuangTao (Jira)
[ https://issues.apache.org/jira/browse/HDFS-14770?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16916552#comment-16916552 ] HuangTao edited comment on HDFS-14770 at 8/27/19 9:46 AM: -- {code:java}

[jira] [Assigned] (HDFS-14770) Erasure Coding: reconstructed internal block is error

2019-08-27 Thread HuangTao (Jira)
[ https://issues.apache.org/jira/browse/HDFS-14770?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] HuangTao reassigned HDFS-14770: --- Assignee: HuangTao > Erasure Coding: reconstructed internal block is error >

[jira] [Commented] (HDFS-14770) Erasure Coding: reconstructed internal block is error

2019-08-27 Thread HuangTao (Jira)
[ https://issues.apache.org/jira/browse/HDFS-14770?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16916560#comment-16916560 ] HuangTao commented on HDFS-14770: - [~jojochuang] I provide a PR, PTAL > Erasure Coding: reconstructed

[jira] [Issue Comment Deleted] (HDFS-14770) Erasure Coding: reconstructed internal block is error

2019-08-27 Thread HuangTao (Jira)
[ https://issues.apache.org/jira/browse/HDFS-14770?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] HuangTao updated HDFS-14770: Comment: was deleted (was: [~jojochuang] I provide a PR, PTAL) > Erasure Coding: reconstructed internal

[jira] [Commented] (HDFS-14770) Erasure Coding: reconstructed internal block is error

2019-08-27 Thread HuangTao (Jira)
[ https://issues.apache.org/jira/browse/HDFS-14770?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16916580#comment-16916580 ] HuangTao commented on HDFS-14770: - I just find I miss patch HDFS-8668,

[jira] [Resolved] (HDFS-14770) Erasure Coding: reconstructed internal block is error

2019-08-27 Thread HuangTao (Jira)
[ https://issues.apache.org/jira/browse/HDFS-14770?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] HuangTao resolved HDFS-14770. - Resolution: Not A Problem > Erasure Coding: reconstructed internal block is error >

[jira] [Created] (HDFS-14770) Erasure Coding: reconstructed internal block is error

2019-08-23 Thread HuangTao (Jira)
HuangTao created HDFS-14770: --- Summary: Erasure Coding: reconstructed internal block is error Key: HDFS-14770 URL: https://issues.apache.org/jira/browse/HDFS-14770 Project: Hadoop HDFS Issue Type:

[jira] [Commented] (HDFS-14770) Erasure Coding: reconstructed internal block is error

2019-08-23 Thread HuangTao (Jira)
[ https://issues.apache.org/jira/browse/HDFS-14770?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16914207#comment-16914207 ] HuangTao commented on HDFS-14770: - Right now, I have no clue about why that happen yet. I am trying to

[jira] [Comment Edited] (HDFS-14699) Erasure Coding: Storage not considered in live replica when replication streams hard limit reached to threshold

2019-09-11 Thread HuangTao (Jira)
[ https://issues.apache.org/jira/browse/HDFS-14699?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16927356#comment-16927356 ] HuangTao edited comment on HDFS-14699 at 9/11/19 8:06 AM: -- [~surendrasingh]

[jira] [Commented] (HDFS-14699) Erasure Coding: Storage not considered in live replica when replication streams hard limit reached to threshold

2019-09-11 Thread HuangTao (Jira)
[ https://issues.apache.org/jira/browse/HDFS-14699?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16927356#comment-16927356 ] HuangTao commented on HDFS-14699: - [~surendrasingh] [~zhaoyim] you are right, I take account HDFS-14768

[jira] [Comment Edited] (HDFS-14699) Erasure Coding: Storage not considered in live replica when replication streams hard limit reached to threshold

2019-09-10 Thread HuangTao (Jira)
[ https://issues.apache.org/jira/browse/HDFS-14699?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16926506#comment-16926506 ] HuangTao edited comment on HDFS-14699 at 9/10/19 10:38 AM: --- I make a test, if

[jira] [Commented] (HDFS-14699) Erasure Coding: Storage not considered in live replica when replication streams hard limit reached to threshold

2019-09-10 Thread HuangTao (Jira)
[ https://issues.apache.org/jira/browse/HDFS-14699?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16926506#comment-16926506 ] HuangTao commented on HDFS-14699: - I make a test, if we move "liveBlockIndices.add(blockIndex);" above to

[jira] [Commented] (HDFS-14768) In some cases, erasure blocks are corruption when they are reconstruct.

2019-09-10 Thread HuangTao (Jira)
[ https://issues.apache.org/jira/browse/HDFS-14768?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16927183#comment-16927183 ] HuangTao commented on HDFS-14768: - [~zhaoyim] you can change the

[jira] [Comment Edited] (HDFS-14699) Erasure Coding: Storage not considered in live replica when replication streams hard limit reached to threshold

2019-09-10 Thread HuangTao (Jira)
[ https://issues.apache.org/jira/browse/HDFS-14699?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16927177#comment-16927177 ] HuangTao edited comment on HDFS-14699 at 9/11/19 2:08 AM: -- {quote}3. then

[jira] [Commented] (HDFS-14699) Erasure Coding: Storage not considered in live replica when replication streams hard limit reached to threshold

2019-09-10 Thread HuangTao (Jira)
[ https://issues.apache.org/jira/browse/HDFS-14699?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16927177#comment-16927177 ] HuangTao commented on HDFS-14699: - I have a doubt about "after wait for about 600s (before the heart

[jira] [Comment Edited] (HDFS-14699) Erasure Coding: Storage not considered in live replica when replication streams hard limit reached to threshold

2019-09-10 Thread HuangTao (Jira)
[ https://issues.apache.org/jira/browse/HDFS-14699?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16927177#comment-16927177 ] HuangTao edited comment on HDFS-14699 at 9/11/19 1:40 AM: -- {quote}3. then

[jira] [Comment Edited] (HDFS-14699) Erasure Coding: Storage not considered in live replica when replication streams hard limit reached to threshold

2019-09-11 Thread HuangTao (Jira)
[ https://issues.apache.org/jira/browse/HDFS-14699?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16927356#comment-16927356 ] HuangTao edited comment on HDFS-14699 at 9/11/19 9:02 AM: -- [~surendrasingh]

[jira] [Comment Edited] (HDFS-14847) Blocks are over-replicated while EC decommissioning

2019-09-14 Thread HuangTao (Jira)
[ https://issues.apache.org/jira/browse/HDFS-14847?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16929693#comment-16929693 ] HuangTao edited comment on HDFS-14847 at 9/14/19 6:13 AM: -- [~ayushtkn] Yes, we

[jira] [Commented] (HDFS-14847) Blocks are over-replicated while EC decommissioning

2019-09-14 Thread HuangTao (Jira)
[ https://issues.apache.org/jira/browse/HDFS-14847?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16929693#comment-16929693 ] HuangTao commented on HDFS-14847: - [~ayushtkn] Yes, we had the same issue, however we have fixed it with

[jira] [Commented] (HDFS-14847) Blocks are over-replicated while EC decommissioning

2019-09-14 Thread HuangTao (Jira)
[ https://issues.apache.org/jira/browse/HDFS-14847?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16929715#comment-16929715 ] HuangTao commented on HDFS-14847: - I just verified with [~ferhui]'s UT and my snippet, and failed. I

[jira] [Updated] (HDFS-14849) Erasure Coding: replicate block infinitely when datanode being decommissioning

2019-09-14 Thread HuangTao (Jira)
[ https://issues.apache.org/jira/browse/HDFS-14849?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] HuangTao updated HDFS-14849: Attachment: HDFS-14849.001.patch Fix Version/s: 3.3.0 Target Version/s: 3.3.0

[jira] [Commented] (HDFS-14847) Blocks are over-replicated while EC decommissioning

2019-09-14 Thread HuangTao (Jira)
[ https://issues.apache.org/jira/browse/HDFS-14847?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16929885#comment-16929885 ] HuangTao commented on HDFS-14847: - I have submitted HDFS-14849 to fix my bug > Blocks are

[jira] [Created] (HDFS-14849) Erasure Coding: replicate block infinitely when datanode being decommissioning

2019-09-14 Thread HuangTao (Jira)
HuangTao created HDFS-14849: --- Summary: Erasure Coding: replicate block infinitely when datanode being decommissioning Key: HDFS-14849 URL: https://issues.apache.org/jira/browse/HDFS-14849 Project: Hadoop

[jira] [Updated] (HDFS-14849) Erasure Coding: replicate block infinitely when datanode being decommissioning

2019-09-19 Thread HuangTao (Jira)
[ https://issues.apache.org/jira/browse/HDFS-14849?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] HuangTao updated HDFS-14849: Attachment: scheduleReconstruction.png > Erasure Coding: replicate block infinitely when datanode being

[jira] [Commented] (HDFS-14849) Erasure Coding: replicate block infinitely when datanode being decommissioning

2019-09-19 Thread HuangTao (Jira)
[ https://issues.apache.org/jira/browse/HDFS-14849?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16933376#comment-16933376 ] HuangTao commented on HDFS-14849: - I find a clue: the `chooseSourceDatanodes` get {quote}LIVE=2,

[jira] [Updated] (HDFS-14849) Erasure Coding: replicate block infinitely when datanode being decommissioning

2019-09-19 Thread HuangTao (Jira)
[ https://issues.apache.org/jira/browse/HDFS-14849?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] HuangTao updated HDFS-14849: Attachment: fsck-file.png > Erasure Coding: replicate block infinitely when datanode being decommissioning

[jira] [Updated] (HDFS-14849) Erasure Coding: replicate block infinitely when datanode being decommissioning

2019-09-19 Thread HuangTao (Jira)
[ https://issues.apache.org/jira/browse/HDFS-14849?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] HuangTao updated HDFS-14849: Description: When the datanode keeping in DECOMMISSION_INPROGRESS status, the EC block in that datanode

[jira] [Updated] (HDFS-14849) Erasure Coding: replicate block infinitely when datanode being decommissioning

2019-09-19 Thread HuangTao (Jira)
[ https://issues.apache.org/jira/browse/HDFS-14849?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] HuangTao updated HDFS-14849: Attachment: liveBlockIndices.png > Erasure Coding: replicate block infinitely when datanode being

[jira] [Issue Comment Deleted] (HDFS-14849) Erasure Coding: replicate block infinitely when datanode being decommissioning

2019-09-19 Thread HuangTao (Jira)
[ https://issues.apache.org/jira/browse/HDFS-14849?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] HuangTao updated HDFS-14849: Comment: was deleted (was: !liveBlockIndices.png! ) > Erasure Coding: replicate block infinitely when

[jira] [Comment Edited] (HDFS-14849) Erasure Coding: replicate block infinitely when datanode being decommissioning

2019-09-19 Thread HuangTao (Jira)
[ https://issues.apache.org/jira/browse/HDFS-14849?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16933376#comment-16933376 ] HuangTao edited comment on HDFS-14849 at 9/19/19 1:35 PM: -- I find a clue: the

[jira] [Commented] (HDFS-14849) Erasure Coding: replicate block infinitely when datanode being decommissioning

2019-09-19 Thread HuangTao (Jira)
[ https://issues.apache.org/jira/browse/HDFS-14849?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16933382#comment-16933382 ] HuangTao commented on HDFS-14849: - !liveBlockIndices.png! > Erasure Coding: replicate block infinitely

[jira] [Updated] (HDFS-14849) Erasure Coding: replicate block infinitely when datanode being decommissioning

2019-09-19 Thread HuangTao (Jira)
[ https://issues.apache.org/jira/browse/HDFS-14849?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] HuangTao updated HDFS-14849: Description: When the datanode keeping in DECOMMISSION_INPROGRESS status, the EC block in that datanode

[jira] [Updated] (HDFS-14849) Erasure Coding: replicate block infinitely when datanode being decommissioning

2019-09-19 Thread HuangTao (Jira)
[ https://issues.apache.org/jira/browse/HDFS-14849?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] HuangTao updated HDFS-14849: Description: When the datanode keeping in DECOMMISSION_INPROGRESS status, the EC block in that datanode

[jira] [Updated] (HDFS-14849) Erasure Coding: replicate block infinitely when datanode being decommissioning

2019-09-19 Thread HuangTao (Jira)
[ https://issues.apache.org/jira/browse/HDFS-14849?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] HuangTao updated HDFS-14849: Attachment: HDFS-14849.002.patch > Erasure Coding: replicate block infinitely when datanode being

[jira] [Commented] (HDFS-14849) Erasure Coding: replicate block infinitely when datanode being decommissioning

2019-09-19 Thread HuangTao (Jira)
[ https://issues.apache.org/jira/browse/HDFS-14849?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16933266#comment-16933266 ] HuangTao commented on HDFS-14849: - [~ferhui] We have the same scenario, but our fix can't pass each

[jira] [Comment Edited] (HDFS-14849) Erasure Coding: replicate block infinitely when datanode being decommissioning

2019-09-19 Thread HuangTao (Jira)
[ https://issues.apache.org/jira/browse/HDFS-14849?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16933376#comment-16933376 ] HuangTao edited comment on HDFS-14849 at 9/19/19 1:30 PM: -- I find a clue: the

[jira] [Commented] (HDFS-14849) Erasure Coding: replicate block infinitely when datanode being decommissioning

2019-09-19 Thread HuangTao (Jira)
[ https://issues.apache.org/jira/browse/HDFS-14849?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16933455#comment-16933455 ] HuangTao commented on HDFS-14849: - [~ferhui] I am trying to find the root cause, will sync here. >

[jira] [Commented] (HDFS-14849) Erasure Coding: replicate block infinitely when datanode being decommissioning

2019-09-19 Thread HuangTao (Jira)
[ https://issues.apache.org/jira/browse/HDFS-14849?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16933911#comment-16933911 ] HuangTao commented on HDFS-14849: - Yes [~ferhui], just more than one, sometimes the number of an internal

[jira] [Commented] (HDFS-14768) In some cases, erasure blocks are corruption when they are reconstruct.

2019-09-18 Thread HuangTao (Jira)
[ https://issues.apache.org/jira/browse/HDFS-14768?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16932966#comment-16932966 ] HuangTao commented on HDFS-14768: - I think the reason is that the busy node can't be in busy status

[jira] [Updated] (HDFS-14849) Erasure Coding: the internal block is replicated many times when datanode is decommissioning

2019-09-22 Thread HuangTao (Jira)
[ https://issues.apache.org/jira/browse/HDFS-14849?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] HuangTao updated HDFS-14849: Description: When the datanode keeping in DECOMMISSION_INPROGRESS status, the EC internal block in that

[jira] [Updated] (HDFS-14849) Erasure Coding: the internal block is replicated many times when datanode is decommissioning

2019-09-22 Thread HuangTao (Jira)
[ https://issues.apache.org/jira/browse/HDFS-14849?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] HuangTao updated HDFS-14849: Summary: Erasure Coding: the internal block is replicated many times when datanode is decommissioning

[jira] [Commented] (HDFS-15013) Reduce NameNode overview tab response time

2019-11-26 Thread HuangTao (Jira)
[ https://issues.apache.org/jira/browse/HDFS-15013?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16982235#comment-16982235 ] HuangTao commented on HDFS-15013: - [~inigoiri] I have checked federationhealth.js, there is no seperate

[jira] [Created] (HDFS-15013) Reduce NameNode overview tab response time

2019-11-25 Thread HuangTao (Jira)
HuangTao created HDFS-15013: --- Summary: Reduce NameNode overview tab response time Key: HDFS-15013 URL: https://issues.apache.org/jira/browse/HDFS-15013 Project: Hadoop HDFS Issue Type: Improvement

[jira] [Updated] (HDFS-15013) Reduce NameNode overview tab response time

2019-11-25 Thread HuangTao (Jira)
[ https://issues.apache.org/jira/browse/HDFS-15013?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] HuangTao updated HDFS-15013: Description: Now, the overview tab load /conf synchronously as follow picture.

[jira] [Updated] (HDFS-15013) Reduce NameNode overview tab response time

2019-11-25 Thread HuangTao (Jira)
[ https://issues.apache.org/jira/browse/HDFS-15013?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] HuangTao updated HDFS-15013: Description: Now, the overview tab load /conf synchronously as follow picture.

[jira] [Updated] (HDFS-15013) Reduce NameNode overview tab response time

2019-11-25 Thread HuangTao (Jira)
[ https://issues.apache.org/jira/browse/HDFS-15013?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] HuangTao updated HDFS-15013: Attachment: HDFS-15013.001.patch Status: Patch Available (was: Open) > Reduce NameNode overview

[jira] [Commented] (HDFS-15013) Reduce NameNode overview tab response time

2019-11-25 Thread HuangTao (Jira)
[ https://issues.apache.org/jira/browse/HDFS-15013?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16982089#comment-16982089 ] HuangTao commented on HDFS-15013: - [~surendrasingh] PTAL > Reduce NameNode overview tab response time >

[jira] [Commented] (HDFS-15013) Reduce NameNode overview tab response time

2019-11-27 Thread HuangTao (Jira)
[ https://issues.apache.org/jira/browse/HDFS-15013?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16984072#comment-16984072 ] HuangTao commented on HDFS-15013: - [~surendrasingh] [~ayushtkn] sorry, I didn't think of non-HA mode, I

[jira] [Updated] (HDFS-15013) Reduce NameNode overview tab response time

2019-11-27 Thread HuangTao (Jira)
[ https://issues.apache.org/jira/browse/HDFS-15013?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] HuangTao updated HDFS-15013: Fix Version/s: 3.3.0 > Reduce NameNode overview tab response time >

[jira] [Commented] (HDFS-15013) Reduce NameNode overview tab response time

2019-11-27 Thread HuangTao (Jira)
[ https://issues.apache.org/jira/browse/HDFS-15013?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16984093#comment-16984093 ] HuangTao commented on HDFS-15013: - Upload v002 patch. fix non-ha mode > Reduce NameNode overview tab

[jira] [Comment Edited] (HDFS-15013) Reduce NameNode overview tab response time

2019-11-27 Thread HuangTao (Jira)
[ https://issues.apache.org/jira/browse/HDFS-15013?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16984072#comment-16984072 ] HuangTao edited comment on HDFS-15013 at 11/28/19 2:32 AM: --- [~surendrasingh]  

[jira] [Comment Edited] (HDFS-15013) Reduce NameNode overview tab response time

2019-11-27 Thread HuangTao (Jira)
[ https://issues.apache.org/jira/browse/HDFS-15013?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16984072#comment-16984072 ] HuangTao edited comment on HDFS-15013 at 11/28/19 1:41 AM: --- [~surendrasingh]  

[jira] [Updated] (HDFS-15013) Reduce NameNode overview tab response time

2019-11-27 Thread HuangTao (Jira)
[ https://issues.apache.org/jira/browse/HDFS-15013?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] HuangTao updated HDFS-15013: Attachment: HDFS-15013.002.patch > Reduce NameNode overview tab response time >

[jira] [Commented] (HDFS-15186) Erasure Coding: Decommission may generate the parity block's content with all 0 in some case

2020-02-20 Thread HuangTao (Jira)
[ https://issues.apache.org/jira/browse/HDFS-15186?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17040767#comment-17040767 ] HuangTao commented on HDFS-15186: - In our production, we meet same issue, and I have discussed with

[jira] [Updated] (HDFS-15240) Erasure Coding: dirty buffer causes reconstruction block error

2020-04-09 Thread HuangTao (Jira)
[ https://issues.apache.org/jira/browse/HDFS-15240?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] HuangTao updated HDFS-15240: Affects Version/s: (was: 3.3.0) 3.3.1 > Erasure Coding: dirty buffer causes

[jira] [Commented] (HDFS-15274) NN doesn't remove the blocks from the failed DatanodeStorageInfo

2020-04-13 Thread HuangTao (Jira)
[ https://issues.apache.org/jira/browse/HDFS-15274?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17082348#comment-17082348 ] HuangTao commented on HDFS-15274: - All failed test cases can pass locally > NN doesn't remove the blocks

[jira] [Commented] (HDFS-15274) NN doesn't remove the blocks from the failed DatanodeStorageInfo

2020-04-13 Thread HuangTao (Jira)
[ https://issues.apache.org/jira/browse/HDFS-15274?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17082347#comment-17082347 ] HuangTao commented on HDFS-15274: - retrigger QA > NN doesn't remove the blocks from the failed

[jira] [Commented] (HDFS-15140) Replace FoldedTreeSet in Datanode with SortedSet or TreeMap

2020-04-15 Thread HuangTao (Jira)
[ https://issues.apache.org/jira/browse/HDFS-15140?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17083996#comment-17083996 ] HuangTao commented on HDFS-15140: - about the patch, I find two points need to change {code}

[jira] [Updated] (HDFS-15240) Erasure Coding: dirty buffer causes reconstruction block error

2020-04-07 Thread HuangTao (Jira)
[ https://issues.apache.org/jira/browse/HDFS-15240?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] HuangTao updated HDFS-15240: Affects Version/s: 3.3.0 > Erasure Coding: dirty buffer causes reconstruction block error >

[jira] [Comment Edited] (HDFS-15240) Erasure Coding: dirty buffer causes reconstruction block error

2020-04-07 Thread HuangTao (Jira)
[ https://issues.apache.org/jira/browse/HDFS-15240?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17077745#comment-17077745 ] HuangTao edited comment on HDFS-15240 at 4/8/20, 2:10 AM: -- Hi watchers, anyone

[jira] [Commented] (HDFS-15240) Erasure Coding: dirty buffer causes reconstruction block error

2020-04-07 Thread HuangTao (Jira)
[ https://issues.apache.org/jira/browse/HDFS-15240?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17077745#comment-17077745 ] HuangTao commented on HDFS-15240: - Hi watchers, anyone would like to help take a review? > Erasure

[jira] [Commented] (HDFS-15274) NN doesn't remove the blocks from the failed DatanodeStorageInfo

2020-04-13 Thread HuangTao (Jira)
[ https://issues.apache.org/jira/browse/HDFS-15274?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17082766#comment-17082766 ] HuangTao commented on HDFS-15274: - cc [~hexiaoqiao] could you help me trigger jenkins again? > NN

[jira] [Updated] (HDFS-15274) NN doesn't remove the blocks from the failed DatanodeStorageInfo

2020-04-12 Thread HuangTao (Jira)
[ https://issues.apache.org/jira/browse/HDFS-15274?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] HuangTao updated HDFS-15274: Fix Version/s: (was: 3.4.0) > NN doesn't remove the blocks from the failed DatanodeStorageInfo >

[jira] [Updated] (HDFS-15274) NN doesn't remove the blocks from the failed DatanodeStorageInfo

2020-04-12 Thread HuangTao (Jira)
[ https://issues.apache.org/jira/browse/HDFS-15274?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] HuangTao updated HDFS-15274: Affects Version/s: 3.4.0 > NN doesn't remove the blocks from the failed DatanodeStorageInfo >

[jira] [Created] (HDFS-15274) NN doesn't remove the blocks from the failed DatanodeStorageInfo

2020-04-12 Thread HuangTao (Jira)
HuangTao created HDFS-15274: --- Summary: NN doesn't remove the blocks from the failed DatanodeStorageInfo Key: HDFS-15274 URL: https://issues.apache.org/jira/browse/HDFS-15274 Project: Hadoop HDFS

[jira] [Updated] (HDFS-15274) NN doesn't remove the blocks from the failed DatanodeStorageInfo

2020-04-12 Thread HuangTao (Jira)
[ https://issues.apache.org/jira/browse/HDFS-15274?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] HuangTao updated HDFS-15274: Attachment: HDFS-15274.001.patch Status: Patch Available (was: Open) > NN doesn't remove the

[jira] [Commented] (HDFS-15274) NN doesn't remove the blocks from the failed DatanodeStorageInfo

2020-04-14 Thread HuangTao (Jira)
[ https://issues.apache.org/jira/browse/HDFS-15274?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17082972#comment-17082972 ] HuangTao commented on HDFS-15274: - Thanks, [~hexiaoqiao]. There are still some failed tests, I will check

[jira] [Commented] (HDFS-15274) NN doesn't remove the blocks from the failed DatanodeStorageInfo

2020-04-13 Thread HuangTao (Jira)
[ https://issues.apache.org/jira/browse/HDFS-15274?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17082132#comment-17082132 ] HuangTao commented on HDFS-15274: - [~daryn] please help to take a look. > NN doesn't remove the blocks

[jira] [Updated] (HDFS-15240) Erasure Coding: dirty buffer causes reconstruction block error

2020-04-10 Thread HuangTao (Jira)
[ https://issues.apache.org/jira/browse/HDFS-15240?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] HuangTao updated HDFS-15240: Affects Version/s: (was: 3.3.1) > Erasure Coding: dirty buffer causes reconstruction block error >

[jira] [Updated] (HDFS-15240) Erasure Coding: dirty buffer causes reconstruction block error

2020-04-10 Thread HuangTao (Jira)
[ https://issues.apache.org/jira/browse/HDFS-15240?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] HuangTao updated HDFS-15240: Fix Version/s: 3.4.0 > Erasure Coding: dirty buffer causes reconstruction block error >

[jira] [Updated] (HDFS-15274) NN doesn't remove the blocks from the failed DatanodeStorageInfo

2020-04-13 Thread HuangTao (Jira)
[ https://issues.apache.org/jira/browse/HDFS-15274?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] HuangTao updated HDFS-15274: Attachment: HDFS-15274.002.patch > NN doesn't remove the blocks from the failed DatanodeStorageInfo >

[jira] [Updated] (HDFS-15260) Change from List to Set to improve the chooseTarget performance

2020-04-03 Thread HuangTao (Jira)
[ https://issues.apache.org/jira/browse/HDFS-15260?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] HuangTao updated HDFS-15260: Attachment: (was: choose_target_flamegraph.png) > Change from List to Set to improve the chooseTarget

[jira] [Resolved] (HDFS-15260) Change from List to Set to improve the chooseTarget performance

2020-04-04 Thread HuangTao (Jira)
[ https://issues.apache.org/jira/browse/HDFS-15260?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] HuangTao resolved HDFS-15260. - Resolution: Invalid It is our custom version > Change from List to Set to improve the chooseTarget

[jira] [Created] (HDFS-15260) Change from List to Set to improve the chooseTarget performance

2020-04-03 Thread HuangTao (Jira)
HuangTao created HDFS-15260: --- Summary: Change from List to Set to improve the chooseTarget performance Key: HDFS-15260 URL: https://issues.apache.org/jira/browse/HDFS-15260 Project: Hadoop HDFS

[jira] [Updated] (HDFS-15260) Change from List to Set to improve the chooseTarget performance

2020-04-03 Thread HuangTao (Jira)
[ https://issues.apache.org/jira/browse/HDFS-15260?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] HuangTao updated HDFS-15260: Attachment: choose_target_flamegraph.png > Change from List to Set to improve the chooseTarget performance

[jira] [Updated] (HDFS-15240) Erasure Coding: dirty buffer causes reconstruction block error

2020-04-03 Thread HuangTao (Jira)
[ https://issues.apache.org/jira/browse/HDFS-15240?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] HuangTao updated HDFS-15240: Attachment: HDFS-15240.003.patch > Erasure Coding: dirty buffer causes reconstruction block error >

[jira] [Commented] (HDFS-15240) Erasure Coding: dirty buffer causes reconstruction block error

2020-04-03 Thread HuangTao (Jira)
[ https://issues.apache.org/jira/browse/HDFS-15240?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17075021#comment-17075021 ] HuangTao commented on HDFS-15240: - fix checkstyle and the unit case > Erasure Coding: dirty buffer

[jira] [Updated] (HDFS-15240) Erasure Coding: dirty buffer causes reconstruction block error

2020-04-04 Thread HuangTao (Jira)
[ https://issues.apache.org/jira/browse/HDFS-15240?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] HuangTao updated HDFS-15240: Attachment: HDFS-15240.004.patch > Erasure Coding: dirty buffer causes reconstruction block error >

[jira] [Commented] (HDFS-15240) Erasure Coding: dirty buffer causes reconstruction block error

2020-04-05 Thread HuangTao (Jira)
[ https://issues.apache.org/jira/browse/HDFS-15240?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17075854#comment-17075854 ] HuangTao commented on HDFS-15240: - the failed test is unrelated with this patch. [~weichiu] PTAL, Thanks

[jira] [Updated] (HDFS-15240) Erasure Coding: dirty buffer causes reconstruction block error

2020-04-05 Thread HuangTao (Jira)
[ https://issues.apache.org/jira/browse/HDFS-15240?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] HuangTao updated HDFS-15240: Attachment: HDFS-15240.005.patch > Erasure Coding: dirty buffer causes reconstruction block error >

[jira] [Commented] (HDFS-15240) Erasure Coding: dirty buffer causes reconstruction block error

2020-03-26 Thread HuangTao (Jira)
[ https://issues.apache.org/jira/browse/HDFS-15240?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17067871#comment-17067871 ] HuangTao commented on HDFS-15240: - I will try to add an UT > Erasure Coding: dirty buffer causes

  1   2   >