[jira] [Commented] (HDFS-13901) INode access time is ignored because of race between open and rename

2019-10-21 Thread Wei-Chiu Chuang (Jira)
[ https://issues.apache.org/jira/browse/HDFS-13901?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16956560#comment-16956560 ] Wei-Chiu Chuang commented on HDFS-13901: Pushed to trunk. The patch has conflicts with branch-3.2

[jira] [Updated] (HDFS-13901) INode access time is ignored because of race between open and rename

2019-10-21 Thread Wei-Chiu Chuang (Jira)
[ https://issues.apache.org/jira/browse/HDFS-13901?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wei-Chiu Chuang updated HDFS-13901: --- Fix Version/s: 3.3.0 > INode access time is ignored because of race between open and rename

[jira] [Commented] (HDFS-14854) Create improved decommission monitor implementation

2019-10-21 Thread Wei-Chiu Chuang (Jira)
[ https://issues.apache.org/jira/browse/HDFS-14854?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16956552#comment-16956552 ] Wei-Chiu Chuang commented on HDFS-14854: Thanks [~sodonnell] I finally gave it a hard look at the

[jira] [Resolved] (HDFS-12049) Recommissioning live nodes stalls the NN

2019-10-21 Thread Wei-Chiu Chuang (Jira)
[ https://issues.apache.org/jira/browse/HDFS-12049?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wei-Chiu Chuang resolved HDFS-12049. Resolution: Duplicate > Recommissioning live nodes stalls the NN >

[jira] [Commented] (HDFS-13901) INode access time is ignored because of race between open and rename

2019-10-21 Thread Wei-Chiu Chuang (Jira)
[ https://issues.apache.org/jira/browse/HDFS-13901?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16956438#comment-16956438 ] Wei-Chiu Chuang commented on HDFS-13901: +1. Thanks [~LiJinglun] you're obviously smarter than I

[jira] [Updated] (HDFS-14768) EC : Busy DN replica should be consider in live replica check.

2019-10-21 Thread Wei-Chiu Chuang (Jira)
[ https://issues.apache.org/jira/browse/HDFS-14768?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wei-Chiu Chuang updated HDFS-14768: --- Fix Version/s: (was: 3.3.0) > EC : Busy DN replica should be consider in live replica

[jira] [Commented] (HDFS-14884) Add sanity check that zone key equals feinfo key while setting Xattrs

2019-10-21 Thread Wei-Chiu Chuang (Jira)
[ https://issues.apache.org/jira/browse/HDFS-14884?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16956328#comment-16956328 ] Wei-Chiu Chuang commented on HDFS-14884: LGTM +1 pending Jenkins. > Add sanity check that zone

[jira] [Commented] (HDFS-14884) Add sanity check that zone key equals feinfo key while setting Xattrs

2019-10-21 Thread Wei-Chiu Chuang (Jira)
[ https://issues.apache.org/jira/browse/HDFS-14884?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16956276#comment-16956276 ] Wei-Chiu Chuang commented on HDFS-14884: LGTM Nit: {code} import static

[jira] [Comment Edited] (HDFS-14884) Add sanity check that zone key equals feinfo key while setting Xattrs

2019-10-21 Thread Wei-Chiu Chuang (Jira)
[ https://issues.apache.org/jira/browse/HDFS-14884?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16956276#comment-16956276 ] Wei-Chiu Chuang edited comment on HDFS-14884 at 10/21/19 4:55 PM: -- Looks

[jira] [Commented] (HDFS-14912) Set dfs.image.string-tables.expanded default to false in branch-2.7

2019-10-18 Thread Wei-Chiu Chuang (Jira)
[ https://issues.apache.org/jira/browse/HDFS-14912?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16955002#comment-16955002 ] Wei-Chiu Chuang commented on HDFS-14912: Started one for you:

[jira] [Assigned] (HDFS-14910) TestRenameWithSnapshots#testRename2PreDescendant failing consistently

2019-10-17 Thread Wei-Chiu Chuang (Jira)
[ https://issues.apache.org/jira/browse/HDFS-14910?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wei-Chiu Chuang reassigned HDFS-14910: -- Assignee: Wei-Chiu Chuang > TestRenameWithSnapshots#testRename2PreDescendant failing

[jira] [Commented] (HDFS-14908) LeaseManager should check parent-child relationship when filter open files.

2019-10-15 Thread Wei-Chiu Chuang (Jira)
[ https://issues.apache.org/jira/browse/HDFS-14908?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16952094#comment-16952094 ] Wei-Chiu Chuang commented on HDFS-14908: [~linyiqun] wanna take a look? > LeaseManager should

[jira] [Updated] (HDFS-14908) LeaseManager should check parent-child relationship when filter open files.

2019-10-15 Thread Wei-Chiu Chuang (Jira)
[ https://issues.apache.org/jira/browse/HDFS-14908?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wei-Chiu Chuang updated HDFS-14908: --- Affects Version/s: 3.1.0 3.0.1 > LeaseManager should check

[jira] [Updated] (HDFS-14271) [SBN read] StandbyException is logged if Observer is the first NameNode

2019-10-13 Thread Wei-Chiu Chuang (Jira)
[ https://issues.apache.org/jira/browse/HDFS-14271?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wei-Chiu Chuang updated HDFS-14271: --- Labels: multi-sbnn (was: ) > [SBN read] StandbyException is logged if Observer is the first

[jira] [Commented] (HDFS-6524) Choosing datanode retries times considering with block replica number

2019-10-13 Thread Wei-Chiu Chuang (Jira)
[ https://issues.apache.org/jira/browse/HDFS-6524?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16950369#comment-16950369 ] Wei-Chiu Chuang commented on HDFS-6524: --- Looks like a good improvement, thanks [~leosun08].

[jira] [Commented] (HDFS-14190) Copying folders containing = - characters between hdfs (using webhdfs) does not work in distcp

2019-10-09 Thread Wei-Chiu Chuang (Jira)
[ https://issues.apache.org/jira/browse/HDFS-14190?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16947850#comment-16947850 ] Wei-Chiu Chuang commented on HDFS-14190: I suspect HDFS-14323 or HDFS-14423 fixed it. > Copying

[jira] [Updated] (HDFS-14902) RBF: NullPointer When Misconfigured

2019-10-08 Thread Wei-Chiu Chuang (Jira)
[ https://issues.apache.org/jira/browse/HDFS-14902?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wei-Chiu Chuang updated HDFS-14902: --- Summary: RBF: NullPointer When Misconfigured (was: NullPointer When Misconfigured) > RBF:

[jira] [Updated] (HDFS-13806) EC: No error message for unsetting EC policy of the directory inherits the erasure coding policy from an ancestor directory

2019-10-04 Thread Wei-Chiu Chuang (Jira)
[ https://issues.apache.org/jira/browse/HDFS-13806?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wei-Chiu Chuang updated HDFS-13806: --- Fix Version/s: 3.1.4 > EC: No error message for unsetting EC policy of the directory

[jira] [Updated] (HDFS-14497) Write lock held by metasave impact following RPC processing

2019-10-04 Thread Wei-Chiu Chuang (Jira)
[ https://issues.apache.org/jira/browse/HDFS-14497?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wei-Chiu Chuang updated HDFS-14497: --- Fix Version/s: 3.2.2 3.1.4 > Write lock held by metasave impact following

[jira] [Commented] (HDFS-2470) NN should automatically set permissions on dfs.namenode.*.dir

2019-10-04 Thread Wei-Chiu Chuang (Jira)
[ https://issues.apache.org/jira/browse/HDFS-2470?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16944767#comment-16944767 ] Wei-Chiu Chuang commented on HDFS-2470: --- Thanks! Just in time! > NN should automatically set

[jira] [Updated] (HDFS-14890) Setting permissions on name directory fails on non posix compliant filesystems

2019-10-04 Thread Wei-Chiu Chuang (Jira)
[ https://issues.apache.org/jira/browse/HDFS-14890?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wei-Chiu Chuang updated HDFS-14890: --- Fix Version/s: 3.1.4 > Setting permissions on name directory fails on non posix compliant

[jira] [Commented] (HDFS-2470) NN should automatically set permissions on dfs.namenode.*.dir

2019-10-04 Thread Wei-Chiu Chuang (Jira)
[ https://issues.apache.org/jira/browse/HDFS-2470?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16944750#comment-16944750 ] Wei-Chiu Chuang commented on HDFS-2470: --- Pushed to branch-3.1 with trivial conflicts. Attached

[jira] [Updated] (HDFS-2470) NN should automatically set permissions on dfs.namenode.*.dir

2019-10-04 Thread Wei-Chiu Chuang (Jira)
[ https://issues.apache.org/jira/browse/HDFS-2470?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wei-Chiu Chuang updated HDFS-2470: -- Attachment: HDFS-2470.branch-3.1.patch > NN should automatically set permissions on

[jira] [Updated] (HDFS-2470) NN should automatically set permissions on dfs.namenode.*.dir

2019-10-04 Thread Wei-Chiu Chuang (Jira)
[ https://issues.apache.org/jira/browse/HDFS-2470?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wei-Chiu Chuang updated HDFS-2470: -- Fix Version/s: 3.1.4 > NN should automatically set permissions on dfs.namenode.*.dir >

[jira] [Updated] (HDFS-14892) Close the output stream if createWrappedOutputStream() fails

2019-10-04 Thread Wei-Chiu Chuang (Jira)
[ https://issues.apache.org/jira/browse/HDFS-14892?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wei-Chiu Chuang updated HDFS-14892: --- Component/s: encryption > Close the output stream if createWrappedOutputStream() fails >

[jira] [Updated] (HDFS-14686) HttpFS: HttpFSFileSystem#getErasureCodingPolicy always returns null

2019-10-04 Thread Wei-Chiu Chuang (Jira)
[ https://issues.apache.org/jira/browse/HDFS-14686?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wei-Chiu Chuang updated HDFS-14686: --- Fix Version/s: 3.2.2 3.1.4 > HttpFS:

[jira] [Commented] (HDFS-14892) Close the output stream if createWrappedOutputStream() fails

2019-10-04 Thread Wei-Chiu Chuang (Jira)
[ https://issues.apache.org/jira/browse/HDFS-14892?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16944675#comment-16944675 ] Wei-Chiu Chuang commented on HDFS-14892: Good finding, Kihwal. I am aware the client creates an

[jira] [Updated] (HDFS-13693) Remove unnecessary search in INodeDirectory.addChild during image loading

2019-10-04 Thread Wei-Chiu Chuang (Jira)
[ https://issues.apache.org/jira/browse/HDFS-13693?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wei-Chiu Chuang updated HDFS-13693: --- Fix Version/s: 3.2.2 3.1.4 > Remove unnecessary search in

[jira] [Updated] (HDFS-14637) Namenode may not replicate blocks to meet the policy after enabling upgradeDomain

2019-10-03 Thread Wei-Chiu Chuang (Jira)
[ https://issues.apache.org/jira/browse/HDFS-14637?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wei-Chiu Chuang updated HDFS-14637: --- Fix Version/s: 3.2.2 3.1.4 3.3.0 Resolution:

[jira] [Updated] (HDFS-14637) Namenode may not replicate blocks to meet the policy after enabling upgradeDomain

2019-10-03 Thread Wei-Chiu Chuang (Jira)
[ https://issues.apache.org/jira/browse/HDFS-14637?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wei-Chiu Chuang updated HDFS-14637: --- Attachment: HDFS-14637.branch-3.2.patch HDFS-14637.branch-3.1.patch >

[jira] [Updated] (HDFS-14850) Optimize FileSystemAccessService#getFileSystemConfiguration

2019-10-03 Thread Wei-Chiu Chuang (Jira)
[ https://issues.apache.org/jira/browse/HDFS-14850?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wei-Chiu Chuang updated HDFS-14850: --- Fix Version/s: 3.2.2 3.1.4 > Optimize

[jira] [Updated] (HDFS-14527) Stop all DataNodes may result in NN terminate

2019-10-03 Thread Wei-Chiu Chuang (Jira)
[ https://issues.apache.org/jira/browse/HDFS-14527?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wei-Chiu Chuang updated HDFS-14527: --- Fix Version/s: 3.1.4 > Stop all DataNodes may result in NN terminate >

[jira] [Updated] (HDFS-14624) When decommissioning a node, log remaining blocks to replicate periodically

2019-10-03 Thread Wei-Chiu Chuang (Jira)
[ https://issues.apache.org/jira/browse/HDFS-14624?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wei-Chiu Chuang updated HDFS-14624: --- Fix Version/s: 3.2.2 3.1.4 > When decommissioning a node, log remaining

[jira] [Updated] (HDFS-14499) Misleading REM_QUOTA value with snapshot and trash feature enabled for a directory

2019-10-03 Thread Wei-Chiu Chuang (Jira)
[ https://issues.apache.org/jira/browse/HDFS-14499?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wei-Chiu Chuang updated HDFS-14499: --- Fix Version/s: 3.2.2 3.1.4 > Misleading REM_QUOTA value with snapshot and

[jira] [Updated] (HDFS-14113) EC : Add Configuration to restrict UserDefined Policies

2019-10-03 Thread Wei-Chiu Chuang (Jira)
[ https://issues.apache.org/jira/browse/HDFS-14113?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wei-Chiu Chuang updated HDFS-14113: --- Fix Version/s: 3.2.2 3.1.4 > EC : Add Configuration to restrict

[jira] [Updated] (HDFS-14124) EC : Support EC Commands (set/get/unset EcPolicy) via WebHdfs

2019-10-03 Thread Wei-Chiu Chuang (Jira)
[ https://issues.apache.org/jira/browse/HDFS-14124?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wei-Chiu Chuang updated HDFS-14124: --- Attachment: HDFS-14124.branch-3.1.patch > EC : Support EC Commands (set/get/unset EcPolicy)

[jira] [Updated] (HDFS-14124) EC : Support EC Commands (set/get/unset EcPolicy) via WebHdfs

2019-10-03 Thread Wei-Chiu Chuang (Jira)
[ https://issues.apache.org/jira/browse/HDFS-14124?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wei-Chiu Chuang updated HDFS-14124: --- Fix Version/s: 3.1.4 > EC : Support EC Commands (set/get/unset EcPolicy) via WebHdfs >

[jira] [Commented] (HDFS-14124) EC : Support EC Commands (set/get/unset EcPolicy) via WebHdfs

2019-10-03 Thread Wei-Chiu Chuang (Jira)
[ https://issues.apache.org/jira/browse/HDFS-14124?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16944129#comment-16944129 ] Wei-Chiu Chuang commented on HDFS-14124: Pushed to branch-3.1. There's just a trivial conflict in

[jira] [Updated] (HDFS-14187) Make warning message more clear when there are not enough data nodes for EC write

2019-10-03 Thread Wei-Chiu Chuang (Jira)
[ https://issues.apache.org/jira/browse/HDFS-14187?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wei-Chiu Chuang updated HDFS-14187: --- Fix Version/s: 3.2.2 3.1.4 > Make warning message more clear when there

[jira] [Commented] (HDFS-14064) WEBHDFS: Support Enable/Disable EC Policy

2019-10-03 Thread Wei-Chiu Chuang (Jira)
[ https://issues.apache.org/jira/browse/HDFS-14064?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16944111#comment-16944111 ] Wei-Chiu Chuang commented on HDFS-14064: Pushed to branch-3.1 There's just a trivial conflict.

[jira] [Updated] (HDFS-14064) WEBHDFS: Support Enable/Disable EC Policy

2019-10-03 Thread Wei-Chiu Chuang (Jira)
[ https://issues.apache.org/jira/browse/HDFS-14064?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wei-Chiu Chuang updated HDFS-14064: --- Fix Version/s: 3.1.4 > WEBHDFS: Support Enable/Disable EC Policy >

[jira] [Updated] (HDFS-14064) WEBHDFS: Support Enable/Disable EC Policy

2019-10-03 Thread Wei-Chiu Chuang (Jira)
[ https://issues.apache.org/jira/browse/HDFS-14064?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wei-Chiu Chuang updated HDFS-14064: --- Attachment: HDFS-14064.branch-3.1.patch > WEBHDFS: Support Enable/Disable EC Policy >

[jira] [Updated] (HDFS-14064) WEBHDFS: Support Enable/Disable EC Policy

2019-10-03 Thread Wei-Chiu Chuang (Jira)
[ https://issues.apache.org/jira/browse/HDFS-14064?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wei-Chiu Chuang updated HDFS-14064: --- Component/s: webhdfs erasure-coding > WEBHDFS: Support Enable/Disable EC

[jira] [Commented] (HDFS-14849) Erasure Coding: the internal block is replicated many times when datanode is decommissioning

2019-10-03 Thread Wei-Chiu Chuang (Jira)
[ https://issues.apache.org/jira/browse/HDFS-14849?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16944096#comment-16944096 ] Wei-Chiu Chuang commented on HDFS-14849: Cherrypicked the commit to branch-3.2 without conflicts.

[jira] [Updated] (HDFS-14849) Erasure Coding: the internal block is replicated many times when datanode is decommissioning

2019-10-03 Thread Wei-Chiu Chuang (Jira)
[ https://issues.apache.org/jira/browse/HDFS-14849?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wei-Chiu Chuang updated HDFS-14849: --- Fix Version/s: 3.1.4 > Erasure Coding: the internal block is replicated many times when

[jira] [Updated] (HDFS-14849) Erasure Coding: the internal block is replicated many times when datanode is decommissioning

2019-10-03 Thread Wei-Chiu Chuang (Jira)
[ https://issues.apache.org/jira/browse/HDFS-14849?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wei-Chiu Chuang updated HDFS-14849: --- Attachment: HDFS-14849.branch-3.1.patch > Erasure Coding: the internal block is replicated

[jira] [Updated] (HDFS-14849) Erasure Coding: the internal block is replicated many times when datanode is decommissioning

2019-10-03 Thread Wei-Chiu Chuang (Jira)
[ https://issues.apache.org/jira/browse/HDFS-14849?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wei-Chiu Chuang updated HDFS-14849: --- Fix Version/s: 3.2.2 > Erasure Coding: the internal block is replicated many times when

[jira] [Commented] (HDFS-14754) Erasure Coding : The number of Under-Replicated Blocks never reduced

2019-10-03 Thread Wei-Chiu Chuang (Jira)
[ https://issues.apache.org/jira/browse/HDFS-14754?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16943826#comment-16943826 ] Wei-Chiu Chuang commented on HDFS-14754: [~surendrasingh] [~hemanthboyina] the test doesn't seem

[jira] [Commented] (HDFS-14637) Namenode may not replicate blocks to meet the policy after enabling upgradeDomain

2019-10-03 Thread Wei-Chiu Chuang (Jira)
[ https://issues.apache.org/jira/browse/HDFS-14637?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16943740#comment-16943740 ] Wei-Chiu Chuang commented on HDFS-14637: +1 > Namenode may not replicate blocks to meet the

[jira] [Updated] (HDFS-14216) NullPointerException happens in NamenodeWebHdfs

2019-10-02 Thread Wei-Chiu Chuang (Jira)
[ https://issues.apache.org/jira/browse/HDFS-14216?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wei-Chiu Chuang updated HDFS-14216: --- Resolution: Fixed Status: Resolved (was: Patch Available) > NullPointerException

[jira] [Updated] (HDFS-14216) NullPointerException happens in NamenodeWebHdfs

2019-10-02 Thread Wei-Chiu Chuang (Jira)
[ https://issues.apache.org/jira/browse/HDFS-14216?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wei-Chiu Chuang updated HDFS-14216: --- Fix Version/s: 3.1.4 > NullPointerException happens in NamenodeWebHdfs >

[jira] [Commented] (HDFS-14216) NullPointerException happens in NamenodeWebHdfs

2019-10-02 Thread Wei-Chiu Chuang (Jira)
[ https://issues.apache.org/jira/browse/HDFS-14216?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16943294#comment-16943294 ] Wei-Chiu Chuang commented on HDFS-14216: Failure doesn't look related. Pushing it to branch-3.1

[jira] [Updated] (HDFS-14678) Allow triggerBlockReport to a specific namenode

2019-10-02 Thread Wei-Chiu Chuang (Jira)
[ https://issues.apache.org/jira/browse/HDFS-14678?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wei-Chiu Chuang updated HDFS-14678: --- Fix Version/s: 3.2.2 3.1.4 > Allow triggerBlockReport to a specific

[jira] [Commented] (HDFS-14678) Allow triggerBlockReport to a specific namenode

2019-10-02 Thread Wei-Chiu Chuang (Jira)
[ https://issues.apache.org/jira/browse/HDFS-14678?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16943259#comment-16943259 ] Wei-Chiu Chuang commented on HDFS-14678: Cherry picking the commit into branch-3.2 and

[jira] [Updated] (HDFS-8881) Erasure Coding: internal blocks got missed and got over-replicated at the same time

2019-10-02 Thread Wei-Chiu Chuang (Jira)
[ https://issues.apache.org/jira/browse/HDFS-8881?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wei-Chiu Chuang updated HDFS-8881: -- Resolution: Duplicate Status: Resolved (was: Patch Available) > Erasure Coding:

[jira] [Updated] (HDFS-14523) Remove excess read lock for NetworkToplogy

2019-10-02 Thread Wei-Chiu Chuang (Jira)
[ https://issues.apache.org/jira/browse/HDFS-14523?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wei-Chiu Chuang updated HDFS-14523: --- Fix Version/s: 3.2.2 3.1.4 > Remove excess read lock for NetworkToplogy >

[jira] [Updated] (HDFS-14618) Incorrect synchronization of ArrayList field (ArrayList is thread-unsafe).

2019-10-02 Thread Wei-Chiu Chuang (Jira)
[ https://issues.apache.org/jira/browse/HDFS-14618?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wei-Chiu Chuang updated HDFS-14618: --- Fix Version/s: 3.2.2 3.1.4 > Incorrect synchronization of ArrayList field

[jira] [Updated] (HDFS-14216) NullPointerException happens in NamenodeWebHdfs

2019-10-02 Thread Wei-Chiu Chuang (Jira)
[ https://issues.apache.org/jira/browse/HDFS-14216?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wei-Chiu Chuang updated HDFS-14216: --- Attachment: HDFS-14216.branch-3.1.patch > NullPointerException happens in NamenodeWebHdfs >

[jira] [Updated] (HDFS-14216) NullPointerException happens in NamenodeWebHdfs

2019-10-02 Thread Wei-Chiu Chuang (Jira)
[ https://issues.apache.org/jira/browse/HDFS-14216?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wei-Chiu Chuang updated HDFS-14216: --- Status: Patch Available (was: Reopened) > NullPointerException happens in NamenodeWebHdfs >

[jira] [Reopened] (HDFS-14216) NullPointerException happens in NamenodeWebHdfs

2019-10-02 Thread Wei-Chiu Chuang (Jira)
[ https://issues.apache.org/jira/browse/HDFS-14216?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wei-Chiu Chuang reopened HDFS-14216: Reopen for branch-3.1. The only thing different is the LOG class change. Can't use

[jira] [Updated] (HDFS-14610) HashMap is not thread safe. Field storageMap is typically synchronized by storageMap. However, in one place, field storageMap is not protected with synchronized.

2019-10-02 Thread Wei-Chiu Chuang (Jira)
[ https://issues.apache.org/jira/browse/HDFS-14610?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wei-Chiu Chuang updated HDFS-14610: --- Fix Version/s: 3.2.2 3.1.4 > HashMap is not thread safe. Field storageMap

[jira] [Commented] (HDFS-14527) Stop all DataNodes may result in NN terminate

2019-10-02 Thread Wei-Chiu Chuang (Jira)
[ https://issues.apache.org/jira/browse/HDFS-14527?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16943165#comment-16943165 ] Wei-Chiu Chuang commented on HDFS-14527: Patch applies cleanly in branch-3.2 also. But it doesn't

[jira] [Updated] (HDFS-14527) Stop all DataNodes may result in NN terminate

2019-10-02 Thread Wei-Chiu Chuang (Jira)
[ https://issues.apache.org/jira/browse/HDFS-14527?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wei-Chiu Chuang updated HDFS-14527: --- Fix Version/s: 3.2.2 > Stop all DataNodes may result in NN terminate >

[jira] [Updated] (HDFS-14808) EC: Improper size values for corrupt ec block in LOG

2019-10-02 Thread Wei-Chiu Chuang (Jira)
[ https://issues.apache.org/jira/browse/HDFS-14808?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wei-Chiu Chuang updated HDFS-14808: --- Fix Version/s: 3.2.2 3.1.4 > EC: Improper size values for corrupt ec

[jira] [Updated] (HDFS-14699) Erasure Coding: Storage not considered in live replica when replication streams hard limit reached to threshold

2019-10-02 Thread Wei-Chiu Chuang (Jira)
[ https://issues.apache.org/jira/browse/HDFS-14699?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wei-Chiu Chuang updated HDFS-14699: --- Fix Version/s: 3.1.4 > Erasure Coding: Storage not considered in live replica when

[jira] [Updated] (HDFS-14754) Erasure Coding : The number of Under-Replicated Blocks never reduced

2019-10-02 Thread Wei-Chiu Chuang (Jira)
[ https://issues.apache.org/jira/browse/HDFS-14754?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wei-Chiu Chuang updated HDFS-14754: --- Status: Patch Available (was: Reopened) Reopen & submit the branch-3.1 patch. Branch-3.2

[jira] [Reopened] (HDFS-14754) Erasure Coding : The number of Under-Replicated Blocks never reduced

2019-10-02 Thread Wei-Chiu Chuang (Jira)
[ https://issues.apache.org/jira/browse/HDFS-14754?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wei-Chiu Chuang reopened HDFS-14754: > Erasure Coding : The number of Under-Replicated Blocks never reduced >

[jira] [Updated] (HDFS-14754) Erasure Coding : The number of Under-Replicated Blocks never reduced

2019-10-02 Thread Wei-Chiu Chuang (Jira)
[ https://issues.apache.org/jira/browse/HDFS-14754?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wei-Chiu Chuang updated HDFS-14754: --- Fix Version/s: 3.2.2 > Erasure Coding : The number of Under-Replicated Blocks never reduced

[jira] [Commented] (HDFS-14313) Get hdfs used space from FsDatasetImpl#volumeMap#ReplicaInfo in memory instead of df/du

2019-10-01 Thread Wei-Chiu Chuang (Jira)
[ https://issues.apache.org/jira/browse/HDFS-14313?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16942401#comment-16942401 ] Wei-Chiu Chuang commented on HDFS-14313: Conflicts are trivial so I pushed them into branch-3.2

[jira] [Updated] (HDFS-14313) Get hdfs used space from FsDatasetImpl#volumeMap#ReplicaInfo in memory instead of df/du

2019-10-01 Thread Wei-Chiu Chuang (Jira)
[ https://issues.apache.org/jira/browse/HDFS-14313?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wei-Chiu Chuang updated HDFS-14313: --- Fix Version/s: 3.2.2 3.1.4 > Get hdfs used space from

[jira] [Updated] (HDFS-14313) Get hdfs used space from FsDatasetImpl#volumeMap#ReplicaInfo in memory instead of df/du

2019-10-01 Thread Wei-Chiu Chuang (Jira)
[ https://issues.apache.org/jira/browse/HDFS-14313?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wei-Chiu Chuang updated HDFS-14313: --- Attachment: HDFS-14313.branch-3.2.patch HDFS-14313.branch-3.1.patch > Get

[jira] [Commented] (HDFS-14313) Get hdfs used space from FsDatasetImpl#volumeMap#ReplicaInfo in memory instead of df/du

2019-10-01 Thread Wei-Chiu Chuang (Jira)
[ https://issues.apache.org/jira/browse/HDFS-14313?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16942389#comment-16942389 ] Wei-Chiu Chuang commented on HDFS-14313: This is missing from branch-3.2 and branch-3.1

[jira] [Updated] (HDFS-14192) Track missing DFS operations in Statistics and StorageStatistics

2019-10-01 Thread Wei-Chiu Chuang (Jira)
[ https://issues.apache.org/jira/browse/HDFS-14192?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wei-Chiu Chuang updated HDFS-14192: --- Resolution: Fixed Status: Resolved (was: Patch Available) Resolve. The failed tests

[jira] [Updated] (HDFS-14192) Track missing DFS operations in Statistics and StorageStatistics

2019-10-01 Thread Wei-Chiu Chuang (Jira)
[ https://issues.apache.org/jira/browse/HDFS-14192?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wei-Chiu Chuang updated HDFS-14192: --- Fix Version/s: 3.2.2 3.1.4 > Track missing DFS operations in Statistics

[jira] [Updated] (HDFS-14349) Edit log may be rolled more frequently than necessary with multiple Standby nodes

2019-10-01 Thread Wei-Chiu Chuang (Jira)
[ https://issues.apache.org/jira/browse/HDFS-14349?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wei-Chiu Chuang updated HDFS-14349: --- Labels: multi-sbnn (was: ) > Edit log may be rolled more frequently than necessary with

[jira] [Updated] (HDFS-14192) Track missing DFS operations in Statistics and StorageStatistics

2019-10-01 Thread Wei-Chiu Chuang (Jira)
[ https://issues.apache.org/jira/browse/HDFS-14192?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wei-Chiu Chuang updated HDFS-14192: --- Attachment: HDFS-14192.branch-3.1.patch > Track missing DFS operations in Statistics and

[jira] [Updated] (HDFS-14192) Track missing DFS operations in Statistics and StorageStatistics

2019-10-01 Thread Wei-Chiu Chuang (Jira)
[ https://issues.apache.org/jira/browse/HDFS-14192?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wei-Chiu Chuang updated HDFS-14192: --- Status: Patch Available (was: Reopened) > Track missing DFS operations in Statistics and

[jira] [Reopened] (HDFS-14192) Track missing DFS operations in Statistics and StorageStatistics

2019-10-01 Thread Wei-Chiu Chuang (Jira)
[ https://issues.apache.org/jira/browse/HDFS-14192?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wei-Chiu Chuang reopened HDFS-14192: Reopen for the branch-3.1 backport. > Track missing DFS operations in Statistics and

[jira] [Commented] (HDFS-14192) Track missing DFS operations in Statistics and StorageStatistics

2019-10-01 Thread Wei-Chiu Chuang (Jira)
[ https://issues.apache.org/jira/browse/HDFS-14192?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16942226#comment-16942226 ] Wei-Chiu Chuang commented on HDFS-14192: Commit applies cleanly in branch-3.2, but because

[jira] [Updated] (HDFS-14460) DFSUtil#getNamenodeWebAddr should return HTTPS address based on policy configured

2019-10-01 Thread Wei-Chiu Chuang (Jira)
[ https://issues.apache.org/jira/browse/HDFS-14460?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wei-Chiu Chuang updated HDFS-14460: --- Fix Version/s: 3.2.2 3.1.4 > DFSUtil#getNamenodeWebAddr should return

[jira] [Updated] (HDFS-14175) EC: Native XOR decoder should reset the output buffer before using it.

2019-10-01 Thread Wei-Chiu Chuang (Jira)
[ https://issues.apache.org/jira/browse/HDFS-14175?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wei-Chiu Chuang updated HDFS-14175: --- Fix Version/s: 3.2.2 3.1.4 > EC: Native XOR decoder should reset the

[jira] [Updated] (HDFS-14202) "dfs.disk.balancer.max.disk.throughputInMBperSec" property is not working as per set value.

2019-10-01 Thread Wei-Chiu Chuang (Jira)
[ https://issues.apache.org/jira/browse/HDFS-14202?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wei-Chiu Chuang updated HDFS-14202: --- Fix Version/s: 3.2.2 3.1.4 >

[jira] [Updated] (HDFS-14418) Remove redundant super user priveledge checks from namenode.

2019-10-01 Thread Wei-Chiu Chuang (Jira)
[ https://issues.apache.org/jira/browse/HDFS-14418?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wei-Chiu Chuang updated HDFS-14418: --- Fix Version/s: 3.2.2 3.1.4 > Remove redundant super user priveledge

[jira] [Resolved] (HDFS-14418) Remove redundant super user priveledge checks from namenode.

2019-10-01 Thread Wei-Chiu Chuang (Jira)
[ https://issues.apache.org/jira/browse/HDFS-14418?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wei-Chiu Chuang resolved HDFS-14418. Resolution: Fixed > Remove redundant super user priveledge checks from namenode. >

[jira] [Updated] (HDFS-14418) Remove redundant super user priveledge checks from namenode.

2019-10-01 Thread Wei-Chiu Chuang (Jira)
[ https://issues.apache.org/jira/browse/HDFS-14418?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wei-Chiu Chuang updated HDFS-14418: --- Attachment: HDFS-14418.branch-3.1.001.patch > Remove redundant super user priveledge checks

[jira] [Commented] (HDFS-14418) Remove redundant super user priveledge checks from namenode.

2019-10-01 Thread Wei-Chiu Chuang (Jira)
[ https://issues.apache.org/jira/browse/HDFS-14418?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16942118#comment-16942118 ] Wei-Chiu Chuang commented on HDFS-14418: Patch applies cleanly in branch-3.2. There's a trivial

[jira] [Reopened] (HDFS-14418) Remove redundant super user priveledge checks from namenode.

2019-10-01 Thread Wei-Chiu Chuang (Jira)
[ https://issues.apache.org/jira/browse/HDFS-14418?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wei-Chiu Chuang reopened HDFS-14418: Reopen to add this in branch-3.1 > Remove redundant super user priveledge checks from

[jira] [Commented] (HDFS-14235) Handle ArrayIndexOutOfBoundsException in DataNodeDiskMetrics#slowDiskDetectionDaemon

2019-09-30 Thread Wei-Chiu Chuang (Jira)
[ https://issues.apache.org/jira/browse/HDFS-14235?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16941411#comment-16941411 ] Wei-Chiu Chuang commented on HDFS-14235: Commit applies cleanly in branch-3.1. Updated fix

[jira] [Updated] (HDFS-14235) Handle ArrayIndexOutOfBoundsException in DataNodeDiskMetrics#slowDiskDetectionDaemon

2019-09-30 Thread Wei-Chiu Chuang (Jira)
[ https://issues.apache.org/jira/browse/HDFS-14235?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wei-Chiu Chuang updated HDFS-14235: --- Fix Version/s: 3.1.4 > Handle ArrayIndexOutOfBoundsException in >

[jira] [Commented] (HDFS-10648) Expose Balancer metrics through Metrics2

2019-09-30 Thread Wei-Chiu Chuang (Jira)
[ https://issues.apache.org/jira/browse/HDFS-10648?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16941389#comment-16941389 ] Wei-Chiu Chuang commented on HDFS-10648: Thanks for doing this. Without metrics HDFS-13783 isn't

[jira] [Updated] (HDFS-14808) EC: Improper size values for corrupt ec block in LOG

2019-09-30 Thread Wei-Chiu Chuang (Jira)
[ https://issues.apache.org/jira/browse/HDFS-14808?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wei-Chiu Chuang updated HDFS-14808: --- Component/s: ec > EC: Improper size values for corrupt ec block in LOG >

[jira] [Reopened] (HDFS-7134) Replication count for a block should not update till the blocks have settled on Datanodes

2019-09-30 Thread Wei-Chiu Chuang (Jira)
[ https://issues.apache.org/jira/browse/HDFS-7134?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wei-Chiu Chuang reopened HDFS-7134: --- > Replication count for a block should not update till the blocks have settled > on Datanodes >

[jira] [Resolved] (HDFS-7134) Replication count for a block should not update till the blocks have settled on Datanodes

2019-09-30 Thread Wei-Chiu Chuang (Jira)
[ https://issues.apache.org/jira/browse/HDFS-7134?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wei-Chiu Chuang resolved HDFS-7134. --- Resolution: Cannot Reproduce Resolve as cannot reproduce. > Replication count for a block

[jira] [Commented] (HDFS-14754) Erasure Coding : The number of Under-Replicated Blocks never reduced

2019-09-30 Thread Wei-Chiu Chuang (Jira)
[ https://issues.apache.org/jira/browse/HDFS-14754?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16941376#comment-16941376 ] Wei-Chiu Chuang commented on HDFS-14754: Too bad this one didn't land in 3.2.1 and 3.1.3. Let's

[jira] [Comment Edited] (HDFS-14754) Erasure Coding : The number of Under-Replicated Blocks never reduced

2019-09-30 Thread Wei-Chiu Chuang (Jira)
[ https://issues.apache.org/jira/browse/HDFS-14754?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16941376#comment-16941376 ] Wei-Chiu Chuang edited comment on HDFS-14754 at 9/30/19 10:20 PM: -- Too

[jira] [Updated] (HDFS-14754) Erasure Coding : The number of Under-Replicated Blocks never reduced

2019-09-30 Thread Wei-Chiu Chuang (Jira)
[ https://issues.apache.org/jira/browse/HDFS-14754?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wei-Chiu Chuang updated HDFS-14754: --- Component/s: ec > Erasure Coding : The number of Under-Replicated Blocks never reduced >

[jira] [Updated] (HDFS-14528) Failover from Active to Standby Failed

2019-09-30 Thread Wei-Chiu Chuang (Jira)
[ https://issues.apache.org/jira/browse/HDFS-14528?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wei-Chiu Chuang updated HDFS-14528: --- Labels: multi-sbnn (was: ) > Failover from Active to Standby Failed >

[jira] [Updated] (HDFS-14855) client always print standbyexception info with multi standby namenode

2019-09-30 Thread Wei-Chiu Chuang (Jira)
[ https://issues.apache.org/jira/browse/HDFS-14855?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wei-Chiu Chuang updated HDFS-14855: --- Labels: multi-sbnn (was: ) > client always print standbyexception info with multi standby

[jira] [Updated] (HDFS-14201) Ability to disallow safemode NN to become active

2019-09-30 Thread Wei-Chiu Chuang (Jira)
[ https://issues.apache.org/jira/browse/HDFS-14201?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wei-Chiu Chuang updated HDFS-14201: --- Labels: multi-sbnn (was: ) > Ability to disallow safemode NN to become active >

[jira] [Commented] (HDFS-14305) Serial number in BlockTokenSecretManager could overlap between different namenodes

2019-09-30 Thread Wei-Chiu Chuang (Jira)
[ https://issues.apache.org/jira/browse/HDFS-14305?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16941238#comment-16941238 ] Wei-Chiu Chuang commented on HDFS-14305: HDFS-6440 was backported into branch-2 by HDFS-14205.

  1   2   3   4   5   6   7   8   9   10   >