[jira] [Commented] (HDFS-11353) Improve the unit tests relevant to DataNode volume failure testing

[ https://issues.apache.org/jira/browse/HDFS-11353?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15849588#comment-15849588 ] Xiao Chen commented on HDFS-11353: -- Thanks [~linyiqun], tracking that in another jira SGTM. +1 on patch

[jira] [Created] (HDFS-11385) DistCp CopyCommitter should issue warning instead of throw exception when failed to delete file.

Yongjun Zhang created HDFS-11385: Summary: DistCp CopyCommitter should issue warning instead of throw exception when failed to delete file. Key: HDFS-11385 URL: https://issues.apache.org/jira/browse/HDFS-11385

[jira] [Commented] (HDFS-11375) Display the volume storage type in datanode UI

[ https://issues.apache.org/jira/browse/HDFS-11375?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15849415#comment-15849415 ] Brahma Reddy Battula commented on HDFS-11375: - [~surendrasingh] thanks for reporting.. Patch

[jira] [Commented] (HDFS-7343) HDFS smart storage management

[ https://issues.apache.org/jira/browse/HDFS-7343?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15849385#comment-15849385 ] Andrew Wang commented on HDFS-7343: --- Hi Wei, thanks for posting the new doc. At a high-level I think the

[jira] [Commented] (HDFS-11353) Improve the unit tests relevant to DataNode volume failure testing

[ https://issues.apache.org/jira/browse/HDFS-11353?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15849357#comment-15849357 ] Hadoop QA commented on HDFS-11353: -- | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote || Subsystem ||

[jira] [Commented] (HDFS-10860) Switch HttpFS from Tomcat to Jetty

[ https://issues.apache.org/jira/browse/HDFS-10860?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15849319#comment-15849319 ] Hadoop QA commented on HDFS-10860: -- | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote || Subsystem ||

[jira] [Commented] (HDFS-4025) QJM: Sychronize past log segments to JNs that missed them

[ https://issues.apache.org/jira/browse/HDFS-4025?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15849312#comment-15849312 ] Hadoop QA commented on HDFS-4025: - | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote || Subsystem ||

[jira] [Commented] (HDFS-10899) Add functionality to re-encrypt EDEKs.

[ https://issues.apache.org/jira/browse/HDFS-10899?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15849308#comment-15849308 ] Hadoop QA commented on HDFS-10899: -- | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote || Subsystem ||

[jira] [Commented] (HDFS-11112) Journal Nodes should refuse to format non-empty directories

[ https://issues.apache.org/jira/browse/HDFS-2?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15849276#comment-15849276 ] Hudson commented on HDFS-2: --- SUCCESS: Integrated in Jenkins build Hadoop-trunk-Commit #11197 (See

[jira] [Commented] (HDFS-11377) Balancer hung due to no available mover threads

[ https://issues.apache.org/jira/browse/HDFS-11377?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15849272#comment-15849272 ] Yiqun Lin commented on HDFS-11377: -- LGTM, +1. Thanks [~zhaoyunjiong] for updating the patch. Will commit

[jira] [Updated] (HDFS-11353) Improve the unit tests relevant to DataNode volume failure testing

[ https://issues.apache.org/jira/browse/HDFS-11353?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yiqun Lin updated HDFS-11353: - Attachment: HDFS-11353.006.patch > Improve the unit tests relevant to DataNode volume failure testing >

[jira] [Commented] (HDFS-11353) Improve the unit tests relevant to DataNode volume failure testing

[ https://issues.apache.org/jira/browse/HDFS-11353?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15849245#comment-15849245 ] Yiqun Lin commented on HDFS-11353: -- Thanks [~xiaochen] for the review! {quote} and ideally we should see

[jira] [Commented] (HDFS-11363) Need more diagnosis info when seeing Slow waitForAckedSeqno

[ https://issues.apache.org/jira/browse/HDFS-11363?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15849214#comment-15849214 ] Xiao Chen commented on HDFS-11363: -- Oops, thanks [~yzhangal], good catch! I'd like to fix it at commit

[jira] [Commented] (HDFS-11363) Need more diagnosis info when seeing Slow waitForAckedSeqno

[ https://issues.apache.org/jira/browse/HDFS-11363?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15849211#comment-15849211 ] Hadoop QA commented on HDFS-11363: -- | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote || Subsystem ||

[jira] [Commented] (HDFS-11370) Optimize NamenodeFsck#getReplicaInfo

[ https://issues.apache.org/jira/browse/HDFS-11370?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15849203#comment-15849203 ] Takanobu Asanuma commented on HDFS-11370: - Thank you for reviewing and committing the patch,

[jira] [Updated] (HDFS-11112) Journal Nodes should refuse to format non-empty directories

[ https://issues.apache.org/jira/browse/HDFS-2?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Arpit Agarwal updated HDFS-2: - Resolution: Fixed Hadoop Flags: Reviewed Fix Version/s: 2.9.0 Status:

[jira] [Commented] (HDFS-11363) Need more diagnosis info when seeing Slow waitForAckedSeqno

[ https://issues.apache.org/jira/browse/HDFS-11363?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15849184#comment-15849184 ] Yongjun Zhang commented on HDFS-11363: -- Thanks [~xiaochen] for the new rev, except for an extra ":",

[jira] [Commented] (HDFS-11112) Journal Nodes should refuse to format non-empty directories

[ https://issues.apache.org/jira/browse/HDFS-2?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15849181#comment-15849181 ] Hadoop QA commented on HDFS-2: -- | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote || Subsystem ||

[jira] [Updated] (HDFS-11363) Need more diagnosis info when seeing Slow waitForAckedSeqno

[ https://issues.apache.org/jira/browse/HDFS-11363?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Chen updated HDFS-11363: - Attachment: HDFS-11363.02.patch Thanks [~yzhangal] for the review. The placeholder in slf4j supports

[jira] [Updated] (HDFS-10899) Add functionality to re-encrypt EDEKs.

[ https://issues.apache.org/jira/browse/HDFS-10899?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Chen updated HDFS-10899: - Attachment: HDFS-10899.07.patch editsStored Patch 7 rebases to latest trunk, and added a

[jira] [Updated] (HDFS-10860) Switch HttpFS from Tomcat to Jetty

[ https://issues.apache.org/jira/browse/HDFS-10860?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] John Zhuge updated HDFS-10860: -- Attachment: HDFS-10860.009.patch Patch 009 - Update ServerSetup.md.vm TESTING DONE - Verify

[jira] [Updated] (HDFS-4025) QJM: Sychronize past log segments to JNs that missed them

[ https://issues.apache.org/jira/browse/HDFS-4025?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hanisha Koneru updated HDFS-4025: - Attachment: HDFS-4025.010.patch Thank you [~jingzhao] for reviewing the patch. I have addressed

[jira] [Commented] (HDFS-10983) OIV tool should make an EC file explicit

[ https://issues.apache.org/jira/browse/HDFS-10983?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15849101#comment-15849101 ] Andrew Wang commented on HDFS-10983: Thanks for the writeup Manoj. I think we shouldn't change

[jira] [Commented] (HDFS-6984) In Hadoop 3, make FileStatus serialize itself via protobuf

[ https://issues.apache.org/jira/browse/HDFS-6984?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15849075#comment-15849075 ] Andrew Wang commented on HDFS-6984: --- I think this is a "user-level file format" and covered by:

[jira] [Commented] (HDFS-4025) QJM: Sychronize past log segments to JNs that missed them

[ https://issues.apache.org/jira/browse/HDFS-4025?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15849052#comment-15849052 ] Jing Zhao commented on HDFS-4025: - Thanks for the updating the patch, [~hanishakoneru]. The latest patch

[jira] [Commented] (HDFS-11363) Need more diagnosis info when seeing Slow waitForAckedSeqno

[ https://issues.apache.org/jira/browse/HDFS-11363?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15849039#comment-15849039 ] Yongjun Zhang commented on HDFS-11363: -- Thanks [~xiaochen] for the patch. The patch is largely what

[jira] [Comment Edited] (HDFS-11112) Journal Nodes should refuse to format non-empty directories

[ https://issues.apache.org/jira/browse/HDFS-2?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15848991#comment-15848991 ] Arpit Agarwal edited comment on HDFS-2 at 2/1/17 9:59 PM: -- Sorry I missed

[jira] [Commented] (HDFS-11112) Journal Nodes should refuse to format non-empty directories

[ https://issues.apache.org/jira/browse/HDFS-2?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15848991#comment-15848991 ] Arpit Agarwal commented on HDFS-2: -- Sorry I missed this one [~linyiqun]. +1 pending Jenkins. >

[jira] [Updated] (HDFS-11112) Journal Nodes should refuse to format non-empty directories

[ https://issues.apache.org/jira/browse/HDFS-2?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Arpit Agarwal updated HDFS-2: - Status: Patch Available (was: Open) > Journal Nodes should refuse to format non-empty

[jira] [Updated] (HDFS-11112) Journal Nodes should refuse to format non-empty directories

[ https://issues.apache.org/jira/browse/HDFS-2?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Arpit Agarwal updated HDFS-2: - Status: Open (was: Patch Available) > Journal Nodes should refuse to format non-empty

[jira] [Commented] (HDFS-11377) Balancer hung due to no available mover threads

[ https://issues.apache.org/jira/browse/HDFS-11377?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15848954#comment-15848954 ] Hadoop QA commented on HDFS-11377: -- | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote || Subsystem ||

[jira] [Commented] (HDFS-11353) Improve the unit tests relevant to DataNode volume failure testing

[ https://issues.apache.org/jira/browse/HDFS-11353?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15848946#comment-15848946 ] Xiao Chen commented on HDFS-11353: -- Thanks [~linyiqun], +1 pending: 1. The check in

[jira] [Updated] (HDFS-11384) Add option for balancer to disperse getBlocks calls to avoid NameNode's rpc.CallQueueLength spike

[ https://issues.apache.org/jira/browse/HDFS-11384?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] yunjiong zhao updated HDFS-11384: - Attachment: balancer.day.png balancer.week.png

[jira] [Updated] (HDFS-11384) Add option for balancer to disperse getBlocks calls to avoid NameNode's rpc.CallQueueLength spike

[ https://issues.apache.org/jira/browse/HDFS-11384?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] yunjiong zhao updated HDFS-11384: - Status: Patch Available (was: Open) > Add option for balancer to disperse getBlocks calls to

[jira] [Created] (HDFS-11384) Add option for balancer to disperse getBlocks calls to avoid NameNode's rpc.CallQueueLength spike

yunjiong zhao created HDFS-11384: Summary: Add option for balancer to disperse getBlocks calls to avoid NameNode's rpc.CallQueueLength spike Key: HDFS-11384 URL: https://issues.apache.org/jira/browse/HDFS-11384

[jira] [Commented] (HDFS-11370) Optimize NamenodeFsck#getReplicaInfo

[ https://issues.apache.org/jira/browse/HDFS-11370?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15848862#comment-15848862 ] Hudson commented on HDFS-11370: --- SUCCESS: Integrated in Jenkins build Hadoop-trunk-Commit #11196 (See

[jira] [Commented] (HDFS-11370) Optimize NamenodeFsck#getReplicaInfo

[ https://issues.apache.org/jira/browse/HDFS-11370?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15848812#comment-15848812 ] Jing Zhao commented on HDFS-11370: -- The latest patch looks good to me. And the failed test should be

[jira] [Updated] (HDFS-11370) Optimize NamenodeFsck#getReplicaInfo

[ https://issues.apache.org/jira/browse/HDFS-11370?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jing Zhao updated HDFS-11370: - Resolution: Fixed Hadoop Flags: Reviewed Fix Version/s: 3.0.0-alpha3 Status:

[jira] [Comment Edited] (HDFS-11377) Balancer hung due to no available mover threads

[ https://issues.apache.org/jira/browse/HDFS-11377?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15848792#comment-15848792 ] yunjiong zhao edited comment on HDFS-11377 at 2/1/17 7:03 PM: -- Removed unused

[jira] [Updated] (HDFS-11377) Balancer hung due to no available mover threads

[ https://issues.apache.org/jira/browse/HDFS-11377?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] yunjiong zhao updated HDFS-11377: - Attachment: HDFS-11377.002.patch Remove unused variable MAX_NO_PENDING_MOVE_ITERATIONS. Thanks

[jira] [Commented] (HDFS-10860) Switch HttpFS from Tomcat to Jetty

[ https://issues.apache.org/jira/browse/HDFS-10860?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15848757#comment-15848757 ] Xiao Chen commented on HDFS-10860: -- Thanks [~jzhuge], patch 8 looks good to me. Will let this soak for a

[jira] [Updated] (HDFS-11210) Enhance key rolling to guarantee new KeyVersion is returned from generateEncryptedKeys after a key is rolled

[ https://issues.apache.org/jira/browse/HDFS-11210?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Chen updated HDFS-11210: - Attachment: HDFS-11210.03.patch Patch 3 to fix the 2 related test failure: invalidate cache after

[jira] [Assigned] (HDFS-11383) String duplication in org.apache.hadoop.fs.BlockLocation

[ https://issues.apache.org/jira/browse/HDFS-11383?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Manoj Govindassamy reassigned HDFS-11383: - Assignee: Manoj Govindassamy > String duplication in

[jira] [Commented] (HDFS-6984) In Hadoop 3, make FileStatus serialize itself via protobuf

[ https://issues.apache.org/jira/browse/HDFS-6984?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15848285#comment-15848285 ] Steve Loughran commented on HDFS-6984: -- What is there in the hadoop compat guidelines about

[jira] [Commented] (HDFS-10860) Switch HttpFS from Tomcat to Jetty

[ https://issues.apache.org/jira/browse/HDFS-10860?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15848283#comment-15848283 ] Hadoop QA commented on HDFS-10860: -- | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote || Subsystem ||

[jira] [Commented] (HDFS-11026) Convert BlockTokenIdentifier to use Protobuf

[ https://issues.apache.org/jira/browse/HDFS-11026?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15848279#comment-15848279 ] Hadoop QA commented on HDFS-11026: -- | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote || Subsystem ||

[jira] [Commented] (HDFS-11210) Enhance key rolling to guarantee new KeyVersion is returned from generateEncryptedKeys after a key is rolled

[ https://issues.apache.org/jira/browse/HDFS-11210?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15848245#comment-15848245 ] Hadoop QA commented on HDFS-11210: -- | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote || Subsystem ||

[jira] [Updated] (HDFS-6708) StorageType should be encoded in the block token

[ https://issues.apache.org/jira/browse/HDFS-6708?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ewan Higgs updated HDFS-6708: - Attachment: HDFS-6708.0001.patch Attached is a patch that will check the StorageType of the writes as

[jira] [Updated] (HDFS-11026) Convert BlockTokenIdentifier to use Protobuf

[ https://issues.apache.org/jira/browse/HDFS-11026?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ewan Higgs updated HDFS-11026: -- Attachment: HDFS-11026.005.patch Jenkins revealed an issue in the new tests I added: when attempting to

[jira] [Updated] (HDFS-10860) Switch HttpFS from Tomcat to Jetty

[ https://issues.apache.org/jira/browse/HDFS-10860?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] John Zhuge updated HDFS-10860: -- Attachment: HDFS-10860.008.patch Patch 008 - Add config property hadoop.httpfs.http.administrators to

[jira] [Updated] (HDFS-11210) Enhance key rolling to guarantee new KeyVersion is returned from generateEncryptedKeys after a key version is rolled

[ https://issues.apache.org/jira/browse/HDFS-11210?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Chen updated HDFS-11210: - Summary: Enhance key rolling to guarantee new KeyVersion is returned from generateEncryptedKeys after a

[jira] [Updated] (HDFS-11210) Enhance key rolling to guarantee new KeyVersion is returned from generateEncryptedKeys after a key is rolled

[ https://issues.apache.org/jira/browse/HDFS-11210?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Chen updated HDFS-11210: - Summary: Enhance key rolling to guarantee new KeyVersion is returned from generateEncryptedKeys after a

[jira] [Updated] (HDFS-11210) Enhance key rolling to guarantee new KeyVersion is returned from generateEncryptedKeys after a key is rolled

[ https://issues.apache.org/jira/browse/HDFS-11210?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Chen updated HDFS-11210: - Status: Patch Available (was: Open) > Enhance key rolling to guarantee new KeyVersion is returned from