[jira] [Updated] (HDFS-12749) DN may not send block report to NN after NN restart

2018-02-22 Thread He Xiaoqiao (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12749?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] He Xiaoqiao updated HDFS-12749: --- Component/s: datanode > DN may not send block report to NN after NN restart >

[jira] [Updated] (HDFS-12749) DN may not send block report to NN after NN restart

2018-02-22 Thread He Xiaoqiao (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12749?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] He Xiaoqiao updated HDFS-12749: --- Attachment: HDFS-12749-branch-2.7.002.patch > DN may not send block report to NN after NN restart >

[jira] [Commented] (HDFS-12749) DN may not send block report to NN after NN restart

2018-02-22 Thread He Xiaoqiao (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12749?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16372595#comment-16372595 ] He Xiaoqiao commented on HDFS-12749: ping [~kihwal], [~xkrogen] I upload new patch rebase branch-2.7,

[jira] [Commented] (HDFS-13124) hadoop-daemon.sh exits with 1 when running HDFS balancer on balanced cluster

2018-02-22 Thread Jianfei Jiang (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-13124?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16372593#comment-16372593 ] Jianfei Jiang commented on HDFS-13124: -- Also meet this problem in 3.0. In the code of trunk, the

[jira] [Updated] (HDFS-12749) DN may not send block report to NN after NN restart

2018-02-22 Thread He Xiaoqiao (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12749?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] He Xiaoqiao updated HDFS-12749: --- Affects Version/s: 2.9.1 2.8.3 2.7.5

[jira] [Commented] (HDFS-13162) Create Replica Trash directory on DN startup

2018-02-22 Thread Nanda kumar (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-13162?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16372802#comment-16372802 ] Nanda kumar commented on HDFS-13162: Thanks Bharat for updating the patch. Patch looks good to me.

[jira] [Commented] (HDFS-12749) DN may not send block report to NN after NN restart

2018-02-22 Thread He Xiaoqiao (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12749?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16372749#comment-16372749 ] He Xiaoqiao commented on HDFS-12749: check failed UT(#TestFileCreationEmpty) and test locally, it

[jira] [Commented] (HDFS-12749) DN may not send block report to NN after NN restart

2018-02-22 Thread genericqa (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12749?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16372717#comment-16372717 ] genericqa commented on HDFS-12749: -- | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote || Subsystem ||

[jira] [Commented] (HDFS-11699) Ozone:SCM: Add support for close containers in SCM

2018-02-22 Thread Nanda kumar (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11699?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16372963#comment-16372963 ] Nanda kumar commented on HDFS-11699: Anu, the patch is not applying anymore. Can you rebase the patch

[jira] [Commented] (HDFS-13162) Create Replica Trash directory on DN startup

2018-02-22 Thread Nanda kumar (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-13162?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16373221#comment-16373221 ] Nanda kumar commented on HDFS-13162: Thanks Bharat for updating the patch. LGTM, +1 on patch v03,

[jira] [Commented] (HDFS-13162) Create Replica Trash directory on DN startup

2018-02-22 Thread Bharat Viswanadham (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-13162?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16373205#comment-16373205 ] Bharat Viswanadham commented on HDFS-13162: --- Hi [~nandakumar131] Thanks for review. Addressed

[jira] [Updated] (HDFS-13162) Create Replica Trash directory on DN startup

2018-02-22 Thread Bharat Viswanadham (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-13162?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bharat Viswanadham updated HDFS-13162: -- Attachment: HDFS-13162-HDFS-12996.03.patch > Create Replica Trash directory on DN

[jira] [Updated] (HDFS-13055) Aggregate usage statistics from datanodes

2018-02-22 Thread Ajay Kumar (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-13055?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ajay Kumar updated HDFS-13055: -- Attachment: HDFS-13055.003.patch > Aggregate usage statistics from datanodes >

[jira] [Updated] (HDFS-11699) Ozone:SCM: Add support for close containers in SCM

2018-02-22 Thread Anu Engineer (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11699?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Anu Engineer updated HDFS-11699: Attachment: HDFS-11699-HDFS-7240.003.patch > Ozone:SCM: Add support for close containers in SCM >

[jira] [Commented] (HDFS-11699) Ozone:SCM: Add support for close containers in SCM

2018-02-22 Thread Anu Engineer (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11699?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16373074#comment-16373074 ] Anu Engineer commented on HDFS-11699: - Thanks, updated the patch.   > Ozone:SCM: Add support for

[jira] [Updated] (HDFS-13055) Aggregate usage statistics from datanodes

2018-02-22 Thread Ajay Kumar (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-13055?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ajay Kumar updated HDFS-13055: -- Attachment: (was: HDFS-13055.003.patch) > Aggregate usage statistics from datanodes >

[jira] [Commented] (HDFS-11699) Ozone:SCM: Add support for close containers in SCM

2018-02-22 Thread Nanda kumar (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11699?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16373213#comment-16373213 ] Nanda kumar commented on HDFS-11699: Thanks Anu for the update. bq. we need to check both isClosed and

[jira] [Commented] (HDFS-12792) RBF: Test Router-based federation using HDFSContract

2018-02-22 Thread JIRA
[ https://issues.apache.org/jira/browse/HDFS-12792?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16373252#comment-16373252 ] Íñigo Goiri commented on HDFS-12792: As expected, the last run didn't run the new unit tests added

[jira] [Updated] (HDFS-13136) Avoid taking FSN lock while doing group member lookup for FSD permission check

2018-02-22 Thread Xiaoyu Yao (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-13136?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiaoyu Yao updated HDFS-13136: -- Attachment: HDFS-13136-branch-3.0.001.patch > Avoid taking FSN lock while doing group member lookup for

[jira] [Updated] (HDFS-13178) Add a force option to DiskBalancer Execute command

2018-02-22 Thread Bharat Viswanadham (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-13178?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bharat Viswanadham updated HDFS-13178: -- Status: Patch Available (was: Open) > Add a force option to DiskBalancer Execute

[jira] [Updated] (HDFS-13178) Add a force option to DiskBalancer Execute command

2018-02-22 Thread Bharat Viswanadham (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-13178?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bharat Viswanadham updated HDFS-13178: -- Attachment: HDFS-13178.00.patch > Add a force option to DiskBalancer Execute command >

[jira] [Commented] (HDFS-13162) Create Replica Trash directory on DN startup

2018-02-22 Thread genericqa (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-13162?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16373480#comment-16373480 ] genericqa commented on HDFS-13162: -- | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote || Subsystem ||

[jira] [Updated] (HDFS-13178) Add a force option to DiskBalancer Execute command

2018-02-22 Thread Bharat Viswanadham (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-13178?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bharat Viswanadham updated HDFS-13178: -- Description:   Add a force option to DiskBalancer Execute command, which is used for

[jira] [Commented] (HDFS-11699) Ozone:SCM: Add support for close containers in SCM

2018-02-22 Thread genericqa (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11699?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16373406#comment-16373406 ] genericqa commented on HDFS-11699: -- | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote || Subsystem ||

[jira] [Commented] (HDFS-13178) Add a force option to DiskBalancer Execute command

2018-02-22 Thread Anu Engineer (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-13178?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16373417#comment-16373417 ] Anu Engineer commented on HDFS-13178: - I am +1 on this change (completely agree with renamining force

[jira] [Commented] (HDFS-13136) Avoid taking FSN lock while doing group member lookup for FSD permission check

2018-02-22 Thread Hudson (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-13136?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16373461#comment-16373461 ] Hudson commented on HDFS-13136: --- SUCCESS: Integrated in Jenkins build Hadoop-trunk-Commit #13700 (See

[jira] [Commented] (HDFS-13178) Add a force option to DiskBalancer Execute command

2018-02-22 Thread Bharat Viswanadham (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-13178?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16373315#comment-16373315 ] Bharat Viswanadham commented on HDFS-13178: --- In testcases, added force flag in one of the test

[jira] [Comment Edited] (HDFS-13178) Add a force option to DiskBalancer Execute command

2018-02-22 Thread Arpit Agarwal (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-13178?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16373392#comment-16373392 ] Arpit Agarwal edited comment on HDFS-13178 at 2/22/18 8:32 PM: --- Thanks for

[jira] [Commented] (HDFS-13178) Add a force option to DiskBalancer Execute command

2018-02-22 Thread Arpit Agarwal (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-13178?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16373392#comment-16373392 ] Arpit Agarwal commented on HDFS-13178: -- Thanks for the patch [~bharatviswa]. Minor comment: # This

[jira] [Commented] (HDFS-13136) Avoid taking FSN lock while doing group member lookup for FSD permission check

2018-02-22 Thread genericqa (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-13136?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16373531#comment-16373531 ] genericqa commented on HDFS-13136: -- | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote || Subsystem ||

[jira] [Commented] (HDFS-12792) RBF: Test Router-based federation using HDFSContract

2018-02-22 Thread JIRA
[ https://issues.apache.org/jira/browse/HDFS-12792?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16373530#comment-16373530 ] Íñigo Goiri commented on HDFS-12792: Thanks [~ajayydv] for checking. We added recently the safe mode

[jira] [Commented] (HDFS-12070) Failed block recovery leaves files open indefinitely and at risk for data loss

2018-02-22 Thread Kihwal Lee (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12070?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16373276#comment-16373276 ] Kihwal Lee commented on HDFS-12070: --- It sure will be better if we can close the file right away. The

[jira] [Commented] (HDFS-12792) RBF: Test Router-based federation using HDFSContract

2018-02-22 Thread Ajay Kumar (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12792?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16373512#comment-16373512 ] Ajay Kumar commented on HDFS-12792: --- [~elgoiri],[~ywskycn], I tried to run some of the new tests locally

[jira] [Commented] (HDFS-12792) RBF: Test Router-based federation using HDFSContract

2018-02-22 Thread Wei Yan (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12792?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16373256#comment-16373256 ] Wei Yan commented on HDFS-12792: :) We need to wait for the HDFS bug fix day, to fix some other failed

[jira] [Updated] (HDFS-13178) Add a force option to DiskBalancer Execute command

2018-02-22 Thread Bharat Viswanadham (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-13178?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bharat Viswanadham updated HDFS-13178: -- Component/s: diskbalancer > Add a force option to DiskBalancer Execute command >

[jira] [Created] (HDFS-13178) Add a force option to DiskBalancer Execute command

2018-02-22 Thread Bharat Viswanadham (JIRA)
Bharat Viswanadham created HDFS-13178: - Summary: Add a force option to DiskBalancer Execute command Key: HDFS-13178 URL: https://issues.apache.org/jira/browse/HDFS-13178 Project: Hadoop HDFS

[jira] [Commented] (HDFS-13136) Avoid taking FSN lock while doing group member lookup for FSD permission check

2018-02-22 Thread Xiaoyu Yao (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-13136?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16373308#comment-16373308 ] Xiaoyu Yao commented on HDFS-13136: --- Thanks [~szetszwo] for the review. I will commit the patch to trunk

[jira] [Commented] (HDFS-13055) Aggregate usage statistics from datanodes

2018-02-22 Thread Ajay Kumar (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-13055?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16373535#comment-16373535 ] Ajay Kumar commented on HDFS-13055: --- [~arpitagarwal], thanks for review. Updated patch v3 makes changes

[jira] [Commented] (HDFS-13178) Add a force option to DiskBalancer Execute command

2018-02-22 Thread genericqa (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-13178?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16373536#comment-16373536 ] genericqa commented on HDFS-13178: -- | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote || Subsystem ||

[jira] [Updated] (HDFS-13055) Aggregate usage statistics from datanodes

2018-02-22 Thread Ajay Kumar (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-13055?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ajay Kumar updated HDFS-13055: -- Attachment: HDFS-13055.003.patch > Aggregate usage statistics from datanodes >

[jira] [Commented] (HDFS-13055) Aggregate usage statistics from datanodes

2018-02-22 Thread genericqa (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-13055?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16373540#comment-16373540 ] genericqa commented on HDFS-13055: -- | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote || Subsystem ||

[jira] [Commented] (HDFS-11394) Add method for getting erasure coding policy through WebHDFS

2018-02-22 Thread genericqa (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11394?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16373655#comment-16373655 ] genericqa commented on HDFS-11394: -- | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote || Subsystem ||

[jira] [Commented] (HDFS-7240) Scaling HDFS

2018-02-22 Thread Konstantin Shvachko (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-7240?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16373673#comment-16373673 ] Konstantin Shvachko commented on HDFS-7240: --- Hey [~anu], read the security document. It's a good

[jira] [Commented] (HDFS-12749) DN may not send block report to NN after NN restart

2018-02-22 Thread Kihwal Lee (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12749?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16373667#comment-16373667 ] Kihwal Lee commented on HDFS-12749: --- bq. About catching IOException: if the registration fails with a

[jira] [Commented] (HDFS-13164) File not closed if append fail with DSQuotaExceededException

2018-02-22 Thread Yongjun Zhang (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-13164?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16373756#comment-16373756 ] Yongjun Zhang commented on HDFS-13164: -- Thanks [~xiaochen], nice work here! The patch looks good to

[jira] [Commented] (HDFS-12975) Changes to the NameNode to support reads from standby

2018-02-22 Thread Chao Sun (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12975?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16373538#comment-16373538 ] Chao Sun commented on HDFS-12975: - Thanks [~shv] for taking a look! I've fixed the changes for the 1) and

[jira] [Comment Edited] (HDFS-12792) RBF: Test Router-based federation using HDFSContract

2018-02-22 Thread JIRA
[ https://issues.apache.org/jira/browse/HDFS-12792?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16373530#comment-16373530 ] Íñigo Goiri edited comment on HDFS-12792 at 2/22/18 9:59 PM: - Thanks

[jira] [Commented] (HDFS-13089) Add test to validate dfs used and no of blocks when blocks are moved across volumes

2018-02-22 Thread Ajay Kumar (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-13089?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16373574#comment-16373574 ] Ajay Kumar commented on HDFS-13089: --- {{TestDistributedFileSystem#testTotalDfsUsed}} does check DFS used

[jira] [Updated] (HDFS-13179) TestLazyPersistReplicaRecovery#testDnRestartWithSavedReplicas fails intermittently

2018-02-22 Thread Gabor Bota (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-13179?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gabor Bota updated HDFS-13179: -- Description: The error caused by TimeoutException because the test is waiting to ensure that the file

[jira] [Updated] (HDFS-13179) TestLazyPersistReplicaRecovery#testDnRestartWithSavedReplicas fails intermittently

2018-02-22 Thread Gabor Bota (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-13179?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gabor Bota updated HDFS-13179: -- Component/s: fs > TestLazyPersistReplicaRecovery#testDnRestartWithSavedReplicas fails > intermittently

[jira] [Updated] (HDFS-13136) Avoid taking FSN lock while doing group member lookup for FSD permission check

2018-02-22 Thread Xiaoyu Yao (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-13136?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiaoyu Yao updated HDFS-13136: -- Attachment: HDFS-13136-branch-3.0.002.patch > Avoid taking FSN lock while doing group member lookup for

[jira] [Created] (HDFS-13180) Implement security for Hadoop Distributed Storage Layer

2018-02-22 Thread Anu Engineer (JIRA)
Anu Engineer created HDFS-13180: --- Summary: Implement security for Hadoop Distributed Storage Layer Key: HDFS-13180 URL: https://issues.apache.org/jira/browse/HDFS-13180 Project: Hadoop HDFS

[jira] [Commented] (HDFS-10419) Building HDFS on top of new storage layer (HDSL)

2018-02-22 Thread Jitendra Nath Pandey (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10419?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16373542#comment-16373542 ] Jitendra Nath Pandey commented on HDFS-10419: - {quote}I thought that a better approach would

[jira] [Updated] (HDFS-13179) TestLazyPersistReplicaRecovery#testDnRestartWithSavedReplicas fails intermittently

2018-02-22 Thread Gabor Bota (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-13179?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gabor Bota updated HDFS-13179: -- Description: The error caused by TimeoutException because the test is waiting to ensure that the file

[jira] [Created] (HDFS-13179) TestLazyPersistReplicaRecovery#testDnRestartWithSavedReplicas fails intermittently

2018-02-22 Thread Gabor Bota (JIRA)
Gabor Bota created HDFS-13179: - Summary: TestLazyPersistReplicaRecovery#testDnRestartWithSavedReplicas fails intermittently Key: HDFS-13179 URL: https://issues.apache.org/jira/browse/HDFS-13179 Project:

[jira] [Commented] (HDFS-13056) Expose file-level composite CRCs in HDFS which are comparable across different instances/layouts

2018-02-22 Thread Dennis Huo (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-13056?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16373729#comment-16373729 ] Dennis Huo commented on HDFS-13056: --- [~szetszwo] and [~drankye] may also be interested in reviewing;

[jira] [Updated] (HDFS-13056) Expose file-level composite CRCs in HDFS which are comparable across different instances/layouts

2018-02-22 Thread Dennis Huo (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-13056?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dennis Huo updated HDFS-13056: -- Status: Patch Available (was: Open) > Expose file-level composite CRCs in HDFS which are comparable

[jira] [Updated] (HDFS-13180) Implement security for Hadoop Distributed Storage Layer

2018-02-22 Thread Anu Engineer (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-13180?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Anu Engineer updated HDFS-13180: Attachment: HadoopStorageLayerSecurity.pdf > Implement security for Hadoop Distributed Storage

[jira] [Updated] (HDFS-13180) Implement security for Hadoop Distributed Storage Layer

2018-02-22 Thread Anu Engineer (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-13180?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Anu Engineer updated HDFS-13180: Issue Type: New Feature (was: Bug) > Implement security for Hadoop Distributed Storage Layer >

[jira] [Updated] (HDFS-13179) TestLazyPersistReplicaRecovery#testDnRestartWithSavedReplicas fails intermittently

2018-02-22 Thread Gabor Bota (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-13179?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gabor Bota updated HDFS-13179: -- Description: The error caused by TimeoutException because the test is waiting to ensure that the file

[jira] [Updated] (HDFS-13179) TestLazyPersistReplicaRecovery#testDnRestartWithSavedReplicas fails intermittently

2018-02-22 Thread Gabor Bota (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-13179?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gabor Bota updated HDFS-13179: -- Description: The error caused by TimeoutException because the test is waiting to ensure that the file

[jira] [Updated] (HDFS-13056) Expose file-level composite CRCs in HDFS which are comparable across different instances/layouts

2018-02-22 Thread Dennis Huo (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-13056?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dennis Huo updated HDFS-13056: -- Status: Open (was: Patch Available) > Expose file-level composite CRCs in HDFS which are comparable

[jira] [Updated] (HDFS-13056) Expose file-level composite CRCs in HDFS which are comparable across different instances/layouts

2018-02-22 Thread Dennis Huo (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-13056?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dennis Huo updated HDFS-13056: -- Attachment: HDFS-13056.003.patch > Expose file-level composite CRCs in HDFS which are comparable across

[jira] [Commented] (HDFS-11394) Add method for getting erasure coding policy through WebHDFS

2018-02-22 Thread Hanisha Koneru (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11394?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16373651#comment-16373651 ] Hanisha Koneru commented on HDFS-11394: --- Hi [~lewuathe], thanks for working on this.  Can you

[jira] [Commented] (HDFS-7240) Scaling HDFS

2018-02-22 Thread Anu Engineer (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-7240?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16373712#comment-16373712 ] Anu Engineer commented on HDFS-7240: [~shv] Thank you for the comments. I have created HDFS-13180 to

[jira] [Commented] (HDFS-13001) Testcase improvement for DFSAdmin

2018-02-22 Thread genericqa (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-13001?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16373926#comment-16373926 ] genericqa commented on HDFS-13001: -- | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote || Subsystem ||

[jira] [Updated] (HDFS-13164) File not closed if append fail with DSQuotaExceededException

2018-02-22 Thread Xiao Chen (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-13164?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Chen updated HDFS-13164: - Attachment: HDFS-13164.02.patch > File not closed if append fail with DSQuotaExceededException >

[jira] [Updated] (HDFS-12299) Race Between update pipeline and DN Re-Registration

2018-02-22 Thread Konstantin Shvachko (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12299?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Konstantin Shvachko updated HDFS-12299: --- Target Version/s: 2.8.2, 2.7.6 (was: 2.8.2, 2.7.5) +1 for the branch-2.7 patch.

[jira] [Commented] (HDFS-12641) Backport HDFS-11755 into branch-2.7 to fix a regression in HDFS-11445

2018-02-22 Thread Konstantin Shvachko (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12641?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16373817#comment-16373817 ] Konstantin Shvachko commented on HDFS-12641: Same as [~brahmareddy] I see the test succeeding

[jira] [Commented] (HDFS-13056) Expose file-level composite CRCs in HDFS which are comparable across different instances/layouts

2018-02-22 Thread genericqa (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-13056?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16373846#comment-16373846 ] genericqa commented on HDFS-13056: -- | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote || Subsystem ||

[jira] [Commented] (HDFS-8966) Separate the lock used in namespace and block management layer

2018-02-22 Thread He Xiaoqiao (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8966?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16373861#comment-16373861 ] He Xiaoqiao commented on HDFS-8966: --- [~wheat9],[~jingzhao] is this issue still ongoing? > Separate the

[jira] [Updated] (HDFS-13178) Disk Balancer: Add a force option to DiskBalancer Execute command

2018-02-22 Thread Bharat Viswanadham (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-13178?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bharat Viswanadham updated HDFS-13178: -- Summary: Disk Balancer: Add a force option to DiskBalancer Execute command (was: Add a

[jira] [Commented] (HDFS-13001) Testcase improvement for DFSAdmin

2018-02-22 Thread Jianfei Jiang (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-13001?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16373800#comment-16373800 ] Jianfei Jiang commented on HDFS-13001: -- Upload patch 3 to fix the checkstyle error. > Testcase

[jira] [Commented] (HDFS-13136) Avoid taking FSN lock while doing group member lookup for FSD permission check

2018-02-22 Thread genericqa (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-13136?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16373828#comment-16373828 ] genericqa commented on HDFS-13136: -- | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote || Subsystem ||

[jira] [Commented] (HDFS-13181) DiskBalancer: Add an configuration for valid plan hours

2018-02-22 Thread genericqa (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-13181?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16373918#comment-16373918 ] genericqa commented on HDFS-13181: -- | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote || Subsystem ||

[jira] [Updated] (HDFS-13001) Testcase improvement for DFSAdmin

2018-02-22 Thread Jianfei Jiang (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-13001?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jianfei Jiang updated HDFS-13001: - Status: Patch Available (was: In Progress) > Testcase improvement for DFSAdmin >

[jira] [Updated] (HDFS-13001) Testcase improvement for DFSAdmin

2018-02-22 Thread Jianfei Jiang (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-13001?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jianfei Jiang updated HDFS-13001: - Attachment: HDFS-13001.003.patch > Testcase improvement for DFSAdmin >

[jira] [Updated] (HDFS-13001) Testcase improvement for DFSAdmin

2018-02-22 Thread Jianfei Jiang (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-13001?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jianfei Jiang updated HDFS-13001: - Status: In Progress (was: Patch Available) > Testcase improvement for DFSAdmin >

[jira] [Commented] (HDFS-12070) Failed block recovery leaves files open indefinitely and at risk for data loss

2018-02-22 Thread genericqa (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12070?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16373799#comment-16373799 ] genericqa commented on HDFS-12070: -- | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote || Subsystem ||

[jira] [Updated] (HDFS-13136) Avoid taking FSN lock while doing group member lookup for FSD permission check

2018-02-22 Thread Xiaoyu Yao (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-13136?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiaoyu Yao updated HDFS-13136: -- Fix Version/s: 3.2.0 3.1.0 > Avoid taking FSN lock while doing group member lookup

[jira] [Commented] (HDFS-12975) Changes to the NameNode to support reads from standby

2018-02-22 Thread Chao Sun (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12975?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16373919#comment-16373919 ] Chao Sun commented on HDFS-12975: - OK, sounds good. I'll post a patch soon without the failover change.

[jira] [Commented] (HDFS-13164) File not closed if append fail with DSQuotaExceededException

2018-02-22 Thread Xiao Chen (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-13164?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16373944#comment-16373944 ] Xiao Chen commented on HDFS-13164: -- Thanks [~yzhangal] for the review. [^HDFS-13164.02.patch] added the

[jira] [Updated] (HDFS-12070) Failed block recovery leaves files open indefinitely and at risk for data loss

2018-02-22 Thread Kihwal Lee (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12070?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kihwal Lee updated HDFS-12070: -- Attachment: HDFS-12070.1.patch > Failed block recovery leaves files open indefinitely and at risk for

[jira] [Commented] (HDFS-12070) Failed block recovery leaves files open indefinitely and at risk for data loss

2018-02-22 Thread Kihwal Lee (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12070?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16373582#comment-16373582 ] Kihwal Lee commented on HDFS-12070: --- Attaching an updated patch that commits right away excluding

[jira] [Commented] (HDFS-13040) Kerberized inotify client fails despite kinit properly

2018-02-22 Thread Xiao Chen (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-13040?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16373627#comment-16373627 ] Xiao Chen commented on HDFS-13040: -- Verified patch 5 works in Istvan's cluster too. [~daryn] could you

[jira] [Updated] (HDFS-13158) Fix Spelling Mistakes - DECOMISSIONED

2018-02-22 Thread Jianfei Jiang (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-13158?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jianfei Jiang updated HDFS-13158: - Summary: Fix Spelling Mistakes - DECOMISSIONED (was: Fix Spelling MIstakes - DECOMISSIONED) >

[jira] [Updated] (HDFS-13147) Support -c argument for DFS command head and tail

2018-02-22 Thread Jianfei Jiang (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-13147?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jianfei Jiang updated HDFS-13147: - Affects Version/s: 3.0.0 > Support -c argument for DFS command head and tail >

[jira] [Created] (HDFS-13182) Allow Observer to participate in NameNode failover

2018-02-22 Thread Chao Sun (JIRA)
Chao Sun created HDFS-13182: --- Summary: Allow Observer to participate in NameNode failover Key: HDFS-13182 URL: https://issues.apache.org/jira/browse/HDFS-13182 Project: Hadoop HDFS Issue Type:

[jira] [Updated] (HDFS-13164) File not closed if append fail with DSQuotaExceededException

2018-02-22 Thread Xiao Chen (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-13164?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Chen updated HDFS-13164: - Attachment: (was: HDFS-13164.02.patch) > File not closed if append fail with DSQuotaExceededException

[jira] [Updated] (HDFS-13164) File not closed if append fail with DSQuotaExceededException

2018-02-22 Thread Xiao Chen (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-13164?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Chen updated HDFS-13164: - Attachment: HDFS-13164.02.patch > File not closed if append fail with DSQuotaExceededException >

[jira] [Updated] (HDFS-13164) File not closed if append fail with DSQuotaExceededException

2018-02-22 Thread Xiao Chen (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-13164?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Chen updated HDFS-13164: - Target Version/s: 2.8.4, 3.0.2 (was: 2.8.4, 2.7.6, 3.0.2) > File not closed if append fail with

[jira] [Updated] (HDFS-13164) File not closed if append fail with DSQuotaExceededException

2018-02-22 Thread Xiao Chen (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-13164?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Chen updated HDFS-13164: - Affects Version/s: (was: 2.6.5) 2.8.0 > File not closed if append fail with

[jira] [Updated] (HDFS-13147) Support -c argument for DFS command head and tail

2018-02-22 Thread Jianfei Jiang (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-13147?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jianfei Jiang updated HDFS-13147: - Component/s: shell > Support -c argument for DFS command head and tail >

[jira] [Updated] (HDFS-12975) Changes to the NameNode to support reads from standby

2018-02-22 Thread Chao Sun (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12975?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chao Sun updated HDFS-12975: Attachment: HDFS-12975.001.patch > Changes to the NameNode to support reads from standby >

[jira] [Assigned] (HDFS-12976) Introduce StandbyReadProxyProvider

2018-02-22 Thread Chao Sun (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12976?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chao Sun reassigned HDFS-12976: --- Assignee: Chao Sun > Introduce StandbyReadProxyProvider > -- > >

[jira] [Commented] (HDFS-13052) WebHDFS: Add support for snasphot diff

2018-02-22 Thread genericqa (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-13052?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16374029#comment-16374029 ] genericqa commented on HDFS-13052: -- | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote || Subsystem ||

[jira] [Comment Edited] (HDFS-13081) Datanode#checkSecureConfig should check HTTPS and SASL encryption

2018-02-22 Thread Xiaoyu Yao (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-13081?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16373763#comment-16373763 ] Xiaoyu Yao edited comment on HDFS-13081 at 2/23/18 1:05 AM: Thanks [~ajayydv]

[jira] [Commented] (HDFS-13081) Datanode#checkSecureConfig should check HTTPS and SASL encryption

2018-02-22 Thread Xiaoyu Yao (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-13081?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16373763#comment-16373763 ] Xiaoyu Yao commented on HDFS-13081: --- Thanks [~ajayydv] for the update. Patch v2 looks good to me

[jira] [Created] (HDFS-13181) DiskBalancer: Add an configuration for valid plan hours

2018-02-22 Thread Bharat Viswanadham (JIRA)
Bharat Viswanadham created HDFS-13181: - Summary: DiskBalancer: Add an configuration for valid plan hours Key: HDFS-13181 URL: https://issues.apache.org/jira/browse/HDFS-13181 Project: Hadoop HDFS

[jira] [Updated] (HDFS-13181) DiskBalancer: Add an configuration for valid plan hours

2018-02-22 Thread Bharat Viswanadham (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-13181?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bharat Viswanadham updated HDFS-13181: -- Attachment: HDFS-13181.00.patch > DiskBalancer: Add an configuration for valid plan

  1   2   >