[jira] [Commented] (HDFS-11503) Integrate Chocolate Cloud RS coder implementation

2017-04-11 Thread Wei-Chiu Chuang (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11503?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15964470#comment-15964470 ] Wei-Chiu Chuang commented on HDFS-11503: I think you will also need to file an Apache Infra ticket

[jira] [Commented] (HDFS-11530) Use HDFS specific network topology to choose datanode in BlockPlacementPolicyDefault

2017-04-11 Thread Chen Liang (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11530?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15964652#comment-15964652 ] Chen Liang commented on HDFS-11530: --- bq. I am thinking one reasonable way that we can use

[jira] [Updated] (HDFS-11641) Reduce cost of audit logging by using FileStatus instead of HdfsFileStatus

2017-04-11 Thread Daryn Sharp (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11641?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Daryn Sharp updated HDFS-11641: --- Status: Patch Available (was: Open) > Reduce cost of audit logging by using FileStatus instead of

[jira] [Updated] (HDFS-11641) Reduce cost of audit logging by using FileStatus instead of HdfsFileStatus

2017-04-11 Thread Daryn Sharp (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11641?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Daryn Sharp updated HDFS-11641: --- Attachment: HDFS-11641.patch Very simple patch to change to swap HdfsFileStatus to FileStatus.

[jira] [Updated] (HDFS-11190) Namenode support for data stored in external stores.

2017-04-11 Thread Virajith Jalaparti (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11190?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Virajith Jalaparti updated HDFS-11190: -- Attachment: (was: HDFS-11190-HDFS-9806.003.patch) > Namenode support for data

[jira] [Updated] (HDFS-11190) Namenode support for data stored in external stores.

2017-04-11 Thread Virajith Jalaparti (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11190?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Virajith Jalaparti updated HDFS-11190: -- Status: Open (was: Patch Available) > Namenode support for data stored in external

[jira] [Updated] (HDFS-11190) Namenode support for data stored in external stores.

2017-04-11 Thread Virajith Jalaparti (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11190?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Virajith Jalaparti updated HDFS-11190: -- Attachment: HDFS-11190-HDFS-9806.003.patch > Namenode support for data stored in

[jira] [Commented] (HDFS-11163) Mover should move the file blocks to default storage once policy is unset

2017-04-11 Thread Chris Nauroth (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11163?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15964553#comment-15964553 ] Chris Nauroth commented on HDFS-11163: -- [~surendrasingh], it looks like HDFS-11163-branch-2.002.patch

[jira] [Commented] (HDFS-11338) [SPS]: Fix timeout issue in unit tests caused by longger NN down time

2017-04-11 Thread Hadoop QA (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11338?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15964622#comment-15964622 ] Hadoop QA commented on HDFS-11338: -- | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote || Subsystem ||

[jira] [Commented] (HDFS-11641) Reduce cost of audit logging by using FileStatus instead of HdfsFileStatus

2017-04-11 Thread Haohui Mai (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11641?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15964682#comment-15964682 ] Haohui Mai commented on HDFS-11641: --- +1 pending Jenkins. > Reduce cost of audit logging by using

[jira] [Comment Edited] (HDFS-11641) Reduce cost of audit logging by using FileStatus instead of HdfsFileStatus

2017-04-11 Thread Haohui Mai (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11641?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15964682#comment-15964682 ] Haohui Mai edited comment on HDFS-11641 at 4/11/17 5:30 PM: Thanks for the

[jira] [Commented] (HDFS-11163) Mover should move the file blocks to default storage once policy is unset

2017-04-11 Thread Hadoop QA (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11163?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15964549#comment-15964549 ] Hadoop QA commented on HDFS-11163: -- | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote || Subsystem ||

[jira] [Created] (HDFS-11641) Reduce cost of audit logging by using FileStatus instead of HdfsFileStatus

2017-04-11 Thread Daryn Sharp (JIRA)
Daryn Sharp created HDFS-11641: -- Summary: Reduce cost of audit logging by using FileStatus instead of HdfsFileStatus Key: HDFS-11641 URL: https://issues.apache.org/jira/browse/HDFS-11641 Project: Hadoop

[jira] [Updated] (HDFS-11642) Block Storage: fix TestCBlockCLI and TestCBlockServerPersistence cleanup

2017-04-11 Thread Xiaoyu Yao (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11642?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiaoyu Yao updated HDFS-11642: -- Status: Patch Available (was: Open) > Block Storage: fix TestCBlockCLI and TestCBlockServerPersistence

[jira] [Commented] (HDFS-11062) Ozone:SCM: Explore if we can remove nullcommand

2017-04-11 Thread Xiaoyu Yao (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11062?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15964917#comment-15964917 ] Xiaoyu Yao commented on HDFS-11062: --- +1 for v3 patch. The unit test failures are unrelated and tracked

[jira] [Updated] (HDFS-11062) Ozone:SCM: Remove null command

2017-04-11 Thread Xiaoyu Yao (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11062?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiaoyu Yao updated HDFS-11062: -- Summary: Ozone:SCM: Remove null command (was: Ozone:SCM: Explore if we can remove nullcommand) >

[jira] [Commented] (HDFS-11504) Ozone: SCM: Add AllocateBlock/DeleteBlock/GetBlock APIs

2017-04-11 Thread Hadoop QA (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11504?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15964924#comment-15964924 ] Hadoop QA commented on HDFS-11504: -- | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote || Subsystem ||

[jira] [Updated] (HDFS-11062) Ozone:SCM: Remove null command

2017-04-11 Thread Xiaoyu Yao (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11062?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiaoyu Yao updated HDFS-11062: -- Resolution: Fixed Hadoop Flags: Reviewed Status: Resolved (was: Patch Available)

[jira] [Created] (HDFS-11643) Balancer fencing fails when writing erasure coded lock file

2017-04-11 Thread Andrew Wang (JIRA)
Andrew Wang created HDFS-11643: -- Summary: Balancer fencing fails when writing erasure coded lock file Key: HDFS-11643 URL: https://issues.apache.org/jira/browse/HDFS-11643 Project: Hadoop HDFS

[jira] [Commented] (HDFS-11643) Balancer fencing fails when writing erasure coded lock file

2017-04-11 Thread Andrew Wang (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11643?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15964941#comment-15964941 ] Andrew Wang commented on HDFS-11643: Heads up [~Sammi], this relates to our discussion about a

[jira] [Commented] (HDFS-11641) Reduce cost of audit logging by using FileStatus instead of HdfsFileStatus

2017-04-11 Thread Daryn Sharp (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11641?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15964847#comment-15964847 ] Daryn Sharp commented on HDFS-11641: Ug. Need to update to remove 2 lines that slipped in. This is

[jira] [Commented] (HDFS-11642) Block Storage: fix TestCBlockCLI and TestCBlockServerPersistence cleanup

2017-04-11 Thread Xiaoyu Yao (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11642?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15964913#comment-15964913 ] Xiaoyu Yao commented on HDFS-11642: ---

[jira] [Commented] (HDFS-11641) Reduce cost of audit logging by using FileStatus instead of HdfsFileStatus

2017-04-11 Thread Hadoop QA (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11641?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15964835#comment-15964835 ] Hadoop QA commented on HDFS-11641: -- | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote || Subsystem ||

[jira] [Created] (HDFS-11642) Block Storage: fix TestCBlockCLI and TestCBlockServerPersistence cleanup

2017-04-11 Thread Xiaoyu Yao (JIRA)
Xiaoyu Yao created HDFS-11642: - Summary: Block Storage: fix TestCBlockCLI and TestCBlockServerPersistence cleanup Key: HDFS-11642 URL: https://issues.apache.org/jira/browse/HDFS-11642 Project: Hadoop

[jira] [Updated] (HDFS-11642) Block Storage: fix TestCBlockCLI and TestCBlockServerPersistence cleanup

2017-04-11 Thread Xiaoyu Yao (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11642?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiaoyu Yao updated HDFS-11642: -- Attachment: HDFS-11642-HDFS-7240.001.patch > Block Storage: fix TestCBlockCLI and

[jira] [Updated] (HDFS-11640) Datanodes should use a unique identifier when reading from external stores

2017-04-11 Thread Virajith Jalaparti (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11640?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Virajith Jalaparti updated HDFS-11640: -- Description: Use a unique identifier when reading from external stores to ensure that

[jira] [Updated] (HDFS-11640) Datanodes should use a unique identifier when reading from external stores

2017-04-11 Thread Virajith Jalaparti (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11640?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Virajith Jalaparti updated HDFS-11640: -- Summary: Datanodes should use a unique identifier when reading from external stores

[jira] [Commented] (HDFS-11608) HDFS write crashed with block size greater than 2 GB

2017-04-11 Thread Xiaobing Zhou (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11608?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15964731#comment-15964731 ] Xiaobing Zhou commented on HDFS-11608: -- Posted 2.7 patch. Thanks [~xyao] for committing it and all

[jira] [Updated] (HDFS-11608) HDFS write crashed with block size greater than 2 GB

2017-04-11 Thread Xiaobing Zhou (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11608?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiaobing Zhou updated HDFS-11608: - Attachment: HDFS-11608-branch-2.7.003.patch > HDFS write crashed with block size greater than 2

[jira] [Updated] (HDFS-11608) HDFS write crashed with block size greater than 2 GB

2017-04-11 Thread Arpit Agarwal (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11608?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Arpit Agarwal updated HDFS-11608: - Target Version/s: (was: 3.0.0-alpha3, 2.8.1) Fix Version/s: 2.7.4 +1 for the branch-2.7

[jira] [Updated] (HDFS-11504) Ozone: SCM: Add AllocateBlock/DeleteBlock/GetBlock APIs

2017-04-11 Thread Xiaoyu Yao (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11504?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiaoyu Yao updated HDFS-11504: -- Attachment: HDFS-11504-HDFS-7240.001.patch Attach an initial patch with unit test. > Ozone: SCM: Add

[jira] [Updated] (HDFS-11504) Ozone: SCM: Add AllocateBlock/DeleteBlock/GetBlock APIs

2017-04-11 Thread Xiaoyu Yao (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11504?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiaoyu Yao updated HDFS-11504: -- Attachment: (was: HDFS-11504-HDFS-7240.001.patch) > Ozone: SCM: Add

[jira] [Updated] (HDFS-11504) Ozone: SCM: Add AllocateBlock/DeleteBlock/GetBlock APIs

2017-04-11 Thread Xiaoyu Yao (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11504?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiaoyu Yao updated HDFS-11504: -- Attachment: HDFS-11504-HDFS-7240.001.patch > Ozone: SCM: Add AllocateBlock/DeleteBlock/GetBlock APIs >

[jira] [Updated] (HDFS-11190) Namenode support for data stored in external stores.

2017-04-11 Thread Virajith Jalaparti (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11190?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Virajith Jalaparti updated HDFS-11190: -- Status: Patch Available (was: Open) > Namenode support for data stored in external

[jira] [Updated] (HDFS-11504) Ozone: SCM: Add AllocateBlock/DeleteBlock/GetBlock APIs

2017-04-11 Thread Xiaoyu Yao (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11504?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiaoyu Yao updated HDFS-11504: -- Status: Patch Available (was: Open) > Ozone: SCM: Add AllocateBlock/DeleteBlock/GetBlock APIs >

[jira] [Updated] (HDFS-11163) Mover should move the file blocks to default storage once policy is unset

2017-04-11 Thread Surendra Singh Lilhore (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11163?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Surendra Singh Lilhore updated HDFS-11163: -- Attachment: HDFS-11163-branch-2.003.patch Sorry, Its my mistake. I didn't pull

[jira] [Updated] (HDFS-11565) Use compact identifiers for built-in ECPolicies in HdfsFileStatus

2017-04-11 Thread Andrew Wang (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11565?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Wang updated HDFS-11565: --- Attachment: HDFS-11565.003.patch Sure, good idea Wei-chiu, added some Precondition checks and new

[jira] [Commented] (HDFS-11530) Use HDFS specific network topology to choose datanode in BlockPlacementPolicyDefault

2017-04-11 Thread Hadoop QA (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11530?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15965097#comment-15965097 ] Hadoop QA commented on HDFS-11530: -- | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote || Subsystem ||

[jira] [Commented] (HDFS-11558) BPServiceActor thread name is too long

2017-04-11 Thread Hudson (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11558?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15965095#comment-15965095 ] Hudson commented on HDFS-11558: --- ABORTED: Integrated in Jenkins build Hadoop-trunk-Commit #11572 (See

[jira] [Updated] (HDFS-11551) Handle SlowDiskReport from DataNode at the NameNode

2017-04-11 Thread Hanisha Koneru (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11551?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hanisha Koneru updated HDFS-11551: -- Attachment: HDFS-11551-branch-2.002.patch > Handle SlowDiskReport from DataNode at the NameNode

[jira] [Created] (HDFS-11644) DFSStripedOutputStream should not implement Syncable

2017-04-11 Thread Andrew Wang (JIRA)
Andrew Wang created HDFS-11644: -- Summary: DFSStripedOutputStream should not implement Syncable Key: HDFS-11644 URL: https://issues.apache.org/jira/browse/HDFS-11644 Project: Hadoop HDFS Issue

[jira] [Commented] (HDFS-11504) Ozone: SCM: Add AllocateBlock/DeleteBlock/GetBlock APIs

2017-04-11 Thread Hadoop QA (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11504?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15965148#comment-15965148 ] Hadoop QA commented on HDFS-11504: -- | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote || Subsystem ||

[jira] [Updated] (HDFS-11645) DataXceiver thread should log the actual error when getting InvalidMagicNumberException

2017-04-11 Thread Chen Liang (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11645?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chen Liang updated HDFS-11645: -- Attachment: HDFS-11645.001.patch > DataXceiver thread should log the actual error when getting >

[jira] [Created] (HDFS-11645) DataXceiver thread should log the actual error when getting InvalidMagicNumberException

2017-04-11 Thread Chen Liang (JIRA)
Chen Liang created HDFS-11645: - Summary: DataXceiver thread should log the actual error when getting InvalidMagicNumberException Key: HDFS-11645 URL: https://issues.apache.org/jira/browse/HDFS-11645

[jira] [Commented] (HDFS-11643) Balancer fencing fails when writing erasure coded lock file

2017-04-11 Thread SammiChen (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11643?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15965269#comment-15965269 ] SammiChen commented on HDFS-11643: -- Hi [~andrew.wang], yea, we need face this a "replicated" EC policy

[jira] [Commented] (HDFS-11338) [SPS]: Fix timeout issue in unit tests caused by longger NN down time

2017-04-11 Thread Rakesh R (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11338?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15965333#comment-15965333 ] Rakesh R commented on HDFS-11338: - Thank you [~umamaheswararao] for the reviews and commits. > [SPS]: Fix

[jira] [Assigned] (HDFS-11338) [SPS]: Fix timeout issue in unit tests caused by longger NN down time

2017-04-11 Thread Rakesh R (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11338?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rakesh R reassigned HDFS-11338: --- Assignee: Rakesh R (was: Wei Zhou) > [SPS]: Fix timeout issue in unit tests caused by longger NN

[jira] [Commented] (HDFS-11645) DataXceiver thread should log the actual error when getting InvalidMagicNumberException

2017-04-11 Thread Chen Liang (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11645?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15965149#comment-15965149 ] Chen Liang commented on HDFS-11645: --- Post v001 patch with the exception logged. > DataXceiver thread

[jira] [Updated] (HDFS-11645) DataXceiver thread should log the actual error when getting InvalidMagicNumberException

2017-04-11 Thread Chen Liang (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11645?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chen Liang updated HDFS-11645: -- Status: Patch Available (was: Open) > DataXceiver thread should log the actual error when getting >

[jira] [Commented] (HDFS-11558) BPServiceActor thread name is too long

2017-04-11 Thread Mingliang Liu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11558?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15965054#comment-15965054 ] Mingliang Liu commented on HDFS-11558: -- Committed to {{trunk}} branch. Keep this open for

[jira] [Commented] (HDFS-11634) Optimize BlockIterator when interating starts in the middle.

2017-04-11 Thread Hadoop QA (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11634?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15965340#comment-15965340 ] Hadoop QA commented on HDFS-11634: -- | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote || Subsystem ||

[jira] [Updated] (HDFS-11558) BPServiceActor thread name is too long

2017-04-11 Thread Mingliang Liu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11558?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mingliang Liu updated HDFS-11558: - Resolution: Fixed Fix Version/s: 3.0.0-alpha3 Target Version/s: 3.0.0-alpha3,

[jira] [Commented] (HDFS-11470) Ozone: SCM: Add SCM CLI

2017-04-11 Thread Xiaoyu Yao (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11470?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15965067#comment-15965067 ] Xiaoyu Yao commented on HDFS-11470: --- Thanks [~anu] for writing up and updating the doc and all for the

[jira] [Updated] (HDFS-11558) BPServiceActor thread name is too long

2017-04-11 Thread Xiaobing Zhou (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11558?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiaobing Zhou updated HDFS-11558: - Attachment: HDFS-11558-branch-2.006.patch > BPServiceActor thread name is too long >

[jira] [Commented] (HDFS-11636) Ozone: TestContainerPlacement fails because of string mismatch in expected Message

2017-04-11 Thread Anu Engineer (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11636?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15965127#comment-15965127 ] Anu Engineer commented on HDFS-11636: - +1, thanks for catching and fix this. I will commit this

[jira] [Updated] (HDFS-11630) TestThrottledAsyncCheckerTimeout fails intermittently in Jenkins builds

2017-04-11 Thread Hanisha Koneru (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11630?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hanisha Koneru updated HDFS-11630: -- Attachment: HDFS-11630.002.patch [~arpitagarwal], thank you for the suggestions. - Used mockito

[jira] [Updated] (HDFS-11551) Handle SlowDiskReport from DataNode at the NameNode

2017-04-11 Thread Arpit Agarwal (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11551?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Arpit Agarwal updated HDFS-11551: - Fix Version/s: 2.9.0 +1 for the v2 branch-2 patch. I've committed it. Thanks [~hanishakoneru]. >

[jira] [Commented] (HDFS-11576) Block recovery will fail indefinitely if recovery time > heartbeat interval

2017-04-11 Thread Inigo Goiri (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11576?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15965185#comment-15965185 ] Inigo Goiri commented on HDFS-11576: I would log the events the other way around: * Info: when a block

[jira] [Resolved] (HDFS-11552) Erasure Coding: Support Parity Blocks placement onto same nodes hosting Data Blocks when DataNodes are insufficient

2017-04-11 Thread Andrew Wang (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11552?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Wang resolved HDFS-11552. Resolution: Not A Problem > Erasure Coding: Support Parity Blocks placement onto same nodes hosting

[jira] [Commented] (HDFS-11546) Federation Router RPC server

2017-04-11 Thread Chris Douglas (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11546?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15965226#comment-15965226 ] Chris Douglas commented on HDFS-11546: -- Looks good, overall. A few questions: * Many fields e.g.,

[jira] [Updated] (HDFS-11640) Datanodes should use a unique identifier when reading from external stores

2017-04-11 Thread Virajith Jalaparti (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11640?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Virajith Jalaparti updated HDFS-11640: -- Attachment: HDFS-11640-HDFS-9806.001.patch Attaching a PoC patch that is based on the

[jira] [Updated] (HDFS-11634) Optimize BlockIterator when interating starts in the middle.

2017-04-11 Thread Konstantin Shvachko (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11634?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Konstantin Shvachko updated HDFS-11634: --- Attachment: HDFS-11634.003.patch * Fixed the typos. Thanks. * Yes this the corner

[jira] [Comment Edited] (HDFS-11558) BPServiceActor thread name is too long

2017-04-11 Thread Mingliang Liu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11558?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15965054#comment-15965054 ] Mingliang Liu edited comment on HDFS-11558 at 4/11/17 10:38 PM: Committed

[jira] [Commented] (HDFS-11615) FSNamesystemLock metrics can be inaccurate due to millisecond precision

2017-04-11 Thread Andrew Wang (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11615?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15965062#comment-15965062 ] Andrew Wang commented on HDFS-11615: These metrics are normally consumed by a metrics system and

[jira] [Commented] (HDFS-9342) Erasure coding: client should update and commit block based on acknowledged size

2017-04-11 Thread Andrew Wang (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9342?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15965164#comment-15965164 ] Andrew Wang commented on HDFS-9342: --- LGTM, thanks Sammi for refining, only some minors: * Please add a

[jira] [Commented] (HDFS-10531) Add EC policy and storage policy related usage summarization function to dfs du command

2017-04-11 Thread Andrew Wang (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10531?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15965223#comment-15965223 ] Andrew Wang commented on HDFS-10531: Hi Sammi, thanks for the comment, bq. From user's point of view,

[jira] [Commented] (HDFS-11642) Block Storage: fix TestCBlockCLI and TestCBlockServerPersistence cleanup

2017-04-11 Thread Hadoop QA (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11642?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15965061#comment-15965061 ] Hadoop QA commented on HDFS-11642: -- | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote || Subsystem ||

[jira] [Commented] (HDFS-11558) BPServiceActor thread name is too long

2017-04-11 Thread Xiaobing Zhou (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11558?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15965089#comment-15965089 ] Xiaobing Zhou commented on HDFS-11558: -- Posted branch-2 patch. Thanks all. > BPServiceActor thread

[jira] [Commented] (HDFS-10996) Ability to specify per-file EC policy at create time

2017-04-11 Thread Andrew Wang (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10996?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15965109#comment-15965109 ] Andrew Wang commented on HDFS-10996: Hi Sammi, thanks for working on this, sorry for the slow review:

[jira] [Commented] (HDFS-11530) Use HDFS specific network topology to choose datanode in BlockPlacementPolicyDefault

2017-04-11 Thread Chen Liang (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11530?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15965124#comment-15965124 ] Chen Liang commented on HDFS-11530: --- The failed tests seem unrelated, they all passed in my local run.

[jira] [Commented] (HDFS-11641) Reduce cost of audit logging by using FileStatus instead of HdfsFileStatus

2017-04-11 Thread Hadoop QA (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11641?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15965194#comment-15965194 ] Hadoop QA commented on HDFS-11641: -- | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote || Subsystem ||

[jira] [Commented] (HDFS-11552) Erasure Coding: Support Parity Blocks placement onto same nodes hosting Data Blocks when DataNodes are insufficient

2017-04-11 Thread Andrew Wang (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11552?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15965214#comment-15965214 ] Andrew Wang commented on HDFS-11552: I think with the XOR-2-1 policy this also becomes less important,

[jira] [Updated] (HDFS-11604) Define and parse erasure code codecs, schemas and policies

2017-04-11 Thread SammiChen (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11604?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] SammiChen updated HDFS-11604: - Status: Patch Available (was: Open) > Define and parse erasure code codecs, schemas and policies >

[jira] [Updated] (HDFS-11641) Reduce cost of audit logging by using FileStatus instead of HdfsFileStatus

2017-04-11 Thread Daryn Sharp (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11641?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Daryn Sharp updated HDFS-11641: --- Attachment: HDFS-11641.1.patch Merely undid 1-line change in FSDirAttrOp for creating the toRemove

[jira] [Commented] (HDFS-11565) Use compact identifiers for built-in ECPolicies in HdfsFileStatus

2017-04-11 Thread Hadoop QA (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11565?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15965193#comment-15965193 ] Hadoop QA commented on HDFS-11565: -- | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote || Subsystem ||

[jira] [Commented] (HDFS-11636) Ozone: TestContainerPlacement fails because of string mismatch in expected Message

2017-04-11 Thread Anu Engineer (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11636?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15965212#comment-15965212 ] Anu Engineer commented on HDFS-11636: - There is a merge conflict with the top of the tree, can you

[jira] [Commented] (HDFS-11163) Mover should move the file blocks to default storage once policy is unset

2017-04-11 Thread Hadoop QA (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11163?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15965215#comment-15965215 ] Hadoop QA commented on HDFS-11163: -- | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote || Subsystem ||

[jira] [Commented] (HDFS-11645) DataXceiver thread should log the actual error when getting InvalidMagicNumberException

2017-04-11 Thread Hadoop QA (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11645?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15965237#comment-15965237 ] Hadoop QA commented on HDFS-11645: -- | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote || Subsystem ||

[jira] [Resolved] (HDFS-11630) TestThrottledAsyncCheckerTimeout fails intermittently in Jenkins builds

2017-04-11 Thread Arpit Agarwal (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11630?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Arpit Agarwal resolved HDFS-11630. -- Resolution: Fixed Hadoop Flags: Reviewed Fix Version/s: 3.0.0-alpha3 >

[jira] [Commented] (HDFS-11630) TestThrottledAsyncCheckerTimeout fails intermittently in Jenkins builds

2017-04-11 Thread Arpit Agarwal (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11630?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15965361#comment-15965361 ] Arpit Agarwal commented on HDFS-11630: -- Sorry I just realized after committing it that we didn't get

[jira] [Commented] (HDFS-11384) Add option for balancer to disperse getBlocks calls to avoid NameNode's rpc.CallQueueLength spike

2017-04-11 Thread Konstantin Shvachko (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11384?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15965266#comment-15965266 ] Konstantin Shvachko commented on HDFS-11384: * I am usually very conservative about

[jira] [Updated] (HDFS-11384) Add option for balancer to disperse getBlocks calls to avoid NameNode's rpc.CallQueueLength spike

2017-04-11 Thread Konstantin Shvachko (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11384?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Konstantin Shvachko updated HDFS-11384: --- Attachment: HDFS-11384.004.patch > Add option for balancer to disperse getBlocks

[jira] [Commented] (HDFS-11384) Add option for balancer to disperse getBlocks calls to avoid NameNode's rpc.CallQueueLength spike

2017-04-11 Thread Hadoop QA (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11384?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15965334#comment-15965334 ] Hadoop QA commented on HDFS-11384: -- | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote || Subsystem ||

[jira] [Commented] (HDFS-11630) TestThrottledAsyncCheckerTimeout fails intermittently in Jenkins builds

2017-04-11 Thread Arpit Agarwal (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11630?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15965360#comment-15965360 ] Arpit Agarwal commented on HDFS-11630: -- +1 for the v2 patch, thanks [~hanishakoneru]. I have

[jira] [Updated] (HDFS-11163) Mover should move the file blocks to default storage once policy is unset

2017-04-11 Thread Chris Nauroth (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11163?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chris Nauroth updated HDFS-11163: - Resolution: Fixed Hadoop Flags: Reviewed Fix Version/s: 2.8.1

[jira] [Updated] (HDFS-11636) Ozone: TestContainerPlacement fails because of string mismatch in expected Message

2017-04-11 Thread Mukul Kumar Singh (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11636?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mukul Kumar Singh updated HDFS-11636: - Resolution: Duplicate Status: Resolved (was: Patch Available) HDFS-11062 fixes

[jira] [Commented] (HDFS-11630) TestThrottledAsyncCheckerTimeout fails intermittently in Jenkins builds

2017-04-11 Thread Hudson (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11630?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15965367#comment-15965367 ] Hudson commented on HDFS-11630: --- SUCCESS: Integrated in Jenkins build Hadoop-trunk-Commit #11573 (See

[jira] [Commented] (HDFS-10630) Federation State Store FS Implementation

2017-04-11 Thread Chris Douglas (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10630?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15965369#comment-15965369 ] Chris Douglas commented on HDFS-10630: -- * Are the File impls primarily for testing? Or do routers

[jira] [Commented] (HDFS-11163) Mover should move the file blocks to default storage once policy is unset

2017-04-11 Thread Hudson (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11163?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15965402#comment-15965402 ] Hudson commented on HDFS-11163: --- SUCCESS: Integrated in Jenkins build Hadoop-trunk-Commit #11574 (See

[jira] [Updated] (HDFS-11530) Use HDFS specific network topology to choose datanode in BlockPlacementPolicyDefault

2017-04-11 Thread Chen Liang (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11530?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chen Liang updated HDFS-11530: -- Attachment: HDFS-11530.008.patch Attaching v008 patch to fix the issue. [~linyiqun] [~arpitagarwal]

[jira] [Commented] (HDFS-11338) [SPS]: Fix timeout issue in unit tests caused by longger NN down time

2017-04-11 Thread Uma Maheswara Rao G (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11338?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15964982#comment-15964982 ] Uma Maheswara Rao G commented on HDFS-11338: +1 on the latest patch > [SPS]: Fix timeout

[jira] [Updated] (HDFS-11338) [SPS]: Fix timeout issue in unit tests caused by longger NN down time

2017-04-11 Thread Uma Maheswara Rao G (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11338?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Uma Maheswara Rao G updated HDFS-11338: --- Resolution: Fixed Hadoop Flags: Reviewed Fix Version/s: HDFS-10285

[jira] [Commented] (HDFS-11190) Namenode support for data stored in external stores.

2017-04-11 Thread Hadoop QA (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11190?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15965000#comment-15965000 ] Hadoop QA commented on HDFS-11190: -- | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote || Subsystem ||

[jira] [Updated] (HDFS-11504) Ozone: SCM: Add AllocateBlock/DeleteBlock/GetBlock APIs

2017-04-11 Thread Xiaoyu Yao (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11504?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiaoyu Yao updated HDFS-11504: -- Attachment: HDFS-11504-HDFS-7240.002.patch Fix the checkstyle issue. > Ozone: SCM: Add

[jira] [Commented] (HDFS-11503) Integrate Chocolate Cloud RS coder implementation

2017-04-11 Thread Andrew Wang (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11503?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15965022#comment-15965022 ] Andrew Wang commented on HDFS-11503: Hi [~sw0rdf1sh], We already have assignees actively working on

[jira] [Commented] (HDFS-11634) Optimize BlockIterator when interating starts in the middle.

2017-04-11 Thread Rushabh S Shah (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11634?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15965029#comment-15965029 ] Rushabh S Shah commented on HDFS-11634: --- Minor nit: Couple of typos for 'oterator' I don't

[jira] [Comment Edited] (HDFS-11634) Optimize BlockIterator when interating starts in the middle.

2017-04-11 Thread Rushabh S Shah (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11634?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15965029#comment-15965029 ] Rushabh S Shah edited comment on HDFS-11634 at 4/11/17 10:08 PM: - One

[jira] [Commented] (HDFS-7859) Erasure Coding: Persist erasure coding policies in NameNode

2017-04-11 Thread SammiChen (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-7859?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15963957#comment-15963957 ] SammiChen commented on HDFS-7859: - This JIRA will focus on persist erasure coding policies into fsImage,

[jira] [Updated] (HDFS-11338) [SPS]: Fix timeout issue in unit tests caused by longger NN down time

2017-04-11 Thread Rakesh R (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11338?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rakesh R updated HDFS-11338: Attachment: HDFS-11338-HDFS-10285-05.patch Thanks [~umamaheswararao] for the immediate feedback. Attached

[jira] [Assigned] (HDFS-7859) Erasure Coding: Persist erasure coding policies in NameNode

2017-04-11 Thread SammiChen (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-7859?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] SammiChen reassigned HDFS-7859: --- Assignee: SammiChen > Erasure Coding: Persist erasure coding policies in NameNode >

  1   2   >