[jira] [Commented] (HDFS-7889) Subclass DFSOutputStream to support writing striping layout files

2015-04-02 Thread Vinayakumar B (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-7889?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14392260#comment-14392260 ] Vinayakumar B commented on HDFS-7889: - {quote}Moving newStreamForCreate to DFSClient: I

[jira] [Commented] (HDFS-7717) Erasure Coding: distribute replication to stripping erasure coding conversion work to DataNode

2015-04-02 Thread Kai Zheng (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-7717?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14392297#comment-14392297 ] Kai Zheng commented on HDFS-7717: - I will duplicate of this for non-stripping erasure

[jira] [Commented] (HDFS-7999) FsDatasetImpl#createTemporary sometimes holds the FSDatasetImpl lock for a very long time

2015-04-02 Thread zhouyingchao (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-7999?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14392426#comment-14392426 ] zhouyingchao commented on HDFS-7999: Thanks a lot for the comments, Colin. I would

[jira] [Commented] (HDFS-7717) Erasure Coding: distribute replication to stripping erasure coding conversion work to DataNode

2015-04-02 Thread Kai Zheng (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-7717?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14392329#comment-14392329 ] Kai Zheng commented on HDFS-7717: - OK, glad we're clear and aligned now. Thanks for the

[jira] [Updated] (HDFS-7889) Subclass DFSOutputStream to support writing striping layout files

2015-04-02 Thread Li Bo (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-7889?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Li Bo updated HDFS-7889: Attachment: HDFS-7889-007.patch Subclass DFSOutputStream to support writing striping layout files

[jira] [Commented] (HDFS-7889) Subclass DFSOutputStream to support writing striping layout files

2015-04-02 Thread Li Bo (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-7889?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14392394#comment-14392394 ] Li Bo commented on HDFS-7889: - In patch 007, we will not pad zero-bytes to incomplete stripe

[jira] [Commented] (HDFS-7717) Erasure Coding: distribute replication to stripping erasure coding conversion work to DataNode

2015-04-02 Thread Kai Sasaki (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-7717?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14392298#comment-14392298 ] Kai Sasaki commented on HDFS-7717: -- [~drankye] Thank you. You mean the distribution of

[jira] [Created] (HDFS-8042) Erasure Coding: distribute replication to non-stripping erasure coding conversion work to DataNode

2015-04-02 Thread Kai Zheng (JIRA)
Kai Zheng created HDFS-8042: --- Summary: Erasure Coding: distribute replication to non-stripping erasure coding conversion work to DataNode Key: HDFS-8042 URL: https://issues.apache.org/jira/browse/HDFS-8042

[jira] [Updated] (HDFS-7811) Avoid recursive call getStoragePolicyID in INodeFile#computeQuotaUsage

2015-04-02 Thread Jing Zhao (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-7811?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jing Zhao updated HDFS-7811: Attachment: HDFS-7811.03.patch The 02 patch missed the symlink check before calling {{getStoragePolicyID}}.

[jira] [Updated] (HDFS-7717) Erasure Coding: distribute replication to stripping erasure coding conversion work to DataNode

2015-04-02 Thread Kai Zheng (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-7717?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kai Zheng updated HDFS-7717: Summary: Erasure Coding: distribute replication to stripping erasure coding conversion work to DataNode

[jira] [Commented] (HDFS-7889) Subclass DFSOutputStream to support writing striping layout files

2015-04-02 Thread Vinayakumar B (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-7889?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14392245#comment-14392245 ] Vinayakumar B commented on HDFS-7889: - Hi [~libo-intel], thanks for the nice work.

[jira] [Commented] (HDFS-7717) Erasure Coding: distribute replication to stripping erasure coding conversion work to DataNode

2015-04-02 Thread Kai Sasaki (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-7717?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14392320#comment-14392320 ] Kai Sasaki commented on HDFS-7717: -- [~drankye] Thank you so much for clear explanation! I

[jira] [Updated] (HDFS-7717) Erasure Coding: distribute replication to stripping erasure coding conversion work to DataNode

2015-04-02 Thread Kai Zheng (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-7717?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kai Zheng updated HDFS-7717: Description: In *stripping* erasure coding case, we need some approach to distribute conversion work between

[jira] [Commented] (HDFS-7717) Erasure Coding: distribute replication to stripping erasure coding conversion work to DataNode

2015-04-02 Thread Kai Zheng (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-7717?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14392301#comment-14392301 ] Kai Zheng commented on HDFS-7717: - As I re-purposed this and modified the description, this

[jira] [Commented] (HDFS-7715) Implement the Hitchhiker erasure coding algorithm

2015-04-02 Thread Kai Zheng (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-7715?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14392413#comment-14392413 ] Kai Zheng commented on HDFS-7715: - Jack, If you're worrying about the performance gain thru

[jira] [Commented] (HDFS-7839) Erasure coding: implement facilities in NameNode to create and manage EC zones

2015-04-02 Thread Kai Zheng (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-7839?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14392423#comment-14392423 ] Kai Zheng commented on HDFS-7839: - [~jingzhao], Sorry for I bring the question again, so

[jira] [Commented] (HDFS-7285) Erasure Coding Support inside HDFS

2015-04-02 Thread Kai Zheng (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-7285?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14392290#comment-14392290 ] Kai Zheng commented on HDFS-7285: - Thank you all for the comprehensive discussion and

[jira] [Commented] (HDFS-7889) Subclass DFSOutputStream to support writing striping layout files

2015-04-02 Thread Li Bo (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-7889?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14392385#comment-14392385 ] Li Bo commented on HDFS-7889: - Thanks for Vinayakumar’s careful review. bq. I didn't understand

[jira] [Commented] (HDFS-7782) Erasure coding: pread from files in striped layout

2015-04-02 Thread Walter Su (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-7782?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14392439#comment-14392439 ] Walter Su commented on HDFS-7782: - The logic in 005 patch looks good. Great work! bq. It

[jira] [Updated] (HDFS-7999) FsDatasetImpl#createTemporary sometimes holds the FSDatasetImpl lock for a very long time

2015-04-02 Thread zhouyingchao (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-7999?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] zhouyingchao updated HDFS-7999: --- Attachment: HDFS-7999-002.patch Test with

[jira] [Commented] (HDFS-7980) Incremental BlockReport will dramatically slow down the startup of a namenode

2015-04-02 Thread Tsz Wo Nicholas Sze (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-7980?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14393542#comment-14393542 ] Tsz Wo Nicholas Sze commented on HDFS-7980: --- ... But Mockito doesn't support

[jira] [Commented] (HDFS-7811) Avoid recursive call getStoragePolicyID in INodeFile#computeQuotaUsage

2015-04-02 Thread Tsz Wo Nicholas Sze (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-7811?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14393605#comment-14393605 ] Tsz Wo Nicholas Sze commented on HDFS-7811: --- - INode.getStoragePolicyID(byte)

[jira] [Created] (HDFS-8051) FsVolumeList#addVolume should release volume reference if not put it into BlockScanner.

2015-04-02 Thread Lei (Eddy) Xu (JIRA)
Lei (Eddy) Xu created HDFS-8051: --- Summary: FsVolumeList#addVolume should release volume reference if not put it into BlockScanner. Key: HDFS-8051 URL: https://issues.apache.org/jira/browse/HDFS-8051

[jira] [Updated] (HDFS-8051) FsVolumeList#addVolume should release volume reference if not put it into BlockScanner.

2015-04-02 Thread Lei (Eddy) Xu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8051?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lei (Eddy) Xu updated HDFS-8051: Target Version/s: 3.0.0, 2.7.0 (was: 3.0.0) Status: Patch Available (was: Open)

[jira] [Updated] (HDFS-7839) Erasure coding: implement facilities in NameNode to create and manage EC zones

2015-04-02 Thread Zhe Zhang (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-7839?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhe Zhang updated HDFS-7839: Attachment: HDFS-7839-003.patch Thanks Jing for the review! The new patch addresses most of the comments.

[jira] [Commented] (HDFS-7839) Erasure coding: implement facilities in NameNode to create and manage EC zones

2015-04-02 Thread Zhe Zhang (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-7839?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14393558#comment-14393558 ] Zhe Zhang commented on HDFS-7839: - Thanks Kai for raising the question regarding naming

[jira] [Updated] (HDFS-6666) Abort NameNode and DataNode startup if security is enabled but block access token is not enabled.

2015-04-02 Thread Vijay Bhat (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vijay Bhat updated HDFS-: - Attachment: (was: HDFS-.001.patch) Abort NameNode and DataNode startup if security is enabled but

[jira] [Commented] (HDFS-5241) Provide alternate queuing audit logger to reduce logging contention

2015-04-02 Thread Arpit Agarwal (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-5241?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14393583#comment-14393583 ] Arpit Agarwal commented on HDFS-5241: - Hi [~daryn], do you consider this feature stable

[jira] [Commented] (HDFS-7782) Erasure coding: pread from files in striped layout

2015-04-02 Thread Jing Zhao (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-7782?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14393263#comment-14393263 ] Jing Zhao commented on HDFS-7782: - Thanks for updating the patch, Zhe! Comments on the 005

[jira] [Commented] (HDFS-8046) Allow better control of getContentSummary

2015-04-02 Thread Hadoop QA (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8046?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14393448#comment-14393448 ] Hadoop QA commented on HDFS-8046: - {color:red}-1 overall{color}. Here are the results of

[jira] [Updated] (HDFS-8051) FsVolumeList#addVolume should release volume reference if not put it into BlockScanner.

2015-04-02 Thread Lei (Eddy) Xu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8051?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lei (Eddy) Xu updated HDFS-8051: Attachment: HDFS-8051.000.patch This patch releases {{FsVolumeReference}} in

[jira] [Updated] (HDFS-8035) Move checkBlocksProperlyReplicated() in FSNamesystem to BlockManager

2015-04-02 Thread Tsz Wo Nicholas Sze (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8035?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tsz Wo Nicholas Sze updated HDFS-8035: -- Component/s: namenode Priority: Minor (was: Major) Move

[jira] [Commented] (HDFS-8041) Consider remaining space during block blockplacement if dfs space is highly utilized

2015-04-02 Thread Nathan Roberts (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8041?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14393437#comment-14393437 ] Nathan Roberts commented on HDFS-8041: -- Hi [~kihwal]. Some minor comments on the patch

[jira] [Commented] (HDFS-7893) Update the POM to create a separate hdfs-client jar

2015-04-02 Thread Hadoop QA (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-7893?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14393562#comment-14393562 ] Hadoop QA commented on HDFS-7893: - {color:red}-1 overall{color}. Here are the results of

[jira] [Updated] (HDFS-7811) Avoid recursive call getStoragePolicyID in INodeFile#computeQuotaUsage

2015-04-02 Thread Tsz Wo Nicholas Sze (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-7811?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tsz Wo Nicholas Sze updated HDFS-7811: -- Component/s: (was: datanode) Priority: Minor (was: Major) Hadoop Flags:

[jira] [Commented] (HDFS-7839) Erasure coding: implement facilities in NameNode to create and manage EC zones

2015-04-02 Thread Kai Zheng (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-7839?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14393794#comment-14393794 ] Kai Zheng commented on HDFS-7839: - Thanks [~jingzhao] and [~zhz] for the clarifying. Given

[jira] [Updated] (HDFS-7782) Erasure coding: pread from files in striped layout

2015-04-02 Thread Zhe Zhang (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-7782?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhe Zhang updated HDFS-7782: Attachment: HDFS-7782-006.patch Thanks Jing and Walter for the thorough reviews! bq.

[jira] [Created] (HDFS-8052) Move WebHdfsFileSystem into hadoop-hdfs-client

2015-04-02 Thread Haohui Mai (JIRA)
Haohui Mai created HDFS-8052: Summary: Move WebHdfsFileSystem into hadoop-hdfs-client Key: HDFS-8052 URL: https://issues.apache.org/jira/browse/HDFS-8052 Project: Hadoop HDFS Issue Type:

[jira] [Commented] (HDFS-7859) Erasure Coding: Persist EC schemas in NameNode

2015-04-02 Thread Xinwei Qin (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-7859?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14393895#comment-14393895 ] Xinwei Qin commented on HDFS-7859: --- Hi, [~drankye], I'm interested in this issue, if you

[jira] [Commented] (HDFS-7782) Erasure coding: pread from files in striped layout

2015-04-02 Thread Walter Su (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-7782?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14393987#comment-14393987 ] Walter Su commented on HDFS-7782: - reviewed. The logic of {{fetchBlockByteRange}} looks

[jira] [Created] (HDFS-8056) Decommissioned dead nodes should continue to be counted as dead after NN restart

2015-04-02 Thread Ming Ma (JIRA)
Ming Ma created HDFS-8056: - Summary: Decommissioned dead nodes should continue to be counted as dead after NN restart Key: HDFS-8056 URL: https://issues.apache.org/jira/browse/HDFS-8056 Project: Hadoop HDFS

[jira] [Updated] (HDFS-7811) Avoid recursive call getStoragePolicyID in INodeFile#computeQuotaUsage

2015-04-02 Thread Jing Zhao (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-7811?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jing Zhao updated HDFS-7811: Attachment: HDFS-7811.04.patch Thanks for the review, Xiaoyu and Nicholas! Update the patch to address

[jira] [Created] (HDFS-8053) Move DFSOutputStream and related classes to hadoop-hdfs-client

2015-04-02 Thread Haohui Mai (JIRA)
Haohui Mai created HDFS-8053: Summary: Move DFSOutputStream and related classes to hadoop-hdfs-client Key: HDFS-8053 URL: https://issues.apache.org/jira/browse/HDFS-8053 Project: Hadoop HDFS

[jira] [Commented] (HDFS-7893) Update the POM to create a separate hdfs-client jar

2015-04-02 Thread Hudson (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-7893?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14393674#comment-14393674 ] Hudson commented on HDFS-7893: -- FAILURE: Integrated in Hadoop-trunk-Commit #7498 (See

[jira] [Commented] (HDFS-7859) Erasure Coding: Persist EC schemas in NameNode

2015-04-02 Thread Kai Zheng (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-7859?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14393901#comment-14393901 ] Kai Zheng commented on HDFS-7859: - Great, please take it. Thanks. Erasure Coding: Persist

[jira] [Assigned] (HDFS-7859) Erasure Coding: Persist EC schemas in NameNode

2015-04-02 Thread Xinwei Qin (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-7859?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xinwei Qin reassigned HDFS-7859: - Assignee: Xinwei Qin (was: Kai Zheng) Erasure Coding: Persist EC schemas in NameNode

[jira] [Commented] (HDFS-7663) Erasure Coding: support Append on EC file

2015-04-02 Thread Yong Zhang (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-7663?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14393940#comment-14393940 ] Yong Zhang commented on HDFS-7663: -- Hi Jing Zhao, I want to work on this JIRA. Could you

[jira] [Commented] (HDFS-7715) Implement the Hitchhiker erasure coding algorithm

2015-04-02 Thread jack liuquan (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-7715?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14393991#comment-14393991 ] jack liuquan commented on HDFS-7715: Hi Kai, As we all know, the advantage of

[jira] [Updated] (HDFS-7893) Update the POM to create a separate hdfs-client jar

2015-04-02 Thread Haohui Mai (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-7893?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Haohui Mai updated HDFS-7893: - Resolution: Fixed Fix Version/s: 2.8.0 Status: Resolved (was: Patch Available) I've

[jira] [Commented] (HDFS-7621) Erasure Coding: update the Balancer/Mover data migration logic

2015-04-02 Thread Walter Su (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-7621?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14393882#comment-14393882 ] Walter Su commented on HDFS-7621: - Hi, [~jingzhao]. I'm interested in this jira. I think

[jira] [Commented] (HDFS-6666) Abort NameNode and DataNode startup if security is enabled but block access token is not enabled.

2015-04-02 Thread Hadoop QA (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14393929#comment-14393929 ] Hadoop QA commented on HDFS-: - {color:green}+1 overall{color}. Here are the results of

[jira] [Created] (HDFS-8054) Move DFSInputStream and related classes to hadoop-hdfs-client

2015-04-02 Thread Haohui Mai (JIRA)
Haohui Mai created HDFS-8054: Summary: Move DFSInputStream and related classes to hadoop-hdfs-client Key: HDFS-8054 URL: https://issues.apache.org/jira/browse/HDFS-8054 Project: Hadoop HDFS

[jira] [Commented] (HDFS-8051) FsVolumeList#addVolume should release volume reference if not put it into BlockScanner.

2015-04-02 Thread Hadoop QA (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8051?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14393687#comment-14393687 ] Hadoop QA commented on HDFS-8051: - {color:red}-1 overall{color}. Here are the results of

[jira] [Created] (HDFS-8055) NullPointerException when topology script is missing.

2015-04-02 Thread Anu Engineer (JIRA)
Anu Engineer created HDFS-8055: -- Summary: NullPointerException when topology script is missing. Key: HDFS-8055 URL: https://issues.apache.org/jira/browse/HDFS-8055 Project: Hadoop HDFS Issue

[jira] [Commented] (HDFS-7811) Avoid recursive call getStoragePolicyID in INodeFile#computeQuotaUsage

2015-04-02 Thread Hadoop QA (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-7811?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14393972#comment-14393972 ] Hadoop QA commented on HDFS-7811: - {color:red}-1 overall{color}. Here are the results of

[jira] [Created] (HDFS-8057) Move BlockReader implementation to the client implementation package

2015-04-02 Thread Tsz Wo Nicholas Sze (JIRA)
Tsz Wo Nicholas Sze created HDFS-8057: - Summary: Move BlockReader implementation to the client implementation package Key: HDFS-8057 URL: https://issues.apache.org/jira/browse/HDFS-8057 Project:

[jira] [Updated] (HDFS-8056) Decommissioned dead nodes should continue to be counted as dead after NN restart

2015-04-02 Thread Ming Ma (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8056?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ming Ma updated HDFS-8056: -- Attachment: HDFS-8056.patch Here is the initial patch. It put the dead node under (dead, decommissioned) after

[jira] [Updated] (HDFS-8056) Decommissioned dead nodes should continue to be counted as dead after NN restart

2015-04-02 Thread Ming Ma (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8056?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ming Ma updated HDFS-8056: -- Assignee: Ming Ma Status: Patch Available (was: Open) Decommissioned dead nodes should continue to be

[jira] [Updated] (HDFS-7811) Avoid recursive call getStoragePolicyID in INodeFile#computeQuotaUsage

2015-04-02 Thread Jing Zhao (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-7811?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jing Zhao updated HDFS-7811: Resolution: Fixed Fix Version/s: 2.7.0 Status: Resolved (was: Patch Available) I've

[jira] [Commented] (HDFS-7782) Erasure coding: pread from files in striped layout

2015-04-02 Thread Jing Zhao (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-7782?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14394101#comment-14394101 ] Jing Zhao commented on HDFS-7782: - The 006 patch looks good to me overall. One issue I

[jira] [Commented] (HDFS-7949) WebImageViewer need support file size calculation with striped blocks

2015-04-02 Thread Zhe Zhang (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-7949?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14394100#comment-14394100 ] Zhe Zhang commented on HDFS-7949: - Thanks Rakesh for working on this! The below

[jira] [Commented] (HDFS-7839) Erasure coding: implement facilities in NameNode to create and manage EC zones

2015-04-02 Thread Jing Zhao (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-7839?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14394045#comment-14394045 ] Jing Zhao commented on HDFS-7839: - Thanks Zhe for updating the patch. +1 Erasure coding:

[jira] [Updated] (HDFS-7839) Erasure coding: implement facilities in NameNode to create and manage EC zones

2015-04-02 Thread Jing Zhao (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-7839?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jing Zhao updated HDFS-7839: Hadoop Flags: Reviewed Erasure coding: implement facilities in NameNode to create and manage EC zones

[jira] [Updated] (HDFS-8025) Addendum for HDFS-3087

2015-04-02 Thread Ming Ma (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8025?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ming Ma updated HDFS-8025: -- Attachment: HDFS-8025-2.patch Thanks Andrew. Here is the updated patch with your suggestions. Addendum for

[jira] [Commented] (HDFS-7811) Avoid recursive call getStoragePolicyID in INodeFile#computeQuotaUsage

2015-04-02 Thread Hudson (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-7811?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14394042#comment-14394042 ] Hudson commented on HDFS-7811: -- FAILURE: Integrated in Hadoop-trunk-Commit #7503 (See

[jira] [Commented] (HDFS-7839) Erasure coding: implement facilities in NameNode to create and manage EC zones

2015-04-02 Thread Zhe Zhang (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-7839?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14394090#comment-14394090 ] Zhe Zhang commented on HDFS-7839: - Thanks Jing and Kai for reviewing! I just committed it

[jira] [Resolved] (HDFS-7839) Erasure coding: implement facilities in NameNode to create and manage EC zones

2015-04-02 Thread Zhe Zhang (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-7839?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhe Zhang resolved HDFS-7839. - Resolution: Fixed Fix Version/s: HDFS-7285 Erasure coding: implement facilities in NameNode to

[jira] [Commented] (HDFS-7889) Subclass DFSOutputStream to support writing striping layout files

2015-04-02 Thread Zhe Zhang (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-7889?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14394095#comment-14394095 ] Zhe Zhang commented on HDFS-7889: - {{writeChunkImpl}} was removed from {{DFSOutputStream}}

[jira] [Commented] (HDFS-8049) Annotation client implementation as private

2015-04-02 Thread Takuya Fukudome (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8049?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14394066#comment-14394066 ] Takuya Fukudome commented on HDFS-8049: --- Hi Nicholas, I am interested in working on

[jira] [Commented] (HDFS-7715) Implement the Hitchhiker erasure coding algorithm

2015-04-02 Thread Kai Zheng (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-7715?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14394077#comment-14394077 ] Kai Zheng commented on HDFS-7715: - Hi Jack, It's not easy for others to follow if we

[jira] [Commented] (HDFS-8056) Decommissioned dead nodes should continue to be counted as dead after NN restart

2015-04-02 Thread Hadoop QA (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8056?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14394076#comment-14394076 ] Hadoop QA commented on HDFS-8056: - {color:red}-1 overall{color}. Here are the results of

[jira] [Commented] (HDFS-7621) Erasure Coding: update the Balancer/Mover data migration logic

2015-04-02 Thread Jing Zhao (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-7621?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14394028#comment-14394028 ] Jing Zhao commented on HDFS-7621: - Sure. Assign the jira to you. Thanks for working on

[jira] [Updated] (HDFS-7621) Erasure Coding: update the Balancer/Mover data migration logic

2015-04-02 Thread Jing Zhao (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-7621?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jing Zhao updated HDFS-7621: Assignee: Walter Su (was: Jing Zhao) Erasure Coding: update the Balancer/Mover data migration logic

[jira] [Commented] (HDFS-7949) WebImageViewer need support file size calculation with striped blocks

2015-04-02 Thread Zhe Zhang (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-7949?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14394104#comment-14394104 ] Zhe Zhang commented on HDFS-7949: - Sorry I just note the calculation is from

[jira] [Commented] (HDFS-8035) Move checkBlocksProperlyReplicated() in FSNamesystem to BlockManager

2015-04-02 Thread Hudson (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8035?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14393176#comment-14393176 ] Hudson commented on HDFS-8035: -- FAILURE: Integrated in Hadoop-trunk-Commit #7495 (See

[jira] [Updated] (HDFS-7954) TestBalancer#testBalancerWithPinnedBlocks failed on Windows

2015-04-02 Thread Tsz Wo Nicholas Sze (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-7954?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tsz Wo Nicholas Sze updated HDFS-7954: -- Priority: Minor (was: Major) Hadoop Flags: Reviewed +1 patch looks good.

[jira] [Updated] (HDFS-8045) Incorrect calculation of NonDfsUsed and Remaining

2015-04-02 Thread zhouyingchao (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8045?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] zhouyingchao updated HDFS-8045: --- Attachment: HDFS-8045-001.patch Test with

[jira] [Commented] (HDFS-7922) ShortCircuitCache#close is not releasing ScheduledThreadPoolExecutors

2015-04-02 Thread Hudson (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-7922?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14392596#comment-14392596 ] Hudson commented on HDFS-7922: -- FAILURE: Integrated in Hadoop-Yarn-trunk-Java8 #151 (See

[jira] [Updated] (HDFS-8045) Incorrect calculation of NonDfsUsed and Remaining

2015-04-02 Thread zhouyingchao (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8045?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] zhouyingchao updated HDFS-8045: --- Description: After reserve some space via the param dfs.datanode.du.reserved, we noticed that the

[jira] [Updated] (HDFS-8045) Incorrect calculation of NonDfsUsed and Remaining

2015-04-02 Thread Brahma Reddy Battula (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8045?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Brahma Reddy Battula updated HDFS-8045: --- Resolution: Duplicate Status: Resolved (was: Patch Available) Incorrect

[jira] [Updated] (HDFS-8045) Incorrect calculation of NonDfsUsed and Remaining

2015-04-02 Thread zhouyingchao (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8045?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] zhouyingchao updated HDFS-8045: --- Description: After reserve some space via the param dfs.datanode.du.reserved, we noticed that the

[jira] [Commented] (HDFS-5215) dfs.datanode.du.reserved is not taking effect as it's not considered while getting the available space

2015-04-02 Thread Brahma Reddy Battula (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-5215?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14392614#comment-14392614 ] Brahma Reddy Battula commented on HDFS-5215: Thanks for Review,, I will update

[jira] [Commented] (HDFS-7715) Implement the Hitchhiker erasure coding algorithm

2015-04-02 Thread jack liuquan (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-7715?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14392629#comment-14392629 ] jack liuquan commented on HDFS-7715: Hi Kai, OK, I have seen your test result in

[jira] [Commented] (HDFS-7916) 'reportBadBlocks' from datanodes to standby Node BPServiceActor goes for infinite loop

2015-04-02 Thread Hudson (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-7916?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14392653#comment-14392653 ] Hudson commented on HDFS-7916: -- FAILURE: Integrated in Hadoop-Hdfs-trunk #2083 (See

[jira] [Commented] (HDFS-8045) Incorrect calculation of NonDfsUsed and Remaining

2015-04-02 Thread Brahma Reddy Battula (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8045?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14392606#comment-14392606 ] Brahma Reddy Battula commented on HDFS-8045: This is dupe of HDFS-5215,, Please

[jira] [Commented] (HDFS-8008) Support client-side back off when the datanodes are congested

2015-04-02 Thread Hudson (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8008?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14392648#comment-14392648 ] Hudson commented on HDFS-8008: -- FAILURE: Integrated in Hadoop-Hdfs-trunk #2083 (See

[jira] [Commented] (HDFS-7978) Add LOG.isDebugEnabled() guard for some LOG.debug(..)

2015-04-02 Thread Hudson (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-7978?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14392652#comment-14392652 ] Hudson commented on HDFS-7978: -- FAILURE: Integrated in Hadoop-Hdfs-trunk #2083 (See

[jira] [Commented] (HDFS-7060) Avoid taking locks when sending heartbeats from the DataNode

2015-04-02 Thread Xinwei Qin (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-7060?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14392672#comment-14392672 ] Xinwei Qin commented on HDFS-7060: --- [~cmccabe] , [~wheat9] Thanks a lot for your

[jira] [Commented] (HDFS-8009) Signal congestion on the DataNode

2015-04-02 Thread Hudson (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8009?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14392650#comment-14392650 ] Hudson commented on HDFS-8009: -- FAILURE: Integrated in Hadoop-Hdfs-trunk #2083 (See

[jira] [Commented] (HDFS-7922) ShortCircuitCache#close is not releasing ScheduledThreadPoolExecutors

2015-04-02 Thread Hudson (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-7922?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14392657#comment-14392657 ] Hudson commented on HDFS-7922: -- FAILURE: Integrated in Hadoop-Hdfs-trunk #2083 (See

[jira] [Updated] (HDFS-8041) Consider remaining space during block blockplacement if dfs space is highly utilized

2015-04-02 Thread Kihwal Lee (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8041?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kihwal Lee updated HDFS-8041: - Attachment: HDFS-8041.v2.patch Consider remaining space during block blockplacement if dfs space is

[jira] [Commented] (HDFS-8026) Trace FSOutputSummer#writeChecksumChunks rather than DFSOutputStream#writeChunk

2015-04-02 Thread Hudson (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8026?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14392649#comment-14392649 ] Hudson commented on HDFS-8026: -- FAILURE: Integrated in Hadoop-Hdfs-trunk #2083 (See

[jira] [Commented] (HDFS-8001) RpcProgramNfs3 : wrong parsing of dfs.blocksize

2015-04-02 Thread Hudson (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8001?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14392655#comment-14392655 ] Hudson commented on HDFS-8001: -- FAILURE: Integrated in Hadoop-Hdfs-trunk #2083 (See

[jira] [Comment Edited] (HDFS-8041) Consider remaining space during block blockplacement if dfs space is highly utilized

2015-04-02 Thread Kihwal Lee (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8041?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14392663#comment-14392663 ] Kihwal Lee edited comment on HDFS-8041 at 4/2/15 12:47 PM: --- bq.

[jira] [Commented] (HDFS-8041) Consider remaining space during block blockplacement if dfs space is highly utilized

2015-04-02 Thread Kihwal Lee (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8041?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14392663#comment-14392663 ] Kihwal Lee commented on HDFS-8041: -- bq. -1 findbugs. The patch appears to introduce 1 new

[jira] [Commented] (HDFS-8036) Use snapshot path as source when using snapshot diff report in DistCp

2015-04-02 Thread Hudson (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8036?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14392659#comment-14392659 ] Hudson commented on HDFS-8036: -- FAILURE: Integrated in Hadoop-Hdfs-trunk #2083 (See

[jira] [Commented] (HDFS-8001) RpcProgramNfs3 : wrong parsing of dfs.blocksize

2015-04-02 Thread Hudson (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8001?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14392870#comment-14392870 ] Hudson commented on HDFS-8001: -- FAILURE: Integrated in Hadoop-Mapreduce-trunk-Java8 #151 (See

[jira] [Commented] (HDFS-7916) 'reportBadBlocks' from datanodes to standby Node BPServiceActor goes for infinite loop

2015-04-02 Thread Hudson (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-7916?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14392868#comment-14392868 ] Hudson commented on HDFS-7916: -- FAILURE: Integrated in Hadoop-Mapreduce-trunk-Java8 #151 (See

[jira] [Resolved] (HDFS-3449) TestFileCreation fails

2015-04-02 Thread Tsz Wo Nicholas Sze (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-3449?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tsz Wo Nicholas Sze resolved HDFS-3449. --- Resolution: Duplicate Thanks. Resolving this as a duplicate of HDFS-3966.

[jira] [Commented] (HDFS-8045) Incorrect calculation of NonDfsUsed and Remaining

2015-04-02 Thread Hadoop QA (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8045?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14392845#comment-14392845 ] Hadoop QA commented on HDFS-8045: - {color:red}-1 overall{color}. Here are the results of

  1   2   >