[jira] [Commented] (HDFS-8846) Create edit log files with old layout version for upgrade testing

2015-08-17 Thread Hadoop QA (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8846?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14700815#comment-14700815 ] Hadoop QA commented on HDFS-8846: - \\ \\ | (x) *{color:red}-1 overall{color}* | \\ \\ || Vo

[jira] [Commented] (HDFS-8852) HDFS architecture documentation of version 2.x is outdated about append write support

2015-08-17 Thread Ajith S (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8852?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14700792#comment-14700792 ] Ajith S commented on HDFS-8852: --- Thanks for the input [~ajisakaa] Uploaded the new patch as

[jira] [Updated] (HDFS-8852) HDFS architecture documentation of version 2.x is outdated about append write support

2015-08-17 Thread Ajith S (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8852?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ajith S updated HDFS-8852: -- Attachment: HDFS-8852.2.patch > HDFS architecture documentation of version 2.x is outdated about append write >

[jira] [Commented] (HDFS-8909) Erasure coding: update BlockInfoContiguousUC and BlockInfoStripedUC to use BlockUnderConstructionFeature

2015-08-17 Thread Zhe Zhang (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8909?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14700786#comment-14700786 ] Zhe Zhang commented on HDFS-8909: - Thanks for the discussions folks. I'm OK with both git

[jira] [Commented] (HDFS-8833) Erasure coding: store EC schema and cell size in INodeFile and eliminate notion of EC zones

2015-08-17 Thread Hadoop QA (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8833?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14700751#comment-14700751 ] Hadoop QA commented on HDFS-8833: - \\ \\ | (x) *{color:red}-1 overall{color}* | \\ \\ || Vo

[jira] [Commented] (HDFS-8846) Create edit log files with old layout version for upgrade testing

2015-08-17 Thread Hadoop QA (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8846?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14700746#comment-14700746 ] Hadoop QA commented on HDFS-8846: - \\ \\ | (x) *{color:red}-1 overall{color}* | \\ \\ || Vo

[jira] [Commented] (HDFS-8823) Move replication factor into individual blocks

2015-08-17 Thread Hadoop QA (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8823?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14700639#comment-14700639 ] Hadoop QA commented on HDFS-8823: - \\ \\ | (x) *{color:red}-1 overall{color}* | \\ \\ || Vo

[jira] [Updated] (HDFS-8912) Implement ShrinkableHashMap extends java HashMap and use properly

2015-08-17 Thread Yi Liu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8912?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yi Liu updated HDFS-8912: - Description: Currently {{LightWeightHashSet}} and {{LightWeightLinkedSet}} are used in hdfs, there are two advanta

[jira] [Commented] (HDFS-8912) Implement ShrinkableHashMap extends java HashMap and use properly

2015-08-17 Thread Yi Liu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8912?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14700620#comment-14700620 ] Yi Liu commented on HDFS-8912: -- Hi [~cmccabe], how do you think about it? Thanks. > Implement

[jira] [Created] (HDFS-8912) Implement ShrinkableHashMap extends java HashMap and use properly

2015-08-17 Thread Yi Liu (JIRA)
Yi Liu created HDFS-8912: Summary: Implement ShrinkableHashMap extends java HashMap and use properly Key: HDFS-8912 URL: https://issues.apache.org/jira/browse/HDFS-8912 Project: Hadoop HDFS Issue Ty

[jira] [Commented] (HDFS-8909) Erasure coding: update BlockInfoContiguousUC and BlockInfoStripedUC to use BlockUnderConstructionFeature

2015-08-17 Thread Walter Su (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8909?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14700610#comment-14700610 ] Walter Su commented on HDFS-8909: - bq. Instead of keeping doing git rebase, maybe we should

[jira] [Commented] (HDFS-8862) BlockManager#excessReplicateMap should use a HashMap

2015-08-17 Thread Hudson (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8862?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14700603#comment-14700603 ] Hudson commented on HDFS-8862: -- FAILURE: Integrated in Hadoop-trunk-Commit #8315 (See [https:

[jira] [Commented] (HDFS-8895) Remove deprecated BlockStorageLocation APIs

2015-08-17 Thread Hudson (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8895?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14700605#comment-14700605 ] Hudson commented on HDFS-8895: -- FAILURE: Integrated in Hadoop-trunk-Commit #8315 (See [https:

[jira] [Commented] (HDFS-8845) DiskChecker should not traverse the entire tree

2015-08-17 Thread Hudson (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8845?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14700604#comment-14700604 ] Hudson commented on HDFS-8845: -- FAILURE: Integrated in Hadoop-trunk-Commit #8315 (See [https:

[jira] [Commented] (HDFS-8880) NameNode metrics logging

2015-08-17 Thread Hudson (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8880?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14700606#comment-14700606 ] Hudson commented on HDFS-8880: -- FAILURE: Integrated in Hadoop-trunk-Commit #8315 (See [https:

[jira] [Commented] (HDFS-8278) HDFS Balancer should consider remaining storage % when checking for under-utilized machines

2015-08-17 Thread Hudson (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8278?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14700607#comment-14700607 ] Hudson commented on HDFS-8278: -- FAILURE: Integrated in Hadoop-trunk-Commit #8315 (See [https:

[jira] [Updated] (HDFS-8278) HDFS Balancer should consider remaining storage % when checking for under-utilized machines

2015-08-17 Thread Tsz Wo Nicholas Sze (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8278?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tsz Wo Nicholas Sze updated HDFS-8278: -- Resolution: Fixed Hadoop Flags: Reviewed Fix Version/s: 2.8.0 Statu

[jira] [Commented] (HDFS-8909) Erasure coding: update BlockInfoContiguousUC and BlockInfoStripedUC to use BlockUnderConstructionFeature

2015-08-17 Thread Jing Zhao (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8909?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14700589#comment-14700589 ] Jing Zhao commented on HDFS-8909: - Instead of keeping doing git rebase, maybe we should swi

[jira] [Comment Edited] (HDFS-8862) BlockManager#excessReplicateMap should use a HashMap

2015-08-17 Thread Yi Liu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8862?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14700577#comment-14700577 ] Yi Liu edited comment on HDFS-8862 at 8/18/15 1:39 AM: --- One more disc

[jira] [Commented] (HDFS-8826) Balancer may not move blocks efficiently in some cases

2015-08-17 Thread Arpit Agarwal (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8826?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14700576#comment-14700576 ] Arpit Agarwal commented on HDFS-8826: - Also it may be a good idea to add a separate opt

[jira] [Commented] (HDFS-8862) BlockManager#excessReplicateMap should use a HashMap

2015-08-17 Thread Yi Liu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8862?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14700577#comment-14700577 ] Yi Liu commented on HDFS-8862: -- One more discussion, do you think it's worth to extend java Ha

[jira] [Commented] (HDFS-8826) Balancer may not move blocks efficiently in some cases

2015-08-17 Thread Arpit Agarwal (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8826?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14700575#comment-14700575 ] Arpit Agarwal commented on HDFS-8826: - +1 for the patch. The test failures look unrelat

[jira] [Updated] (HDFS-8862) BlockManager#excessReplicateMap should use a HashMap

2015-08-17 Thread Yi Liu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8862?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yi Liu updated HDFS-8862: - Resolution: Fixed Hadoop Flags: Reviewed Fix Version/s: 2.8.0 Status: Resolved (was: Patch A

[jira] [Updated] (HDFS-8862) BlockManager#excessReplicateMap should use a HashMap

2015-08-17 Thread Yi Liu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8862?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yi Liu updated HDFS-8862: - Summary: BlockManager#excessReplicateMap should use a HashMap (was: Improve BlockManager#excessReplicateMap) > Bl

[jira] [Commented] (HDFS-8862) Improve BlockManager#excessReplicateMap

2015-08-17 Thread Yi Liu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8862?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14700560#comment-14700560 ] Yi Liu commented on HDFS-8862: -- Thanks [~cmccabe] for the review. Will commit the patch later.

[jira] [Commented] (HDFS-8909) Erasure coding: update BlockInfoContiguousUC and BlockInfoStripedUC to use BlockUnderConstructionFeature

2015-08-17 Thread Walter Su (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8909?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14700556#comment-14700556 ] Walter Su commented on HDFS-8909: - I'm afraid the patch makes the earlier branch commits un

[jira] [Commented] (HDFS-8792) BlockManager#postponedMisreplicatedBlocks should use a LightWeightHashSet to save memory

2015-08-17 Thread Yi Liu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8792?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14700543#comment-14700543 ] Yi Liu commented on HDFS-8792: -- Thanks [~cmccabe] for the review and commit! > BlockManager#p

[jira] [Commented] (HDFS-6955) DN should reserve disk space for a full block when creating tmp files

2015-08-17 Thread Hadoop QA (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-6955?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14700530#comment-14700530 ] Hadoop QA commented on HDFS-6955: - \\ \\ | (x) *{color:red}-1 overall{color}* | \\ \\ || Vo

[jira] [Updated] (HDFS-8895) Remove deprecated BlockStorageLocation APIs

2015-08-17 Thread Andrew Wang (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8895?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Wang updated HDFS-8895: -- Resolution: Fixed Fix Version/s: 3.0.0 Release Note: This removes the deprecated DistributedF

[jira] [Commented] (HDFS-8895) Remove deprecated BlockStorageLocation APIs

2015-08-17 Thread Andrew Wang (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8895?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14700527#comment-14700527 ] Andrew Wang commented on HDFS-8895: --- Quite some flakes but they all passed for me locally

[jira] [Commented] (HDFS-8278) HDFS Balancer should consider remaining storage % when checking for under-utilized machines

2015-08-17 Thread Hadoop QA (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8278?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14700522#comment-14700522 ] Hadoop QA commented on HDFS-8278: - \\ \\ | (x) *{color:red}-1 overall{color}* | \\ \\ || Vo

[jira] [Updated] (HDFS-8435) createNonRecursive support needed in WebHdfsFileSystem to support HBase

2015-08-17 Thread Jakob Homan (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8435?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jakob Homan updated HDFS-8435: -- Status: Patch Available (was: Open) > createNonRecursive support needed in WebHdfsFileSystem to support

[jira] [Updated] (HDFS-8435) createNonRecursive support needed in WebHdfsFileSystem to support HBase

2015-08-17 Thread Jakob Homan (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8435?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jakob Homan updated HDFS-8435: -- Attachment: HDFS-8435.004.patch Fixed javadoc and whitespace complaints. Unfortunately, as we're adding

[jira] [Updated] (HDFS-8435) createNonRecursive support needed in WebHdfsFileSystem to support HBase

2015-08-17 Thread Jakob Homan (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8435?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jakob Homan updated HDFS-8435: -- Status: Open (was: Patch Available) > createNonRecursive support needed in WebHdfsFileSystem to support

[jira] [Updated] (HDFS-8911) NameNode Metric : Add Editlog counters as a JMX metric

2015-08-17 Thread Anu Engineer (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8911?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Anu Engineer updated HDFS-8911: --- Summary: NameNode Metric : Add Editlog counters as a JMX metric (was: NameNode Metric : Add WAL counte

[jira] [Updated] (HDFS-8911) NameNode Metric : Add Editlog counters as a JMX metric

2015-08-17 Thread Anu Engineer (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8911?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Anu Engineer updated HDFS-8911: --- Description: Today we log editlog metrics in the log. This JIRA proposes to expose those metrics via JM

[jira] [Commented] (HDFS-8911) NameNode Metric : Add WAL counters as a JMX metric

2015-08-17 Thread Anu Engineer (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8911?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14700503#comment-14700503 ] Anu Engineer commented on HDFS-8911: [~andrew.wang] Thanks for letting me know. I will

[jira] [Commented] (HDFS-8911) NameNode Metric : Add WAL counters as a JMX metric

2015-08-17 Thread Andrew Wang (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8911?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14700500#comment-14700500 ] Andrew Wang commented on HDFS-8911: --- This is a small correction, but the edit log is writ

[jira] [Updated] (HDFS-8909) Erasure coding: update BlockInfoContiguousUC and BlockInfoStripedUC to use BlockUnderConstructionFeature

2015-08-17 Thread Jing Zhao (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8909?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jing Zhao updated HDFS-8909: Attachment: HDFS-8909.000.patch Patch against HDFS-7285-REBASE branch. > Erasure coding: update BlockInfoCon

[jira] [Updated] (HDFS-8880) NameNode metrics logging

2015-08-17 Thread Arpit Agarwal (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8880?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Arpit Agarwal updated HDFS-8880: Resolution: Fixed Hadoop Flags: Reviewed Fix Version/s: 2.8.0 Target Version/

[jira] [Commented] (HDFS-8911) NameNode Metric : Add WAL counters as a JMX metric

2015-08-17 Thread Arpit Agarwal (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8911?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14700485#comment-14700485 ] Arpit Agarwal commented on HDFS-8911: - +1 pending Jenkins. > NameNode Metric : Add WAL

[jira] [Commented] (HDFS-8909) Erasure coding: update BlockInfoContiguousUC and BlockInfoStripedUC to use BlockUnderConstructionFeature

2015-08-17 Thread Jing Zhao (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8909?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14700481#comment-14700481 ] Jing Zhao commented on HDFS-8909: - I think maybe to save us time from now on we can use "gi

[jira] [Updated] (HDFS-8911) NameNode Metric : Add WAL counters as a JMX metric

2015-08-17 Thread Anu Engineer (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8911?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Anu Engineer updated HDFS-8911: --- Attachment: HDFS-8911.001.patch Adds TotalSyncCount and TotalSyncTimes metrics to NameNodeMetrics > Na

[jira] [Commented] (HDFS-8908) TestAppendSnapshotTruncate may fail with IOException: Failed to replace a bad datanode

2015-08-17 Thread Hadoop QA (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8908?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14700478#comment-14700478 ] Hadoop QA commented on HDFS-8908: - \\ \\ | (x) *{color:red}-1 overall{color}* | \\ \\ || Vo

[jira] [Updated] (HDFS-8911) NameNode Metric : Add WAL counters as a JMX metric

2015-08-17 Thread Anu Engineer (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8911?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Anu Engineer updated HDFS-8911: --- Status: Patch Available (was: Open) > NameNode Metric : Add WAL counters as a JMX metric > ---

[jira] [Commented] (HDFS-8880) NameNode metrics logging

2015-08-17 Thread Jitendra Nath Pandey (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8880?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14700457#comment-14700457 ] Jitendra Nath Pandey commented on HDFS-8880: +1 > NameNode metrics logging > -

[jira] [Commented] (HDFS-8909) Erasure coding: update BlockInfoContiguousUC and BlockInfoStripedUC to use BlockUnderConstructionFeature

2015-08-17 Thread Zhe Zhang (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8909?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14700456#comment-14700456 ] Zhe Zhang commented on HDFS-8909: - [~jingzhao] I just realized we said {{HDFS-7285-rebase}}

[jira] [Commented] (HDFS-8833) Erasure coding: store EC schema and cell size in INodeFile and eliminate notion of EC zones

2015-08-17 Thread Zhe Zhang (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8833?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14700454#comment-14700454 ] Zhe Zhang commented on HDFS-8833: - [~jingzhao] [~walter.k.su] [~andrew.wang] Thanks for the

[jira] [Updated] (HDFS-8833) Erasure coding: store EC schema and cell size in INodeFile and eliminate notion of EC zones

2015-08-17 Thread Zhe Zhang (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8833?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhe Zhang updated HDFS-8833: Attachment: HDFS-8833-HDFS-7285-merge.01.patch Updating the patch to: # Remove all references of {{ErasureCod

[jira] [Commented] (HDFS-8895) Remove deprecated BlockStorageLocation APIs

2015-08-17 Thread Hadoop QA (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8895?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14700380#comment-14700380 ] Hadoop QA commented on HDFS-8895: - \\ \\ | (x) *{color:red}-1 overall{color}* | \\ \\ || Vo

[jira] [Created] (HDFS-8911) NameNode Metric : Add WAL counters as a JMX metric

2015-08-17 Thread Anu Engineer (JIRA)
Anu Engineer created HDFS-8911: -- Summary: NameNode Metric : Add WAL counters as a JMX metric Key: HDFS-8911 URL: https://issues.apache.org/jira/browse/HDFS-8911 Project: Hadoop HDFS Issue Type:

[jira] [Updated] (HDFS-8846) Create edit log files with old layout version for upgrade testing

2015-08-17 Thread Zhe Zhang (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8846?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhe Zhang updated HDFS-8846: Attachment: HDFS-8846.01.patch Thanks for the good catch Colin! Updating the patch with the binary diff. > C

[jira] [Commented] (HDFS-8846) Create edit log files with old layout version for upgrade testing

2015-08-17 Thread Colin Patrick McCabe (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8846?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14700361#comment-14700361 ] Colin Patrick McCabe commented on HDFS-8846: Hi [~zhz], It seems that you left

[jira] [Commented] (HDFS-8845) DiskChecker should not traverse the entire tree

2015-08-17 Thread Andrew Wang (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8845?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14700353#comment-14700353 ] Andrew Wang commented on HDFS-8845: --- Yea sounds good to me too, Colin explained to me off

[jira] [Commented] (HDFS-8862) Improve BlockManager#excessReplicateMap

2015-08-17 Thread Colin Patrick McCabe (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8862?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14700352#comment-14700352 ] Colin Patrick McCabe commented on HDFS-8862: I guess the set of datanodes is no

[jira] [Updated] (HDFS-8845) DiskChecker should not traverse the entire tree

2015-08-17 Thread Colin Patrick McCabe (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8845?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Colin Patrick McCabe updated HDFS-8845: --- Resolution: Fixed Fix Version/s: 2.8.0 Target Version/s: 2.8.0

[jira] [Updated] (HDFS-8845) DiskChecker should not traverse the entire tree

2015-08-17 Thread Colin Patrick McCabe (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8845?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Colin Patrick McCabe updated HDFS-8845: --- Summary: DiskChecker should not traverse the entire tree (was: DiskChecker should not

[jira] [Commented] (HDFS-8792) BlockManager#postponedMisreplicatedBlocks should use a LightWeightHashSet to save memory

2015-08-17 Thread Hudson (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8792?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14700333#comment-14700333 ] Hudson commented on HDFS-8792: -- FAILURE: Integrated in Hadoop-trunk-Commit #8314 (See [https:

[jira] [Commented] (HDFS-8908) TestAppendSnapshotTruncate may fail with IOException: Failed to replace a bad datanode

2015-08-17 Thread Hadoop QA (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8908?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14700325#comment-14700325 ] Hadoop QA commented on HDFS-8908: - \\ \\ | (x) *{color:red}-1 overall{color}* | \\ \\ || Vo

[jira] [Commented] (HDFS-7446) HDFS inotify should have the ability to determine what txid it has read up to

2015-08-17 Thread Colin Patrick McCabe (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-7446?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14700274#comment-14700274 ] Colin Patrick McCabe commented on HDFS-7446: I would like to see this backporte

[jira] [Commented] (HDFS-8910) Downgrade log "Operation category READ is not supported in state..." to DEBUG level.

2015-08-17 Thread Arpit Agarwal (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8910?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14700254#comment-14700254 ] Arpit Agarwal commented on HDFS-8910: - I see, thanks for the pointer [~andrew.wang]. We

[jira] [Updated] (HDFS-8846) Create edit log files with old layout version for upgrade testing

2015-08-17 Thread Zhe Zhang (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8846?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhe Zhang updated HDFS-8846: Attachment: HDFS-8846.00.patch The patch does the following: # Added a packed {{/dfs}} dir. I basically start

[jira] [Updated] (HDFS-8846) Create edit log files with old layout version for upgrade testing

2015-08-17 Thread Zhe Zhang (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8846?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhe Zhang updated HDFS-8846: Status: Patch Available (was: Open) > Create edit log files with old layout version for upgrade testing > --

[jira] [Commented] (HDFS-6244) Make Trash Interval configurable for each of the namespaces

2015-08-17 Thread Siqi Li (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-6244?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14700204#comment-14700204 ] Siqi Li commented on HDFS-6244: --- I think Ming is talking about cloning the conf instead of re

[jira] [Commented] (HDFS-8895) Remove deprecated BlockStorageLocation APIs

2015-08-17 Thread Lei (Eddy) Xu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8895?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14700197#comment-14700197 ] Lei (Eddy) Xu commented on HDFS-8895: - This patch just removes code, LGTM. +1 pending

[jira] [Commented] (HDFS-8910) Downgrade log "Operation category READ is not supported in state..." to DEBUG level.

2015-08-17 Thread Andrew Wang (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8910?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14700189#comment-14700189 ] Andrew Wang commented on HDFS-8910: --- This came up at HDFS-3447 before and [~daryn] was ag

[jira] [Commented] (HDFS-8880) NameNode metrics logging

2015-08-17 Thread Arpit Agarwal (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8880?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14700184#comment-14700184 ] Arpit Agarwal commented on HDFS-8880: - The test failures are unrelated and whitespace i

[jira] [Commented] (HDFS-8880) NameNode metrics logging

2015-08-17 Thread Hadoop QA (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8880?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14700182#comment-14700182 ] Hadoop QA commented on HDFS-8880: - \\ \\ | (x) *{color:red}-1 overall{color}* | \\ \\ || Vo

[jira] [Updated] (HDFS-7822) Make webhdfs handling of URI standard compliant

2015-08-17 Thread Jakob Homan (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-7822?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jakob Homan updated HDFS-7822: -- Summary: Make webhdfs handling of URI standard compliant (was: Make webhdfs handling of URI stardard com

[jira] [Commented] (HDFS-8910) Downgrade log "Operation category READ is not supported in state..." to DEBUG level.

2015-08-17 Thread Arpit Agarwal (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8910?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14700142#comment-14700142 ] Arpit Agarwal commented on HDFS-8910: - Hi [~xiaobingo], thanks for the patch. Can we us

[jira] [Commented] (HDFS-8867) Enable optimized block reports

2015-08-17 Thread Jing Zhao (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8867?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14700135#comment-14700135 ] Jing Zhao commented on HDFS-8867: - Any update here? I can also upload a patch to fix if [~d

[jira] [Commented] (HDFS-8909) Erasure coding: update BlockInfoContiguousUC and BlockInfoStripedUC to use BlockUnderConstructionFeature

2015-08-17 Thread Zhe Zhang (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8909?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14700114#comment-14700114 ] Zhe Zhang commented on HDFS-8909: - Thanks for taking on the task Jing. Please base your pat

[jira] [Updated] (HDFS-8823) Move replication factor into individual blocks

2015-08-17 Thread Haohui Mai (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8823?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Haohui Mai updated HDFS-8823: - Attachment: HDFS-8823.005.patch > Move replication factor into individual blocks >

[jira] [Commented] (HDFS-8829) DataNode sets SO_RCVBUF explicitly is disabling tcp auto-tuning

2015-08-17 Thread Colin Patrick McCabe (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8829?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14700059#comment-14700059 ] Colin Patrick McCabe commented on HDFS-8829: I think users should have the opti

[jira] [Commented] (HDFS-6407) Add sorting and pagination in the datanode tab of the NN Web UI

2015-08-17 Thread Hudson (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-6407?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14700047#comment-14700047 ] Hudson commented on HDFS-6407: -- SUCCESS: Integrated in Hadoop-trunk-Commit #8313 (See [https:

[jira] [Commented] (HDFS-8713) Convert DatanodeDescriptor to use SLF4J logging

2015-08-17 Thread Hudson (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8713?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14700049#comment-14700049 ] Hudson commented on HDFS-8713: -- SUCCESS: Integrated in Hadoop-trunk-Commit #8313 (See [https:

[jira] [Commented] (HDFS-8883) NameNode Metrics : Add FSNameSystem lock Queue Length

2015-08-17 Thread Hudson (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8883?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14700050#comment-14700050 ] Hudson commented on HDFS-8883: -- SUCCESS: Integrated in Hadoop-trunk-Commit #8313 (See [https:

[jira] [Updated] (HDFS-8910) Downgrade log "Operation category READ is not supported in state..." to DEBUG level.

2015-08-17 Thread Xiaobing Zhou (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8910?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiaobing Zhou updated HDFS-8910: Attachment: HDFS-8910.1.patch Made a tentative patch for review. > Downgrade log "Operation category

[jira] [Commented] (HDFS-8801) Convert BlockInfoUnderConstruction as a feature

2015-08-17 Thread Hudson (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8801?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14700048#comment-14700048 ] Hudson commented on HDFS-8801: -- SUCCESS: Integrated in Hadoop-trunk-Commit #8313 (See [https:

[jira] [Updated] (HDFS-8792) BlockManager#postponedMisreplicatedBlocks should use a LightWeightHashSet to save memory

2015-08-17 Thread Colin Patrick McCabe (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8792?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Colin Patrick McCabe updated HDFS-8792: --- Resolution: Fixed Fix Version/s: 2.8.0 Target Version/s: 2.8.0

[jira] [Commented] (HDFS-8909) Erasure coding: update BlockInfoContiguousUC and BlockInfoStripedUC to use BlockUnderConstructionFeature

2015-08-17 Thread Jing Zhao (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8909?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14700043#comment-14700043 ] Jing Zhao commented on HDFS-8909: - Just to confirm, should I merge changes into both HDFS-7

[jira] [Updated] (HDFS-6244) Make Trash Interval configurable for each of the namespaces

2015-08-17 Thread Gera Shegalov (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-6244?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gera Shegalov updated HDFS-6244: Status: Open (was: Patch Available) > Make Trash Interval configurable for each of the namespaces >

[jira] [Commented] (HDFS-8909) Erasure coding: update BlockInfoContiguousUC and BlockInfoStripedUC to use BlockUnderConstructionFeature

2015-08-17 Thread Jing Zhao (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8909?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14700042#comment-14700042 ] Jing Zhao commented on HDFS-8909: - Thanks for filing the jira, Zhe! I will try to merge HDF

[jira] [Assigned] (HDFS-8909) Erasure coding: update BlockInfoContiguousUC and BlockInfoStripedUC to use BlockUnderConstructionFeature

2015-08-17 Thread Jing Zhao (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8909?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jing Zhao reassigned HDFS-8909: --- Assignee: Jing Zhao > Erasure coding: update BlockInfoContiguousUC and BlockInfoStripedUC to use > Bl

[jira] [Created] (HDFS-8910) Downgrade log "Operation category READ is not supported in state..." to DEBUG level.

2015-08-17 Thread Xiaobing Zhou (JIRA)
Xiaobing Zhou created HDFS-8910: --- Summary: Downgrade log "Operation category READ is not supported in state..." to DEBUG level. Key: HDFS-8910 URL: https://issues.apache.org/jira/browse/HDFS-8910 Projec

[jira] [Updated] (HDFS-8792) BlockManager#postponedMisreplicatedBlocks should use a LightWeightHashSet to save memory

2015-08-17 Thread Colin Patrick McCabe (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8792?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Colin Patrick McCabe updated HDFS-8792: --- Summary: BlockManager#postponedMisreplicatedBlocks should use a LightWeightHashSet to s

[jira] [Updated] (HDFS-8792) BlockManager#postponedMisreplicatedBlocks should use a LightWeightHashSet to save memory

2015-08-17 Thread Colin Patrick McCabe (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8792?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Colin Patrick McCabe updated HDFS-8792: --- Description: {{LightWeightHashSet}} requires fewer memory than java hashset. (was: {{

[jira] [Commented] (HDFS-8278) HDFS Balancer should consider remaining storage % when checking for under-utilized machines

2015-08-17 Thread Jing Zhao (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8278?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14700036#comment-14700036 ] Jing Zhao commented on HDFS-8278: - +1 pending Jenkins. > HDFS Balancer should consider rem

[jira] [Commented] (HDFS-8792) Improve BlockManager#postponedMisreplicatedBlocks

2015-08-17 Thread Colin Patrick McCabe (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8792?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14700031#comment-14700031 ] Colin Patrick McCabe commented on HDFS-8792: Thanks for this, [~hitliuyi]. +1.

[jira] [Commented] (HDFS-7822) Make webhdfs handling of URI stardard compliant

2015-08-17 Thread Allen Wittenauer (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-7822?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14700030#comment-14700030 ] Allen Wittenauer commented on HDFS-7822: I keep meaning to update this JIRA... We

[jira] [Commented] (HDFS-6244) Make Trash Interval configurable for each of the namespaces

2015-08-17 Thread Gera Shegalov (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-6244?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14700029#comment-14700029 ] Gera Shegalov commented on HDFS-6244: - Thanks for the patch, [~l201514]. My thinking is

[jira] [Comment Edited] (HDFS-8845) DiskChecker should not traverse entire tree

2015-08-17 Thread Colin Patrick McCabe (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8845?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14698920#comment-14698920 ] Colin Patrick McCabe edited comment on HDFS-8845 at 8/17/15 6:50 PM:

[jira] [Updated] (HDFS-7822) Make webhdfs handling of URI stardard compliant

2015-08-17 Thread Jakob Homan (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-7822?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jakob Homan updated HDFS-7822: -- Component/s: webhdfs > Make webhdfs handling of URI stardard compliant >

[jira] [Created] (HDFS-8909) Erasure coding: update BlockInfoContiguousUC and BlockInfoStripedUC to use BlockUnderConstructionFeature

2015-08-17 Thread Zhe Zhang (JIRA)
Zhe Zhang created HDFS-8909: --- Summary: Erasure coding: update BlockInfoContiguousUC and BlockInfoStripedUC to use BlockUnderConstructionFeature Key: HDFS-8909 URL: https://issues.apache.org/jira/browse/HDFS-8909

[jira] [Updated] (HDFS-8801) Convert BlockInfoUnderConstruction as a feature

2015-08-17 Thread Haohui Mai (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8801?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Haohui Mai updated HDFS-8801: - Resolution: Fixed Hadoop Flags: Reviewed Fix Version/s: 2.8.0 Status: Resolved (was:

[jira] [Updated] (HDFS-8278) HDFS Balancer should consider remaining storage % when checking for under-utilized machines

2015-08-17 Thread Tsz Wo Nicholas Sze (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8278?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tsz Wo Nicholas Sze updated HDFS-8278: -- Status: Patch Available (was: Open) > HDFS Balancer should consider remaining storage %

[jira] [Commented] (HDFS-8833) Erasure coding: store EC schema and cell size in INodeFile and eliminate notion of EC zones

2015-08-17 Thread Andrew Wang (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8833?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14699965#comment-14699965 ] Andrew Wang commented on HDFS-8833: --- I mentioned a [possible design|https://issues.apach

[jira] [Commented] (HDFS-8828) Utilize Snapshot diff report to build copy list in distcp

2015-08-17 Thread Jing Zhao (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8828?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14699950#comment-14699950 ] Jing Zhao commented on HDFS-8828: - Yes, we can try to use DistCpSync or a new class to carr

[jira] [Updated] (HDFS-6407) Add sorting and pagination in the datanode tab of the NN Web UI

2015-08-17 Thread Haohui Mai (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-6407?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Haohui Mai updated HDFS-6407: - Resolution: Fixed Hadoop Flags: Reviewed Fix Version/s: 2.8.0 Status: Resolved (was:

[jira] [Updated] (HDFS-6407) Add sorting and pagination in the datanode tab of the NN Web UI

2015-08-17 Thread Haohui Mai (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-6407?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Haohui Mai updated HDFS-6407: - Summary: Add sorting and pagination in the datanode tab of the NN Web UI (was: new namenode UI, lost abili

  1   2   >