[jira] [Commented] (HDFS-9530) huge Non-DFS Used in hadoop 2.6.2 & 2.7.1

2016-02-18 Thread Qiuzhuang Lian (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9530?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15153875#comment-15153875 ] Qiuzhuang Lian commented on HDFS-9530: -- we see this problem too in hadoop 2.6.2. The dfs non used is

[jira] [Updated] (HDFS-9816) Erasure Coding: allow to use multiple EC policies in striping related tests [Part 3]

2016-02-18 Thread Rui Li (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9816?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rui Li updated HDFS-9816: - Attachment: HDFS-9816.2.patch Update patch to address comments. The new static method is added to

[jira] [Updated] (HDFS-9832) Erasure Coding: Improve exception handling in ErasureCodingWorker#ReconstructAndTransferBlock

2016-02-18 Thread Lin Yiqun (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9832?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lin Yiqun updated HDFS-9832: Priority: Major (was: Minor) > Erasure Coding: Improve exception handling in >

[jira] [Commented] (HDFS-9832) Erasure Coding: Improve exception handling in ErasureCodingWorker#ReconstructAndTransferBlock

2016-02-18 Thread Lin Yiqun (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9832?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15153853#comment-15153853 ] Lin Yiqun commented on HDFS-9832: - In addition, when IOException happening in {{inTargetsStreams}}, the

[jira] [Commented] (HDFS-8578) On upgrade, Datanode should process all storage/data dirs in parallel

2016-02-18 Thread Hadoop QA (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8578?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15153849#comment-15153849 ] Hadoop QA commented on HDFS-8578: - | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote || Subsystem ||

[jira] [Commented] (HDFS-9829) Erasure Coding: Improve few exception handling logic of ErasureCodingWorker

2016-02-18 Thread Hadoop QA (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9829?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15153835#comment-15153835 ] Hadoop QA commented on HDFS-9829: - | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote || Subsystem ||

[jira] [Updated] (HDFS-9833) Erasure coding: recomputing block checksum on the fly by reconstructing the missed/corrupt block data

2016-02-18 Thread Kai Zheng (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9833?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kai Zheng updated HDFS-9833: Summary: Erasure coding: recomputing block checksum on the fly by reconstructing the missed/corrupt block

[jira] [Commented] (HDFS-9832) Erasure Coding: Improve exception handling in ErasureCodingWorker#ReconstructAndTransferBlock

2016-02-18 Thread Hadoop QA (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9832?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15153789#comment-15153789 ] Hadoop QA commented on HDFS-9832: - | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote || Subsystem ||

[jira] [Created] (HDFS-9833) Erasure coding: recomputing block checksum on the fly by reconstructing missed/corrupt block

2016-02-18 Thread Kai Zheng (JIRA)
Kai Zheng created HDFS-9833: --- Summary: Erasure coding: recomputing block checksum on the fly by reconstructing missed/corrupt block Key: HDFS-9833 URL: https://issues.apache.org/jira/browse/HDFS-9833

[jira] [Commented] (HDFS-9395) Make HDFS audit logging consistant

2016-02-18 Thread Hadoop QA (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9395?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15153787#comment-15153787 ] Hadoop QA commented on HDFS-9395: - | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote || Subsystem ||

[jira] [Commented] (HDFS-8430) Erasure coding: compute file checksum for stripe files

2016-02-18 Thread Kai Zheng (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8430?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15153775#comment-15153775 ] Kai Zheng commented on HDFS-8430: - Hi [~umamaheswararao] and [~rakeshr], I'm glad to know that you will

[jira] [Commented] (HDFS-9818) Correctly handle EC reconstruction work caused by not enough racks

2016-02-18 Thread Hadoop QA (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9818?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15153749#comment-15153749 ] Hadoop QA commented on HDFS-9818: - | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote || Subsystem ||

[jira] [Updated] (HDFS-9819) FsVolume should tolerate few times check-dir failed due to deletion by mistake

2016-02-18 Thread Lin Yiqun (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9819?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lin Yiqun updated HDFS-9819: Resolution: Invalid Status: Resolved (was: Patch Available) > FsVolume should tolerate few times

[jira] [Commented] (HDFS-9549) TestCacheDirectives#testExceedsCapacity is flaky

2016-02-18 Thread Hadoop QA (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9549?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15153743#comment-15153743 ] Hadoop QA commented on HDFS-9549: - | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote || Subsystem ||

[jira] [Commented] (HDFS-9819) FsVolume should tolerate few times check-dir failed due to deletion by mistake

2016-02-18 Thread Lin Yiqun (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9819?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15153742#comment-15153742 ] Lin Yiqun commented on HDFS-9819: - Thanks [~kihwal]'s comment. I close this jira and related to HDFS-8845.

[jira] [Updated] (HDFS-8578) On upgrade, Datanode should process all storage/data dirs in parallel

2016-02-18 Thread Tsz Wo Nicholas Sze (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8578?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tsz Wo Nicholas Sze updated HDFS-8578: -- Attachment: h8578_20160218.patch h8578_20160218.patch: addresses Chris' comment. > On

[jira] [Commented] (HDFS-9395) Make HDFS audit logging consistant

2016-02-18 Thread Vinayakumar B (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9395?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15153719#comment-15153719 ] Vinayakumar B commented on HDFS-9395: - Thanks [~kshukla] for the update. 2 comments for latest patch.

[jira] [Commented] (HDFS-9774) Update distcp command line options in the documentation

2016-02-18 Thread Daisuke Kobayashi (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9774?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15153716#comment-15153716 ] Daisuke Kobayashi commented on HDFS-9774: - bq. add missing pieces like -filters, -skipcrccheck, and

[jira] [Updated] (HDFS-9774) Update distcp command line options in the documentation

2016-02-18 Thread Daisuke Kobayashi (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9774?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Daisuke Kobayashi updated HDFS-9774: Attachment: HDFS-9774-1.trunk.patch HDFS-9774-1.branch-2.patch

[jira] [Commented] (HDFS-9825) Balancer should not terminate if only one of the namenodes has error

2016-02-18 Thread Hadoop QA (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9825?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15153693#comment-15153693 ] Hadoop QA commented on HDFS-9825: - | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote || Subsystem ||

[jira] [Updated] (HDFS-9829) Erasure Coding: Improve few exception handling logic of ErasureCodingWorker

2016-02-18 Thread Rakesh R (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9829?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rakesh R updated HDFS-9829: --- Attachment: HDFS-9829-002.patch > Erasure Coding: Improve few exception handling logic of ErasureCodingWorker

[jira] [Commented] (HDFS-9829) Erasure Coding: Improve few exception handling logic of ErasureCodingWorker

2016-02-18 Thread Rakesh R (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9829?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15153672#comment-15153672 ] Rakesh R commented on HDFS-9829: Thanks [~jingzhao], attached another patch addressing the comment. >

[jira] [Created] (HDFS-9832) Erasure Coding: Improve exception handling in ErasureCodingWorker#ReconstructAndTransferBlock

2016-02-18 Thread Lin Yiqun (JIRA)
Lin Yiqun created HDFS-9832: --- Summary: Erasure Coding: Improve exception handling in ErasureCodingWorker#ReconstructAndTransferBlock Key: HDFS-9832 URL: https://issues.apache.org/jira/browse/HDFS-9832

[jira] [Updated] (HDFS-9832) Erasure Coding: Improve exception handling in ErasureCodingWorker#ReconstructAndTransferBlock

2016-02-18 Thread Lin Yiqun (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9832?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lin Yiqun updated HDFS-9832: Status: Patch Available (was: Open) > Erasure Coding: Improve exception handling in >

[jira] [Updated] (HDFS-9832) Erasure Coding: Improve exception handling in ErasureCodingWorker#ReconstructAndTransferBlock

2016-02-18 Thread Lin Yiqun (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9832?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lin Yiqun updated HDFS-9832: Attachment: HDFS-9832.001.patch Attach a initial patch, kindly review, thanks! > Erasure Coding: Improve

[jira] [Commented] (HDFS-9803) Proactively refresh ShortCircuitCache entries to avoid latency spikes

2016-02-18 Thread Nick Dimiduk (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9803?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15153656#comment-15153656 ] Nick Dimiduk commented on HDFS-9803: Background thread isn't a requirement. Just want to keep it off

[jira] [Updated] (HDFS-9048) DistCp documentation is out-of-dated

2016-02-18 Thread Daisuke Kobayashi (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9048?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Daisuke Kobayashi updated HDFS-9048: Attachment: HDFS-9048-2.patch Thanks [~jojochuang]! Uploaded a new one. Yea, committing in

[jira] [Updated] (HDFS-9395) Make HDFS audit logging consistant

2016-02-18 Thread Kuhu Shukla (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9395?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kuhu Shukla updated HDFS-9395: -- Attachment: HDFS-9395.006.patch Updating patch based on latest comments. Removed the null check for

[jira] [Updated] (HDFS-9818) Correctly handle EC reconstruction work caused by not enough racks

2016-02-18 Thread Jing Zhao (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9818?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jing Zhao updated HDFS-9818: Attachment: HDFS-9818.003.patch Looks like the IOUtils#cleanup cannot be used in hadoop-hdfs-client because

[jira] [Commented] (HDFS-8923) Add -source flag to balancer usage message

2016-02-18 Thread Hudson (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8923?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15153604#comment-15153604 ] Hudson commented on HDFS-8923: -- FAILURE: Integrated in Hadoop-trunk-Commit #9328 (See

[jira] [Commented] (HDFS-8923) Add -source flag to balancer usage message

2016-02-18 Thread Tsz Wo Nicholas Sze (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8923?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15153600#comment-15153600 ] Tsz Wo Nicholas Sze commented on HDFS-8923: --- I have committed this. Thanks, Chris! > Add

[jira] [Commented] (HDFS-9048) DistCp documentation is out-of-dated

2016-02-18 Thread Wei-Chiu Chuang (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9048?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15153597#comment-15153597 ] Wei-Chiu Chuang commented on HDFS-9048: --- I think it's fine to commit this in 2.x. webhdfs is

[jira] [Updated] (HDFS-8923) Add -source flag to balancer usage message

2016-02-18 Thread Tsz Wo Nicholas Sze (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8923?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tsz Wo Nicholas Sze updated HDFS-8923: -- Resolution: Fixed Fix Version/s: 2.8.0 Status: Resolved (was: Patch

[jira] [Commented] (HDFS-9816) Erasure Coding: allow to use multiple EC policies in striping related tests [Part 3]

2016-02-18 Thread Rui Li (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9816?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15153579#comment-15153579 ] Rui Li commented on HDFS-9816: -- Thanks [~zhz] for the clarifications. How about putting the static method in

[jira] [Updated] (HDFS-9825) Balancer should not terminate if only one of the namenodes has error

2016-02-18 Thread Tsz Wo Nicholas Sze (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9825?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tsz Wo Nicholas Sze updated HDFS-9825: -- Attachment: h9825_20160218b.patch h9825_20160218b.patch: adds a new test. > Balancer

[jira] [Updated] (HDFS-9818) Correctly handle EC reconstruction work caused by not enough racks

2016-02-18 Thread Tsz Wo Nicholas Sze (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9818?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tsz Wo Nicholas Sze updated HDFS-9818: -- Hadoop Flags: Reviewed +1 the new patch looks good. > Correctly handle EC

[jira] [Commented] (HDFS-8923) Add -source flag to balancer usage message

2016-02-18 Thread Hadoop QA (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8923?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15153533#comment-15153533 ] Hadoop QA commented on HDFS-8923: - | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote || Subsystem ||

[jira] [Updated] (HDFS-9549) TestCacheDirectives#testExceedsCapacity is flaky

2016-02-18 Thread Xiao Chen (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9549?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Chen updated HDFS-9549: Attachment: HDFS-9549.03.patch Patch 3 fixes checkstyle warnings. [~cmccabe], could you take another look?

[jira] [Commented] (HDFS-9818) Correctly handle EC reconstruction work caused by not enough racks

2016-02-18 Thread Hadoop QA (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9818?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15153528#comment-15153528 ] Hadoop QA commented on HDFS-9818: - | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote || Subsystem ||

[jira] [Commented] (HDFS-9395) Make HDFS audit logging consistant

2016-02-18 Thread Kuhu Shukla (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9395?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15153517#comment-15153517 ] Kuhu Shukla commented on HDFS-9395: --- Thanks [~cnauroth]! One follow up question, Currently

[jira] [Updated] (HDFS-8923) Add -source flag to balancer usage message

2016-02-18 Thread Tsz Wo Nicholas Sze (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8923?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tsz Wo Nicholas Sze updated HDFS-8923: -- Hadoop Flags: Reviewed Component/s: documentation +1 patch looks good. > Add

[jira] [Commented] (HDFS-9549) TestCacheDirectives#testExceedsCapacity is flaky

2016-02-18 Thread Hadoop QA (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9549?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15153480#comment-15153480 ] Hadoop QA commented on HDFS-9549: - | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote || Subsystem ||

[jira] [Commented] (HDFS-8757) OzoneHandler : Add localStorageHandler support for Buckets

2016-02-18 Thread Hadoop QA (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8757?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15153467#comment-15153467 ] Hadoop QA commented on HDFS-8757: - | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote || Subsystem ||

[jira] [Commented] (HDFS-9816) Erasure Coding: allow to use multiple EC policies in striping related tests [Part 3]

2016-02-18 Thread Zhe Zhang (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9816?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15153436#comment-15153436 ] Zhe Zhang commented on HDFS-9816: - bq. Zhe Zhang, regarding your last comment, which test do you mean by

[jira] [Commented] (HDFS-9804) Allow long-running Balancer in Kerberized Environments

2016-02-18 Thread Xiao Chen (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9804?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15153428#comment-15153428 ] Xiao Chen commented on HDFS-9804: - Test failures seems unrelated. I'm gonna ignore the checkstyles since

[jira] [Updated] (HDFS-9818) Correctly handle EC reconstruction work caused by not enough racks

2016-02-18 Thread Jing Zhao (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9818?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jing Zhao updated HDFS-9818: Attachment: HDFS-9818.002.patch Thanks for the review, Nicholas! Update the patch to address your comments

[jira] [Updated] (HDFS-8923) Add -source flag to balancer usage message

2016-02-18 Thread Chris Trezzo (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8923?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chris Trezzo updated HDFS-8923: --- Attachment: HDFS-8923-trunk-v4.patch [~szetszwo] sounds good! Attached is v4. > Add -source flag to

[jira] [Updated] (HDFS-6133) Add a feature for replica pinning so that a pinned replica will not be moved by Balancer/Mover.

2016-02-18 Thread Tsz Wo Nicholas Sze (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-6133?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tsz Wo Nicholas Sze updated HDFS-6133: -- Summary: Add a feature for replica pinning so that a pinned replica will not be moved by

[jira] [Updated] (HDFS-8757) OzoneHandler : Add localStorageHandler support for Buckets

2016-02-18 Thread Anu Engineer (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8757?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Anu Engineer updated HDFS-8757: --- Status: Open (was: Patch Available) > OzoneHandler : Add localStorageHandler support for Buckets >

[jira] [Updated] (HDFS-8757) OzoneHandler : Add localStorageHandler support for Buckets

2016-02-18 Thread Anu Engineer (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8757?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Anu Engineer updated HDFS-8757: --- Status: Patch Available (was: Open) Resubmitting since build picked up a patch that had an issue >

[jira] [Commented] (HDFS-8757) OzoneHandler : Add localStorageHandler support for Buckets

2016-02-18 Thread Hadoop QA (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8757?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15153302#comment-15153302 ] Hadoop QA commented on HDFS-8757: - | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote || Subsystem ||

[jira] [Commented] (HDFS-9825) Balancer should not terminate if only one of the namenodes has error

2016-02-18 Thread Hadoop QA (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9825?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15153300#comment-15153300 ] Hadoop QA commented on HDFS-9825: - | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote || Subsystem ||

[jira] [Commented] (HDFS-9825) Balancer should not terminate if only one of the namenodes has error

2016-02-18 Thread Tsz Wo Nicholas Sze (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9825?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15153295#comment-15153295 ] Tsz Wo Nicholas Sze commented on HDFS-9825: --- Thanks [~liuml07], let me refactor the code and add

[jira] [Commented] (HDFS-9804) Allow long-running Balancer in Kerberized Environments

2016-02-18 Thread Hadoop QA (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9804?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15153286#comment-15153286 ] Hadoop QA commented on HDFS-9804: - | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote || Subsystem ||

[jira] [Updated] (HDFS-8757) OzoneHandler : Add localStorageHandler support for Buckets

2016-02-18 Thread Anu Engineer (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8757?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Anu Engineer updated HDFS-8757: --- Attachment: HDFS-8757-HDFS-7240.001.patch > OzoneHandler : Add localStorageHandler support for Buckets

[jira] [Updated] (HDFS-8757) OzoneHandler : Add localStorageHandler support for Buckets

2016-02-18 Thread Anu Engineer (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8757?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Anu Engineer updated HDFS-8757: --- Status: Patch Available (was: Open) > OzoneHandler : Add localStorageHandler support for Buckets >

[jira] [Updated] (HDFS-8757) OzoneHandler : Add localStorageHandler support for Buckets

2016-02-18 Thread Anu Engineer (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8757?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Anu Engineer updated HDFS-8757: --- Attachment: (was: HDFS-8757-HDFS-7240.001.patch) > OzoneHandler : Add localStorageHandler support

[jira] [Commented] (HDFS-9818) Correctly handle EC reconstruction work caused by not enough racks

2016-02-18 Thread Hadoop QA (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9818?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15153269#comment-15153269 ] Hadoop QA commented on HDFS-9818: - | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote || Subsystem ||

[jira] [Updated] (HDFS-9549) TestCacheDirectives#testExceedsCapacity is flaky

2016-02-18 Thread Xiao Chen (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9549?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Chen updated HDFS-9549: Attachment: HDFS-9549.02.patch I have talked with Colin offline, and my comment above misinterpreted his

[jira] [Updated] (HDFS-8757) OzoneHandler : Add localStorageHandler support for Buckets

2016-02-18 Thread Anu Engineer (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8757?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Anu Engineer updated HDFS-8757: --- Status: Open (was: Patch Available) > OzoneHandler : Add localStorageHandler support for Buckets >

[jira] [Commented] (HDFS-8578) On upgrade, Datanode should process all storage/data dirs in parallel

2016-02-18 Thread Chris Trezzo (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8578?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15153250#comment-15153250 ] Chris Trezzo commented on HDFS-8578: Thanks [~szetszwo]! I now see that the boolean ensures that

[jira] [Updated] (HDFS-8757) OzoneHandler : Add localStorageHandler support for Buckets

2016-02-18 Thread Anu Engineer (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8757?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Anu Engineer updated HDFS-8757: --- Status: Patch Available (was: Open) > OzoneHandler : Add localStorageHandler support for Buckets >

[jira] [Commented] (HDFS-9825) Balancer should not terminate if only one of the namenodes has error

2016-02-18 Thread Mingliang Liu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9825?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15153243#comment-15153243 ] Mingliang Liu commented on HDFS-9825: - The fix looks pretty good to me. +1 (non-binding) My minor

[jira] [Updated] (HDFS-6984) In Hadoop 3, make FileStatus serialize itself via protobuf

2016-02-18 Thread Andrew Wang (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-6984?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Wang updated HDFS-6984: -- Hadoop Flags: Incompatible change > In Hadoop 3, make FileStatus serialize itself via protobuf >

[jira] [Commented] (HDFS-9818) Correctly handle EC reconstruction work caused by not enough racks

2016-02-18 Thread Tsz Wo Nicholas Sze (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9818?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15153241#comment-15153241 ] Tsz Wo Nicholas Sze commented on HDFS-9818: --- - We may also make chooseSource4SimpleReplication

[jira] [Updated] (HDFS-8757) OzoneHandler : Add localStorageHandler support for Buckets

2016-02-18 Thread Anu Engineer (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8757?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Anu Engineer updated HDFS-8757: --- Attachment: HDFS-8757-HDFS-7240.001.patch > OzoneHandler : Add localStorageHandler support for Buckets

[jira] [Commented] (HDFS-9818) Correctly handle EC reconstruction work caused by not enough racks

2016-02-18 Thread Tsz Wo Nicholas Sze (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9818?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15153225#comment-15153225 ] Tsz Wo Nicholas Sze commented on HDFS-9818: --- - Should we check all targets instead of the first

[jira] [Commented] (HDFS-9820) Improve distcp to support efficient restore to an earlier snapshot

2016-02-18 Thread Yongjun Zhang (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9820?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15153220#comment-15153220 ] Yongjun Zhang commented on HDFS-9820: - Hmm, in theory, I think we could make HDFS-4167 to relax the

[jira] [Commented] (HDFS-7661) Erasure coding: support hflush and hsync

2016-02-18 Thread Zhe Zhang (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-7661?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15153185#comment-15153185 ] Zhe Zhang commented on HDFS-7661: - Good discussions here [~liuml07] / [~demongaorui]. The logic of

[jira] [Commented] (HDFS-9829) Erasure Coding: Improve few exception handling logic of ErasureCodingWorker

2016-02-18 Thread Jing Zhao (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9829?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15153113#comment-15153113 ] Jing Zhao commented on HDFS-9829: - Thanks for working on this, [~rakeshr]. The patch looks good to me. The

[jira] [Commented] (HDFS-9805) TCP_NODELAY not set before SASL handshake in data transfer pipeline

2016-02-18 Thread Gary Helmling (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9805?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15153064#comment-15153064 ] Gary Helmling commented on HDFS-9805: - The checkstyle warning is on the {{DataXceiver::writeBlock()}}

[jira] [Commented] (HDFS-8578) On upgrade, Datanode should process all storage/data dirs in parallel

2016-02-18 Thread Tsz Wo Nicholas Sze (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8578?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15153045#comment-15153045 ] Tsz Wo Nicholas Sze commented on HDFS-8578: --- [~vinayrpet], thanks for reviewing the patch!

[jira] [Commented] (HDFS-9829) Erasure Coding: Improve few exception handling logic of ErasureCodingWorker

2016-02-18 Thread Hadoop QA (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9829?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15153029#comment-15153029 ] Hadoop QA commented on HDFS-9829: - | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote || Subsystem ||

[jira] [Updated] (HDFS-9825) Balancer should not terminate if only one of the namenodes has error

2016-02-18 Thread Tsz Wo Nicholas Sze (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9825?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tsz Wo Nicholas Sze updated HDFS-9825: -- Attachment: h9825_20160218.patch h9825_20160218.patch: fixes a NPE bug. > Balancer

[jira] [Commented] (HDFS-9830) Remove references to hftp in ViewFs documentation in trunk

2016-02-18 Thread Hudson (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9830?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15153025#comment-15153025 ] Hudson commented on HDFS-9830: -- FAILURE: Integrated in Hadoop-trunk-Commit #9325 (See

[jira] [Created] (HDFS-9831) Document webhdfs retry configuration keys introduced by HDFS-5219/HDFS-5122

2016-02-18 Thread Xiaoyu Yao (JIRA)
Xiaoyu Yao created HDFS-9831: Summary: Document webhdfs retry configuration keys introduced by HDFS-5219/HDFS-5122 Key: HDFS-9831 URL: https://issues.apache.org/jira/browse/HDFS-9831 Project: Hadoop

[jira] [Updated] (HDFS-4946) Allow preferLocalNode in BlockPlacementPolicyDefault to be configurable

2016-02-18 Thread Kihwal Lee (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-4946?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kihwal Lee updated HDFS-4946: - Resolution: Fixed Hadoop Flags: Reviewed Fix Version/s: 2.7.3 Status: Resolved

[jira] [Commented] (HDFS-9830) Remove references to hftp in ViewFs documentation in trunk

2016-02-18 Thread Wei-Chiu Chuang (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9830?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15152994#comment-15152994 ] Wei-Chiu Chuang commented on HDFS-9830: --- Thanks [~ajisakaa] for the quick review and commit! >

[jira] [Updated] (HDFS-9830) Remove references to hftp in ViewFs documentation in trunk

2016-02-18 Thread Akira AJISAKA (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9830?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Akira AJISAKA updated HDFS-9830: Resolution: Fixed Hadoop Flags: Reviewed Fix Version/s: 3.0.0 Status: Resolved

[jira] [Commented] (HDFS-9830) Remove references to hftp in ViewFs documentation in trunk

2016-02-18 Thread Akira AJISAKA (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9830?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15152975#comment-15152975 ] Akira AJISAKA commented on HDFS-9830: - LGTM, +1. > Remove references to hftp in ViewFs documentation

[jira] [Commented] (HDFS-4946) Allow preferLocalNode in BlockPlacementPolicyDefault to be configurable

2016-02-18 Thread Hudson (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-4946?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15152955#comment-15152955 ] Hudson commented on HDFS-4946: -- FAILURE: Integrated in Hadoop-trunk-Commit #9323 (See

[jira] [Updated] (HDFS-9818) Correctly handle EC reconstruction work caused by not enough racks

2016-02-18 Thread Jing Zhao (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9818?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jing Zhao updated HDFS-9818: Status: Patch Available (was: Open) > Correctly handle EC reconstruction work caused by not enough racks >

[jira] [Updated] (HDFS-9818) Correctly handle EC reconstruction work caused by not enough racks

2016-02-18 Thread Jing Zhao (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9818?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jing Zhao updated HDFS-9818: Attachment: HDFS-9818.001.patch Update the patch to add new unit test. > Correctly handle EC reconstruction

[jira] [Commented] (HDFS-7955) Improve naming of classes, methods, and variables related to block replication and recovery

2016-02-18 Thread Andrew Wang (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-7955?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15152916#comment-15152916 ] Andrew Wang commented on HDFS-7955: --- SGTM. We can also do these changes incrementally if that helps;

[jira] [Commented] (HDFS-4946) Allow preferLocalNode in BlockPlacementPolicyDefault to be configurable

2016-02-18 Thread Kihwal Lee (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-4946?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15152914#comment-15152914 ] Kihwal Lee commented on HDFS-4946: -- bq. Ideally we would like a client sent hint that influences the

[jira] [Commented] (HDFS-4946) Allow preferLocalNode in BlockPlacementPolicyDefault to be configurable

2016-02-18 Thread Kihwal Lee (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-4946?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15152905#comment-15152905 ] Kihwal Lee commented on HDFS-4946: -- Assigned it to [~nroberts] for now, the credit goes to the both

[jira] [Updated] (HDFS-4946) Allow preferLocalNode in BlockPlacementPolicyDefault to be configurable

2016-02-18 Thread Kihwal Lee (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-4946?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kihwal Lee updated HDFS-4946: - Assignee: Nathan Roberts (was: James Kinley) > Allow preferLocalNode in BlockPlacementPolicyDefault to be

[jira] [Updated] (HDFS-4946) Allow preferLocalNode in BlockPlacementPolicyDefault to be configurable

2016-02-18 Thread Kihwal Lee (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-4946?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kihwal Lee updated HDFS-4946: - Description: Allow preferLocalNode in BlockPlacementPolicyDefault to be disabled in configuration to

[jira] [Updated] (HDFS-9804) Allow long-running Balancer in Kerberized Environments

2016-02-18 Thread Xiao Chen (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9804?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Chen updated HDFS-9804: Attachment: (was: HDFS-9804.02.patch) > Allow long-running Balancer in Kerberized Environments >

[jira] [Updated] (HDFS-9804) Allow long-running Balancer in Kerberized Environments

2016-02-18 Thread Xiao Chen (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9804?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Chen updated HDFS-9804: Attachment: HDFS-9804.02.patch Oops, hdfs-default.xml was just changed recently. Attached an updated patch

[jira] [Commented] (HDFS-9820) Improve distcp to support efficient restore to an earlier snapshot

2016-02-18 Thread Yongjun Zhang (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9820?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15152850#comment-15152850 ] Yongjun Zhang commented on HDFS-9820: - Thanks [~jingzhao]! Agree. HDFS-4167 also has the restriction

[jira] [Commented] (HDFS-9640) Remove hsftp from DistCp in trunk

2016-02-18 Thread Wei-Chiu Chuang (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9640?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15152838#comment-15152838 ] Wei-Chiu Chuang commented on HDFS-9640: --- Need to rebase/resolve conflicts due to HDFS-9764. > Remove

[jira] [Commented] (HDFS-9640) Remove hsftp from DistCp in trunk

2016-02-18 Thread Hadoop QA (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9640?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15152835#comment-15152835 ] Hadoop QA commented on HDFS-9640: - | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote || Subsystem ||

[jira] [Commented] (HDFS-9820) Improve distcp to support efficient restore to an earlier snapshot

2016-02-18 Thread Jing Zhao (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9820?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15152831#comment-15152831 ] Jing Zhao commented on HDFS-9820: - Thanks for creating the jira, [~yzhangal]. I think this jira is not very

[jira] [Commented] (HDFS-9764) DistCp doesn't print arg value for -numListstatusThreads

2016-02-18 Thread Yongjun Zhang (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9764?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15152830#comment-15152830 ] Yongjun Zhang commented on HDFS-9764: - +1 on the patch, will commit soon, Thanks. > DistCp doesn't

[jira] [Commented] (HDFS-4946) Allow preferLocalNode in BlockPlacementPolicyDefault to be configurable

2016-02-18 Thread Hadoop QA (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-4946?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15152816#comment-15152816 ] Hadoop QA commented on HDFS-4946: - | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote || Subsystem ||

[jira] [Commented] (HDFS-9804) Allow long-running Balancer in Kerberized Environments

2016-02-18 Thread Hadoop QA (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9804?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15152814#comment-15152814 ] Hadoop QA commented on HDFS-9804: - | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote || Subsystem ||

[jira] [Commented] (HDFS-4900) Print user when services are started

2016-02-18 Thread Yongjun Zhang (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-4900?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15152806#comment-15152806 ] Yongjun Zhang commented on HDFS-4900: - +1 on the patch 001, will commit soon. Thanks. > Print user

[jira] [Commented] (HDFS-9711) Integrate CSRF prevention filter in WebHDFS.

2016-02-18 Thread Hudson (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9711?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15152802#comment-15152802 ] Hudson commented on HDFS-9711: -- FAILURE: Integrated in Hadoop-trunk-Commit #9322 (See

[jira] [Commented] (HDFS-9830) Remove references to hftp in ViewFs documentation in trunk

2016-02-18 Thread Wei-Chiu Chuang (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9830?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15152801#comment-15152801 ] Wei-Chiu Chuang commented on HDFS-9830: --- The precommit failures are more of an issue with Yetus. This

  1   2   >