[jira] [Updated] (HDFS-9719) Refactoring ErasureCodingWorker into smaller reusable constructs

2016-04-06 Thread Uma Maheswara Rao G (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9719?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Uma Maheswara Rao G updated HDFS-9719: -- Resolution: Fixed Hadoop Flags: Reviewed Fix Version/s: 3.0.0

[jira] [Commented] (HDFS-10269) Invalid value configured for dfs.datanode.failed.volumes.tolerated cause the datanode exit

2016-04-06 Thread Hadoop QA (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10269?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15229718#comment-15229718 ] Hadoop QA commented on HDFS-10269: -- | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote || Subsystem ||

[jira] [Commented] (HDFS-10268) Ozone: end-to-end integration for create/get volumes, buckets and keys.

2016-04-06 Thread Anu Engineer (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10268?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15229707#comment-15229707 ] Anu Engineer commented on HDFS-10268: - [~cnauroth] Thanks for contributing this patch. The code

[jira] [Commented] (HDFS-9719) Refactoring ErasureCodingWorker into smaller reusable constructs

2016-04-06 Thread Kai Zheng (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9719?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15229706#comment-15229706 ] Kai Zheng commented on HDFS-9719: - Thanks [~umamaheswararao] for the once more review! > Refactoring

[jira] [Commented] (HDFS-10186) DirectoryScanner: Improve logs by adding full path of both actual and expected block directories

2016-04-06 Thread Hudson (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10186?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15229693#comment-15229693 ] Hudson commented on HDFS-10186: --- FAILURE: Integrated in Hadoop-trunk-Commit #9574 (See

[jira] [Updated] (HDFS-10186) DirectoryScanner: Improve logs by adding full path of both actual and expected block directories

2016-04-06 Thread Tsz Wo Nicholas Sze (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10186?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tsz Wo Nicholas Sze updated HDFS-10186: --- Resolution: Fixed Fix Version/s: 2.7.3 Status: Resolved (was: Patch

[jira] [Updated] (HDFS-10186) DirectoryScanner: Improve logs by adding full path of both actual and expected block directories

2016-04-06 Thread Tsz Wo Nicholas Sze (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10186?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tsz Wo Nicholas Sze updated HDFS-10186: --- Hadoop Flags: Reviewed +1 patch looks good. > DirectoryScanner: Improve logs by

[jira] [Commented] (HDFS-9719) Refactoring ErasureCodingWorker into smaller reusable constructs

2016-04-06 Thread Uma Maheswara Rao G (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9719?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15229666#comment-15229666 ] Uma Maheswara Rao G commented on HDFS-9719: --- Thanks [~drankye] for the update. Latest patch looks

[jira] [Commented] (HDFS-3702) Add an option for NOT writing the blocks locally if there is a datanode on the same box as the client

2016-04-06 Thread Tsz Wo Nicholas Sze (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-3702?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15229665#comment-15229665 ] Tsz Wo Nicholas Sze commented on HDFS-3702: --- BTW, we should check if

[jira] [Commented] (HDFS-10267) Extra "synchronized" on FsDatasetImpl#recoverAppend and FsDatasetImpl#recoverClose

2016-04-06 Thread Hudson (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10267?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15229663#comment-15229663 ] Hudson commented on HDFS-10267: --- FAILURE: Integrated in Hadoop-trunk-Commit #9573 (See

[jira] [Updated] (HDFS-10267) Extra "synchronized" on FsDatasetImpl#recoverAppend and FsDatasetImpl#recoverClose

2016-04-06 Thread Colin Patrick McCabe (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10267?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Colin Patrick McCabe updated HDFS-10267: Resolution: Fixed Fix Version/s: 2.8.0 Status: Resolved (was:

[jira] [Commented] (HDFS-10269) Invalid value configured for dfs.datanode.failed.volumes.tolerated cause the datanode exit

2016-04-06 Thread Lin Yiqun (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10269?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15229631#comment-15229631 ] Lin Yiqun commented on HDFS-10269: -- I think we can add a case for address your comments. If the value is

[jira] [Commented] (HDFS-9735) DiskBalancer : Refactor moveBlockAcrossStorage to be used by disk balancer

2016-04-06 Thread Anu Engineer (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9735?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15229622#comment-15229622 ] Anu Engineer commented on HDFS-9735: [~vinayrpet] Thank you for explaining this. > DiskBalancer :

[jira] [Updated] (HDFS-8533) Mismatch in displaying the "MissingBlock" count in fsck and in other metric reports

2016-04-06 Thread Javier Maestro (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8533?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Javier Maestro updated HDFS-8533: - Affects Version/s: 2.7.0 > Mismatch in displaying the "MissingBlock" count in fsck and in other

[jira] [Commented] (HDFS-10267) Extra "synchronized" on FsDatasetImpl#recoverAppend and FsDatasetImpl#recoverClose

2016-04-06 Thread Hadoop QA (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10267?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15229609#comment-15229609 ] Hadoop QA commented on HDFS-10267: -- | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote || Subsystem ||

[jira] [Commented] (HDFS-10269) Invalid value configured for dfs.datanode.failed.volumes.tolerated cause the datanode exit

2016-04-06 Thread Brahma Reddy Battula (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10269?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15229606#comment-15229606 ] Brahma Reddy Battula commented on HDFS-10269: - IMO, Logging WARN is not good idea..Since if

[jira] [Commented] (HDFS-10175) add per-operation stats to FileSystem.Statistics

2016-04-06 Thread Mingliang Liu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10175?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15229596#comment-15229596 ] Mingliang Liu commented on HDFS-10175: -- Thanks for your test to quantify the overhead of the per-op

[jira] [Commented] (HDFS-3702) Add an option for NOT writing the blocks locally if there is a datanode on the same box as the client

2016-04-06 Thread Tsz Wo Nicholas Sze (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-3702?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15229594#comment-15229594 ] Tsz Wo Nicholas Sze commented on HDFS-3702: --- As mentioned before, once CreateFlag.NO_LOCAL_WRITE

[jira] [Commented] (HDFS-8356) Document missing properties in hdfs-default.xml

2016-04-06 Thread Ray Chiang (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8356?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15229589#comment-15229589 ] Ray Chiang commented on HDFS-8356: -- After some discussion on HDFS-10266, it looks like those two

[jira] [Updated] (HDFS-10269) Invalid value configured for dfs.datanode.failed.volumes.tolerated cause the datanode exit

2016-04-06 Thread Lin Yiqun (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10269?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lin Yiqun updated HDFS-10269: - Attachment: HDFS-10269.001.patch > Invalid value configured for dfs.datanode.failed.volumes.tolerated

[jira] [Updated] (HDFS-10269) Invalid value configured for dfs.datanode.failed.volumes.tolerated cause the datanode exit

2016-04-06 Thread Lin Yiqun (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10269?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lin Yiqun updated HDFS-10269: - Status: Patch Available (was: Open) Attach a simple path for this. > Invalid value configured for

[jira] [Commented] (HDFS-9530) huge Non-DFS Used in hadoop 2.6.2 & 2.7.1

2016-04-06 Thread Brahma Reddy Battula (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9530?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15229542#comment-15229542 ] Brahma Reddy Battula commented on HDFS-9530: If any of the datanode other than 1st DN is down

[jira] [Commented] (HDFS-10268) Ozone: end-to-end integration for create/get volumes, buckets and keys.

2016-04-06 Thread Hadoop QA (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10268?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15229543#comment-15229543 ] Hadoop QA commented on HDFS-10268: -- | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote || Subsystem ||

[jira] [Created] (HDFS-10269) Invalid value configured for dfs.datanode.failed.volumes.tolerated cause the datanode exit

2016-04-06 Thread Lin Yiqun (JIRA)
Lin Yiqun created HDFS-10269: Summary: Invalid value configured for dfs.datanode.failed.volumes.tolerated cause the datanode exit Key: HDFS-10269 URL: https://issues.apache.org/jira/browse/HDFS-10269

[jira] [Commented] (HDFS-9735) DiskBalancer : Refactor moveBlockAcrossStorage to be used by disk balancer

2016-04-06 Thread Vinayakumar B (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9735?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15229539#comment-15229539 ] Vinayakumar B commented on HDFS-9735: - Thanks [~anu] for the update. bq. In moveBlock(), from line 990

[jira] [Commented] (HDFS-7661) [umbrella] support hflush and hsync for erasure coded files

2016-04-06 Thread Kai Zheng (JIRA)
h-sync-design-v20160323.pdf, > HDFS-EC-file-flush-sync-design-version1.1.pdf, Undo-Log-Design-20160406.jpg > > > We also need to support hflush/hsync and visible length. -- This message was sent by Atlassian JIRA (v6.3.4#6332)

[jira] [Commented] (HDFS-10192) Namenode safemode not coming out during failover

2016-04-06 Thread Brahma Reddy Battula (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10192?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15229503#comment-15229503 ] Brahma Reddy Battula commented on HDFS-10192: - [~jingzhao] thanks a lot for review and commit

[jira] [Commented] (HDFS-10267) Extra "synchronized" on FsDatasetImpl#recoverAppend and FsDatasetImpl#recoverClose

2016-04-06 Thread Lei (Eddy) Xu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10267?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15229493#comment-15229493 ] Lei (Eddy) Xu commented on HDFS-10267: -- +1. Thanks for the explanations, [~cmccabe]. > Extra

[jira] [Commented] (HDFS-7661) [umbrella] support hflush and hsync for erasure coded files

2016-04-06 Thread Todd Lipcon (JIRA)
HDFS-7661-wip.01.patch, > HDFS-EC-file-flush-sync-design-v20160323.pdf, > HDFS-EC-file-flush-sync-design-version1.1.pdf, Undo-Log-Design-20160406.jpg > > > We also need to support hflush/hsync and visible length. -- This message was sent by Atlassian JIRA (v6.3.4#6332)

[jira] [Resolved] (HDFS-10213) there is a corrupt/missing block report, but fsck result is HEALTHY

2016-04-06 Thread Wataru Yukawa (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10213?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wataru Yukawa resolved HDFS-10213. -- Resolution: Duplicate Thank you for your comment. Actually,

[jira] [Commented] (HDFS-10267) Extra "synchronized" on FsDatasetImpl#recoverAppend and FsDatasetImpl#recoverClose

2016-04-06 Thread Xiao Chen (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10267?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15229452#comment-15229452 ] Xiao Chen commented on HDFS-10267: -- Thanks for explaining Colin, that's very helpful. And I agree that

[jira] [Commented] (HDFS-10267) Extra "synchronized" on FsDatasetImpl#recoverAppend and FsDatasetImpl#recoverClose

2016-04-06 Thread Wei-Chiu Chuang (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10267?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15229448#comment-15229448 ] Wei-Chiu Chuang commented on HDFS-10267: Thanks, Colin. The last patch (v04) looks good to me.

[jira] [Commented] (HDFS-7661) [umbrella] support hflush and hsync for erasure coded files

2016-04-06 Thread Kai Zheng (JIRA)
Rui > Attachments: EC-file-flush-and-sync-steps-plan-2015-12-01.png, > HDFS-7661-unitTest-wip-trunk.patch, HDFS-7661-wip.01.patch, > HDFS-EC-file-flush-sync-design-v20160323.pdf, > HDFS-EC-file-flush-sync-design-version1.1.pdf, Undo-Log-Design-20160406.jpg > > > We a

[jira] [Commented] (HDFS-10267) Extra "synchronized" on FsDatasetImpl#recoverAppend and FsDatasetImpl#recoverClose

2016-04-06 Thread Hadoop QA (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10267?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15229440#comment-15229440 ] Hadoop QA commented on HDFS-10267: -- | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote || Subsystem ||

[jira] [Updated] (HDFS-10224) Implement an asynchronous DistributedFileSystem

2016-04-06 Thread Xiaobing Zhou (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10224?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiaobing Zhou updated HDFS-10224: - Description: This is proposed to implement an asynchronous DistributedFileSystem based on

[jira] [Updated] (HDFS-10224) Implement an asynchronous DistributedFileSystem

2016-04-06 Thread Xiaobing Zhou (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10224?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiaobing Zhou updated HDFS-10224: - Description: This is proposed to implement an asynchronous DistributedFileSystem based on

[jira] [Commented] (HDFS-10267) Extra "synchronized" on FsDatasetImpl#recoverAppend and FsDatasetImpl#recoverClose

2016-04-06 Thread Colin Patrick McCabe (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10267?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15229427#comment-15229427 ] Colin Patrick McCabe commented on HDFS-10267: - posted v4 incorporating review comments >

[jira] [Updated] (HDFS-10224) Implement an asynchronous DistributedFileSystem

2016-04-06 Thread Xiaobing Zhou (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10224?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiaobing Zhou updated HDFS-10224: - Summary: Implement an asynchronous DistributedFileSystem (was: Augment DistributedFileSystem to

[jira] [Commented] (HDFS-10267) Extra "synchronized" on FsDatasetImpl#recoverAppend and FsDatasetImpl#recoverClose

2016-04-06 Thread Colin Patrick McCabe (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10267?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15229424#comment-15229424 ] Colin Patrick McCabe commented on HDFS-10267: - Basically the test, at the high level, is

[jira] [Updated] (HDFS-10267) Extra "synchronized" on FsDatasetImpl#recoverAppend and FsDatasetImpl#recoverClose

2016-04-06 Thread Colin Patrick McCabe (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10267?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Colin Patrick McCabe updated HDFS-10267: Attachment: HDFS-10267.004.patch > Extra "synchronized" on

[jira] [Commented] (HDFS-10267) Extra "synchronized" on FsDatasetImpl#recoverAppend and FsDatasetImpl#recoverClose

2016-04-06 Thread Colin Patrick McCabe (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10267?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15229420#comment-15229420 ] Colin Patrick McCabe commented on HDFS-10267: - Thanks for the reviews, [~eddyxu] and

[jira] [Commented] (HDFS-10267) Extra "synchronized" on FsDatasetImpl#recoverAppend and FsDatasetImpl#recoverClose

2016-04-06 Thread Xiao Chen (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10267?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15229404#comment-15229404 ] Xiao Chen commented on HDFS-10267: -- Thanks [~cmccabe] for working on this, patch looks great! Also thanks

[jira] [Comment Edited] (HDFS-10268) Ozone: end-to-end integration for create/get volumes, buckets and keys.

2016-04-06 Thread Chris Nauroth (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10268?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15229396#comment-15229396 ] Chris Nauroth edited comment on HDFS-10268 at 4/7/16 12:09 AM: --- I am

[jira] [Updated] (HDFS-10268) Ozone: end-to-end integration for create/get volumes, buckets and keys.

2016-04-06 Thread Chris Nauroth (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10268?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chris Nauroth updated HDFS-10268: - Status: Patch Available (was: Open) > Ozone: end-to-end integration for create/get volumes,

[jira] [Commented] (HDFS-8791) block ID-based DN storage layout can be very slow for datanode on ext4

2016-04-06 Thread Vinod Kumar Vavilapalli (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8791?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15229398#comment-15229398 ] Vinod Kumar Vavilapalli commented on HDFS-8791: --- Tx [~kihwal]! As [~kihwal] pointed out

[jira] [Updated] (HDFS-10268) Ozone: end-to-end integration for create/get volumes, buckets and keys.

2016-04-06 Thread Chris Nauroth (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10268?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chris Nauroth updated HDFS-10268: - Attachment: HDFS-10268-HDFS-7240.001.patch I am attaching patch v001. [~anu], would you please

[jira] [Comment Edited] (HDFS-10268) Ozone: end-to-end integration for create/get volumes, buckets and keys.

2016-04-06 Thread Chris Nauroth (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10268?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15229396#comment-15229396 ] Chris Nauroth edited comment on HDFS-10268 at 4/7/16 12:09 AM: --- I am

[jira] [Created] (HDFS-10268) Ozone: end-to-end integration for create/get volumes, buckets and keys.

2016-04-06 Thread Chris Nauroth (JIRA)
Chris Nauroth created HDFS-10268: Summary: Ozone: end-to-end integration for create/get volumes, buckets and keys. Key: HDFS-10268 URL: https://issues.apache.org/jira/browse/HDFS-10268 Project:

[jira] [Commented] (HDFS-7661) [umbrella] support hflush and hsync for erasure coded files

2016-04-06 Thread Zhe Zhang (JIRA)
-v20160323.pdf, > HDFS-EC-file-flush-sync-design-version1.1.pdf, Undo-Log-Design-20160406.jpg > > > We also need to support hflush/hsync and visible length. -- This message was sent by Atlassian JIRA (v6.3.4#6332)

[jira] [Commented] (HDFS-8791) block ID-based DN storage layout can be very slow for datanode on ext4

2016-04-06 Thread Chris Trezzo (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8791?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15229360#comment-15229360 ] Chris Trezzo commented on HDFS-8791: Thanks [~kihwal]! > block ID-based DN storage layout can be very

[jira] [Commented] (HDFS-8791) block ID-based DN storage layout can be very slow for datanode on ext4

2016-04-06 Thread Tsz Wo Nicholas Sze (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8791?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15229340#comment-15229340 ] Tsz Wo Nicholas Sze commented on HDFS-8791: --- [~kihwal], thanks for reverting it. +1 BTW, you

[jira] [Commented] (HDFS-10267) Extra "synchronized" on FsDatasetImpl#recoverAppend and FsDatasetImpl#recoverClose

2016-04-06 Thread Hadoop QA (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10267?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15229323#comment-15229323 ] Hadoop QA commented on HDFS-10267: -- | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote || Subsystem ||

[jira] [Commented] (HDFS-10267) Extra "synchronized" on FsDatasetImpl#recoverAppend and FsDatasetImpl#recoverClose

2016-04-06 Thread Lei (Eddy) Xu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10267?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15229322#comment-15229322 ] Lei (Eddy) Xu commented on HDFS-10267: -- [~cmccabe] I have a question for the following code {code}

[jira] [Updated] (HDFS-10266) Remove unused properties dfs.client.file-block-storage-locations.num-threads and dfs.client.file-block-storage-locations.timeout.millis

2016-04-06 Thread Andrew Wang (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10266?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Wang updated HDFS-10266: --- Affects Version/s: (was: 2.7.3) 3.0.0 > Remove unused properties

[jira] [Updated] (HDFS-10266) Remove unused properties dfs.client.file-block-storage-locations.num-threads and dfs.client.file-block-storage-locations.timeout.millis

2016-04-06 Thread Andrew Wang (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10266?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Wang updated HDFS-10266: --- Labels: newbie (was: ) > Remove unused properties

[jira] [Updated] (HDFS-10266) Remove unused properties dfs.client.file-block-storage-locations.num-threads and dfs.client.file-block-storage-locations.timeout.millis

2016-04-06 Thread Andrew Wang (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10266?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Wang updated HDFS-10266: --- Issue Type: Improvement (was: Bug) > Remove unused properties

[jira] [Commented] (HDFS-10266) Remove unused properties dfs.client.file-block-storage-locations.num-threads and dfs.client.file-block-storage-locations.timeout.millis

2016-04-06 Thread Andrew Wang (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10266?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15229284#comment-15229284 ] Andrew Wang commented on HDFS-10266: Ah, you're talking about in trunk, not branch-2. I'll change the

[jira] [Updated] (HDFS-8791) block ID-based DN storage layout can be very slow for datanode on ext4

2016-04-06 Thread Andrew Wang (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8791?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Wang updated HDFS-8791: -- Target Version/s: 2.8.0 (was: 2.8.0, 2.6.5) > block ID-based DN storage layout can be very slow for

[jira] [Updated] (HDFS-8791) block ID-based DN storage layout can be very slow for datanode on ext4

2016-04-06 Thread Kihwal Lee (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8791?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kihwal Lee updated HDFS-8791: - Fix Version/s: (was: 2.7.3) 2.8.0 > block ID-based DN storage layout can be very

[jira] [Commented] (HDFS-9530) huge Non-DFS Used in hadoop 2.6.2 & 2.7.1

2016-04-06 Thread Arpit Agarwal (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9530?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15229271#comment-15229271 ] Arpit Agarwal commented on HDFS-9530: - Hi [~brahmareddy], I didn't quite understand the patch. Could

[jira] [Updated] (HDFS-8791) block ID-based DN storage layout can be very slow for datanode on ext4

2016-04-06 Thread Kihwal Lee (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8791?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kihwal Lee updated HDFS-8791: - Target Version/s: 2.8.0, 2.6.5 (was: 2.8.0, 2.7.3, 2.6.5) > block ID-based DN storage layout can be very

[jira] [Commented] (HDFS-8791) block ID-based DN storage layout can be very slow for datanode on ext4

2016-04-06 Thread Kihwal Lee (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8791?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15229270#comment-15229270 ] Kihwal Lee commented on HDFS-8791: -- Per discussion in the mailing lists and the comments in this jira, I

[jira] [Commented] (HDFS-10213) there is a corrupt/missing block report, but fsck result is HEALTHY

2016-04-06 Thread Javier Maestro (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10213?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15229233#comment-15229233 ] Javier Maestro commented on HDFS-10213: --- [~wyukawa] I think this other JIRA is be relevant:

[jira] [Commented] (HDFS-8533) Mismatch in displaying the "MissingBlock" count in fsck and in other metric reports

2016-04-06 Thread Javier Maestro (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8533?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15229221#comment-15229221 ] Javier Maestro commented on HDFS-8533: -- I've seen this issue as well, I'm running with a replication

[jira] [Commented] (HDFS-10267) Extra "synchronized" on FsDatasetImpl#recoverAppend and FsDatasetImpl#recoverClose

2016-04-06 Thread Colin Patrick McCabe (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10267?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15229216#comment-15229216 ] Colin Patrick McCabe commented on HDFS-10267: - v3 * Refactor {{testStopWorker}} so that it

[jira] [Commented] (HDFS-9735) DiskBalancer : Refactor moveBlockAcrossStorage to be used by disk balancer

2016-04-06 Thread Anu Engineer (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9735?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15229213#comment-15229213 ] Anu Engineer commented on HDFS-9735: That is a good question. This is how the current

[jira] [Updated] (HDFS-10267) Extra "synchronized" on FsDatasetImpl#recoverAppend and FsDatasetImpl#recoverClose

2016-04-06 Thread Colin Patrick McCabe (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10267?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Colin Patrick McCabe updated HDFS-10267: Attachment: HDFS-10267.003.patch > Extra "synchronized" on

[jira] [Commented] (HDFS-9735) DiskBalancer : Refactor moveBlockAcrossStorage to be used by disk balancer

2016-04-06 Thread Lei (Eddy) Xu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9735?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15229176#comment-15229176 ] Lei (Eddy) Xu commented on HDFS-9735: - Hi, [~anu] Thanks a lot for updating the patch. I have one

[jira] [Commented] (HDFS-10266) Remove unused properties dfs.client.file-block-storage-locations.num-threads and dfs.client.file-block-storage-locations.timeout.millis

2016-04-06 Thread Ray Chiang (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10266?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15229154#comment-15229154 ] Ray Chiang commented on HDFS-10266: --- Are you sure? Using grep and IntelliJ, I can find the definitions,

[jira] [Commented] (HDFS-8496) Calling stopWriter() with FSDatasetImpl lock held may block other threads

2016-04-06 Thread Wei-Chiu Chuang (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8496?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15229144#comment-15229144 ] Wei-Chiu Chuang commented on HDFS-8496: --- Link HDFS-10267, which founds/fixes a bug in HDFS-8496. >

[jira] [Commented] (HDFS-8356) Document missing properties in hdfs-default.xml

2016-04-06 Thread Ray Chiang (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8356?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15229128#comment-15229128 ] Ray Chiang commented on HDFS-8356: -- RE: failing unit tests TestHFlush is HDFS-2043 All other tests pass

[jira] [Commented] (HDFS-10266) Remove unused properties dfs.client.file-block-storage-locations.num-threads and dfs.client.file-block-storage-locations.timeout.millis

2016-04-06 Thread Andrew Wang (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10266?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15229127#comment-15229127 ] Andrew Wang commented on HDFS-10266: Hi Ray, these keys also used in DfsClientConf, and the config

[jira] [Commented] (HDFS-10267) Extra "synchronized" on FsDatasetImpl#recoverAppend and FsDatasetImpl#recoverClose

2016-04-06 Thread Wei-Chiu Chuang (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10267?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15229088#comment-15229088 ] Wei-Chiu Chuang commented on HDFS-10267: Thanks for the fix! Since the test catches

[jira] [Commented] (HDFS-9945) Datanode command for evicting writers

2016-04-06 Thread Hudson (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9945?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15229087#comment-15229087 ] Hudson commented on HDFS-9945: -- FAILURE: Integrated in Hadoop-trunk-Commit #9570 (See

[jira] [Commented] (HDFS-10267) Extra "synchronized" on FsDatasetImpl#recoverAppend and FsDatasetImpl#recoverClose

2016-04-06 Thread Lei (Eddy) Xu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10267?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15229077#comment-15229077 ] Lei (Eddy) Xu commented on HDFS-10267: -- One small nit, in {{rr2Thread.run()}} {code}

[jira] [Commented] (HDFS-10267) Extra "synchronized" on FsDatasetImpl#recoverAppend and FsDatasetImpl#recoverClose

2016-04-06 Thread Wei-Chiu Chuang (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10267?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15229078#comment-15229078 ] Wei-Chiu Chuang commented on HDFS-10267: One nitt: Instead of {code} LOG.debug("initiating

[jira] [Commented] (HDFS-8356) Document missing properties in hdfs-default.xml

2016-04-06 Thread Hadoop QA (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8356?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15229051#comment-15229051 ] Hadoop QA commented on HDFS-8356: - | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote || Subsystem ||

[jira] [Updated] (HDFS-10267) Extra "synchronized" on FsDatasetImpl#recoverAppend and FsDatasetImpl#recoverClose

2016-04-06 Thread Colin Patrick McCabe (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10267?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Colin Patrick McCabe updated HDFS-10267: Attachment: HDFS-10267.002.patch v2 * Correct misspelling of

[jira] [Commented] (HDFS-10267) Extra "synchronized" on FsDatasetImpl#recoverAppend and FsDatasetImpl#recoverClose

2016-04-06 Thread Andrew Wang (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10267?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15229030#comment-15229030 ] Andrew Wang commented on HDFS-10267: Nit: I see SOPT where I think you meant STOP. > Extra

[jira] [Commented] (HDFS-9945) Datanode command for evicting writers

2016-04-06 Thread Eric Payne (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9945?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15228984#comment-15228984 ] Eric Payne commented on HDFS-9945: -- Thanks, [~kihwal], for providing this feature. The patch looks good to

[jira] [Updated] (HDFS-10267) Extra "synchronized" on FsDatasetImpl#recoverAppend and FsDatasetImpl#recoverClose

2016-04-06 Thread Colin Patrick McCabe (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10267?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Colin Patrick McCabe updated HDFS-10267: Attachment: HDFS-10267.001.patch > Extra "synchronized" on

[jira] [Updated] (HDFS-6520) hdfs fsck -move passes invalid length value when creating BlockReader

2016-04-06 Thread Colin Patrick McCabe (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-6520?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Colin Patrick McCabe updated HDFS-6520: --- Summary: hdfs fsck -move passes invalid length value when creating BlockReader (was:

[jira] [Updated] (HDFS-10267) Extra "synchronized" on FsDatasetImpl#recoverAppend and FsDatasetImpl#recoverClose

2016-04-06 Thread Colin Patrick McCabe (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10267?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Colin Patrick McCabe updated HDFS-10267: Attachment: (was: HDFS-10267.001.patch) > Extra "synchronized" on

[jira] [Updated] (HDFS-10267) Extra "synchronized" on FsDatasetImpl#recoverAppend and FsDatasetImpl#recoverClose

2016-04-06 Thread Colin Patrick McCabe (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10267?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Colin Patrick McCabe updated HDFS-10267: Attachment: HDFS-10267.001.patch Remove the extra synchronization, and add unit

[jira] [Updated] (HDFS-10267) Extra "synchronized" on FsDatasetImpl#recoverAppend and FsDatasetImpl#recoverClose

2016-04-06 Thread Colin Patrick McCabe (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10267?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Colin Patrick McCabe updated HDFS-10267: Status: Patch Available (was: Open) > Extra "synchronized" on

[jira] [Commented] (HDFS-6520) hdfs fsck passes invalid length value when creating BlockReader

2016-04-06 Thread Hudson (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-6520?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15228940#comment-15228940 ] Hudson commented on HDFS-6520: -- FAILURE: Integrated in Hadoop-trunk-Commit #9569 (See

[jira] [Created] (HDFS-10267) Extra "synchronized" on FsDatasetImpl#recoverAppend and FsDatasetImpl#recoverClose

2016-04-06 Thread Colin Patrick McCabe (JIRA)
Colin Patrick McCabe created HDFS-10267: --- Summary: Extra "synchronized" on FsDatasetImpl#recoverAppend and FsDatasetImpl#recoverClose Key: HDFS-10267 URL: https://issues.apache.org/jira/browse/HDFS-10267

[jira] [Updated] (HDFS-6520) hdfs fsck passes invalid length value when creating BlockReader

2016-04-06 Thread Colin Patrick McCabe (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-6520?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Colin Patrick McCabe updated HDFS-6520: --- Resolution: Fixed Fix Version/s: 2.8.0 Target Version/s: 2.8.0

[jira] [Commented] (HDFS-6520) hdfs fsck passes invalid length value when creating BlockReader

2016-04-06 Thread Xiao Chen (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-6520?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15228843#comment-15228843 ] Xiao Chen commented on HDFS-6520: - Thanks very much [~cmccabe]! Maybe we can update the title to s/hdfs

[jira] [Updated] (HDFS-6520) hdfs fsck passes invalid length value when creating BlockReader

2016-04-06 Thread Colin Patrick McCabe (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-6520?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Colin Patrick McCabe updated HDFS-6520: --- Summary: hdfs fsck passes invalid length value when creating BlockReader (was: hdfs

[jira] [Commented] (HDFS-6520) hdfs fsck -move not working

2016-04-06 Thread Colin Patrick McCabe (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-6520?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15228832#comment-15228832 ] Colin Patrick McCabe commented on HDFS-6520: +1. Thanks, [~xiaochen]. > hdfs fsck -move not

[jira] [Updated] (HDFS-10266) Remove unused properties dfs.client.file-block-storage-locations.num-threads and dfs.client.file-block-storage-locations.timeout.millis

2016-04-06 Thread Ray Chiang (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10266?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ray Chiang updated HDFS-10266: -- Description: The properties: dfs.client.file-block-storage-locations.num-threads

[jira] [Commented] (HDFS-10192) Namenode safemode not coming out during failover

2016-04-06 Thread Hudson (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10192?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15228781#comment-15228781 ] Hudson commented on HDFS-10192: --- FAILURE: Integrated in Hadoop-trunk-Commit #9568 (See

[jira] [Assigned] (HDFS-10264) Logging improvements in FSImageFormatProtobuf.Saver

2016-04-06 Thread Xiaobing Zhou (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10264?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiaobing Zhou reassigned HDFS-10264: Assignee: Xiaobing Zhou > Logging improvements in FSImageFormatProtobuf.Saver >

[jira] [Updated] (HDFS-10192) Namenode safemode not coming out during failover

2016-04-06 Thread Jing Zhao (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10192?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jing Zhao updated HDFS-10192: - Resolution: Fixed Hadoop Flags: Reviewed Fix Version/s: 2.9.0 Status: Resolved

[jira] [Commented] (HDFS-9782) RollingFileSystemSink should have configurable roll interval

2016-04-06 Thread Daniel Templeton (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9782?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15228683#comment-15228683 ] Daniel Templeton commented on HDFS-9782: Test failures are all unrelated. > RollingFileSystemSink

[jira] [Commented] (HDFS-9782) RollingFileSystemSink should have configurable roll interval

2016-04-06 Thread Hadoop QA (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9782?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15228662#comment-15228662 ] Hadoop QA commented on HDFS-9782: - | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote || Subsystem ||

[jira] [Commented] (HDFS-8791) block ID-based DN storage layout can be very slow for datanode on ext4

2016-04-06 Thread Chris Nauroth (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8791?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15228640#comment-15228640 ] Chris Nauroth commented on HDFS-8791: - I am struggling to understand all of the possible permutations

[jira] [Commented] (HDFS-9732) Remove DelegationTokenIdentifier.toString() —for better logging output

2016-04-06 Thread Yongjun Zhang (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9732?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15228622#comment-15228622 ] Yongjun Zhang commented on HDFS-9732: - Hi [~ste...@apache.org], [~cnauroth], [~aw], Wonder if you

[jira] [Commented] (HDFS-8356) Document missing properties in hdfs-default.xml

2016-04-06 Thread Ray Chiang (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8356?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15228620#comment-15228620 ] Ray Chiang commented on HDFS-8356: -- Filed HDFS-10266 as a follow up. > Document missing properties in

  1   2   >