[jira] [Updated] (HDFS-9809) Abstract implementation-specific details from the datanode

2016-08-31 Thread Virajith Jalaparti (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9809?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Virajith Jalaparti updated HDFS-9809: - Attachment: DFSIO_benchmark_HDFS-9809.docx The doc attached shows the results of running

[jira] [Commented] (HDFS-10819) BlockManager fails to store a good block for a datanode storage after it reported a corrupt block — block replication stuck

2016-08-31 Thread Hadoop QA (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10819?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15453300#comment-15453300 ] Hadoop QA commented on HDFS-10819: -- | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote || Subsystem ||

[jira] [Commented] (HDFS-10609) Uncaught InvalidEncryptionKeyException during pipeline recovery may abort downstream applications

2016-08-31 Thread Andrew Wang (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10609?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15453380#comment-15453380 ] Andrew Wang commented on HDFS-10609: Looks pretty good, but a comment about the retry policy class.

[jira] [Updated] (HDFS-10742) Measurement of lock held time in FsDatasetImpl

2016-08-31 Thread Chen Liang (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10742?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chen Liang updated HDFS-10742: -- Attachment: HDFS-10742.011.patch > Measurement of lock held time in FsDatasetImpl >

[jira] [Updated] (HDFS-10742) Measurement of lock held time in FsDatasetImpl

2016-08-31 Thread Chen Liang (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10742?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chen Liang updated HDFS-10742: -- Attachment: HDFS-10742.011.patch > Measurement of lock held time in FsDatasetImpl >

[jira] [Updated] (HDFS-10742) Measurement of lock held time in FsDatasetImpl

2016-08-31 Thread Chen Liang (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10742?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chen Liang updated HDFS-10742: -- Attachment: (was: HDFS-10742.011.patch) > Measurement of lock held time in FsDatasetImpl >

[jira] [Commented] (HDFS-10729) Improve log message for edit loading failures caused by FS limit checks.

2016-08-31 Thread Wei-Chiu Chuang (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10729?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15453288#comment-15453288 ] Wei-Chiu Chuang commented on HDFS-10729: Thanks [~yzhangal] for the suggestion of improving log

[jira] [Commented] (HDFS-10819) BlockManager fails to store a good block for a datanode storage after it reported a corrupt block — block replication stuck

2016-08-31 Thread Manoj Govindassamy (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10819?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15453308#comment-15453308 ] Manoj Govindassamy commented on HDFS-10819: --- Unit test failure is not related to the patch. Have

[jira] [Updated] (HDFS-10817) Add Logging for Long-held NN Read Locks

2016-08-31 Thread Erik Krogen (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10817?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Erik Krogen updated HDFS-10817: --- Attachment: HDFS-10817.004.patch > Add Logging for Long-held NN Read Locks >

[jira] [Commented] (HDFS-10784) Implement WebHdfsFileSystem#listStatusIterator

2016-08-31 Thread Hudson (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10784?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15453497#comment-15453497 ] Hudson commented on HDFS-10784: --- SUCCESS: Integrated in Jenkins build Hadoop-trunk-Commit #10380 (See

[jira] [Commented] (HDFS-10742) Measurement of lock held time in FsDatasetImpl

2016-08-31 Thread Chen Liang (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10742?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15453528#comment-15453528 ] Chen Liang commented on HDFS-10742: --- Thanks [~arpitagarwal] for the review! Updated a patch to reflect

[jira] [Commented] (HDFS-10674) Optimize creating a full path from an inode

2016-08-31 Thread Zhe Zhang (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10674?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15453591#comment-15453591 ] Zhe Zhang commented on HDFS-10674: -- I just committed to branch-2.8 and branch-2.7. Thanks Daryn for

[jira] [Updated] (HDFS-10674) Optimize creating a full path from an inode

2016-08-31 Thread Zhe Zhang (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10674?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhe Zhang updated HDFS-10674: - Fix Version/s: 2.7.4 2.8.0 > Optimize creating a full path from an inode >

[jira] [Commented] (HDFS-10822) Log DataNodes in the write pipeline

2016-08-31 Thread John Zhuge (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10822?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15453441#comment-15453441 ] John Zhuge commented on HDFS-10822: --- Thanks [~kihwal] for bringing up HDFS-8791. > Log DataNodes in the

[jira] [Commented] (HDFS-10713) Throttle FsNameSystem lock warnings

2016-08-31 Thread Jing Zhao (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10713?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15453607#comment-15453607 ] Jing Zhao commented on HDFS-10713: -- bq. we have a look at the feasibility to expose this information to

[jira] [Commented] (HDFS-10817) Add Logging for Long-held NN Read Locks

2016-08-31 Thread Erik Krogen (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10817?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15453363#comment-15453363 ] Erik Krogen commented on HDFS-10817: It is true that you would see an NPE. I initially ignored this

[jira] [Commented] (HDFS-8901) Use ByteBuffer in striping positional read

2016-08-31 Thread Zhe Zhang (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8901?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15453470#comment-15453470 ] Zhe Zhang commented on HDFS-8901: - Thanks Sammi for the update! Sorry it's taking me long to review. I

[jira] [Commented] (HDFS-10817) Add Logging for Long-held NN Read Locks

2016-08-31 Thread Hadoop QA (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10817?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15453602#comment-15453602 ] Hadoop QA commented on HDFS-10817: -- | (/) *{color:green}+1 overall{color}* | \\ \\ || Vote || Subsystem

[jira] [Commented] (HDFS-10800) [SPS]: Storage Policy Satisfier daemon thread in Namenode to find the blocks which were placed in other storages than what NN is expecting.

2016-08-31 Thread Hadoop QA (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10800?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15453254#comment-15453254 ] Hadoop QA commented on HDFS-10800: -- | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote || Subsystem ||

[jira] [Updated] (HDFS-10713) Throttle FsNameSystem lock warnings

2016-08-31 Thread Hanisha Koneru (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10713?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hanisha Koneru updated HDFS-10713: -- Attachment: HDFS-10713.002.patch Thanks [~arpitagarwal] for the review. I have made the above

[jira] [Commented] (HDFS-10817) Add Logging for Long-held NN Read Locks

2016-08-31 Thread Zhe Zhang (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10817?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15453500#comment-15453500 ] Zhe Zhang commented on HDFS-10817: -- Thanks Erik. +1 on v4 patch. I'll commit after Jenkins. > Add

[jira] [Updated] (HDFS-10784) Implement WebHdfsFileSystem#listStatusIterator

2016-08-31 Thread Andrew Wang (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10784?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Wang updated HDFS-10784: --- Resolution: Fixed Fix Version/s: 3.0.0-alpha2 2.9.0 Status:

[jira] [Updated] (HDFS-7766) Add a flag to WebHDFS op=CREATE to not respond with a 307 redirect

2016-08-31 Thread Andrew Wang (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-7766?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Wang updated HDFS-7766: -- Fix Version/s: 2.9.0 I've backported this to branch-2 as well, clean pick. > Add a flag to WebHDFS

[jira] [Comment Edited] (HDFS-10713) Throttle FsNameSystem lock warnings

2016-08-31 Thread Mingliang Liu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10713?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15453359#comment-15453359 ] Mingliang Liu edited comment on HDFS-10713 at 8/31/16 9:42 PM: --- One concern

[jira] [Updated] (HDFS-10742) Measurement of lock held time in FsDatasetImpl

2016-08-31 Thread Chen Liang (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10742?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chen Liang updated HDFS-10742: -- Attachment: (was: HDFS-10742.011.patch) > Measurement of lock held time in FsDatasetImpl >

[jira] [Commented] (HDFS-10822) Log DataNodes in the write pipeline

2016-08-31 Thread Kihwal Lee (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10822?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15453244#comment-15453244 ] Kihwal Lee commented on HDFS-10822: --- bq. Trying to diagnose a slow HDFS flush, taking longer than 10

[jira] [Commented] (HDFS-10656) Optimize conversion of byte arrays back to path string

2016-08-31 Thread Zhe Zhang (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10656?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15453578#comment-15453578 ] Zhe Zhang commented on HDFS-10656: -- I just committed the change to branch-2.8 and branch-2.7. > Optimize

[jira] [Updated] (HDFS-10656) Optimize conversion of byte arrays back to path string

2016-08-31 Thread Zhe Zhang (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10656?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhe Zhang updated HDFS-10656: - Fix Version/s: 2.7.4 2.8.0 > Optimize conversion of byte arrays back to path string >

[jira] [Commented] (HDFS-10713) Throttle FsNameSystem lock warnings

2016-08-31 Thread Mingliang Liu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10713?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15453359#comment-15453359 ] Mingliang Liu commented on HDFS-10713: -- {code:title=DFSConfigKeys.java} 414 // Threshold for

[jira] [Updated] (HDFS-10742) Measurement of lock held time in FsDatasetImpl

2016-08-31 Thread Chen Liang (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10742?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chen Liang updated HDFS-10742: -- Attachment: HDFS-10742.011.patch > Measurement of lock held time in FsDatasetImpl >

[jira] [Reopened] (HDFS-10662) Optimize UTF8 string/byte conversions

2016-08-31 Thread Zhe Zhang (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10662?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhe Zhang reopened HDFS-10662: -- Sorry to reopen the JIRA. I'm backporting to branch-2.7 and it was quite messy. [~daryn] [~kihwal] I'd

[jira] [Updated] (HDFS-10662) Optimize UTF8 string/byte conversions

2016-08-31 Thread Zhe Zhang (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10662?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhe Zhang updated HDFS-10662: - Status: Patch Available (was: Reopened) > Optimize UTF8 string/byte conversions >

[jira] [Updated] (HDFS-10824) MiniDFSCluster#storageCapacities has no effects on real capacity

2016-08-31 Thread Xiaobing Zhou (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10824?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiaobing Zhou updated HDFS-10824: - Description: It has been noticed MiniDFSCluster#storageCapacities has no effects on real

[jira] [Updated] (HDFS-10662) Optimize UTF8 string/byte conversions

2016-08-31 Thread Zhe Zhang (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10662?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhe Zhang updated HDFS-10662: - Attachment: HDFS-10662-branch-2.7.00.patch > Optimize UTF8 string/byte conversions >

[jira] [Updated] (HDFS-10824) MiniDFSCluster#storageCapacities has no effects on real capacity

2016-08-31 Thread Xiaobing Zhou (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10824?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiaobing Zhou updated HDFS-10824: - Description: It has been noticed MiniDFSCluster#storageCapacities has no effects on real

[jira] [Commented] (HDFS-9781) FsDatasetImpl#getBlockReports can occasionally throw NullPointerException

2016-08-31 Thread Manoj Govindassamy (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9781?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15454036#comment-15454036 ] Manoj Govindassamy commented on HDFS-9781: -- Ref: {code:title=FsDatasetImpl.java|borderStyle=solid}

[jira] [Commented] (HDFS-10817) Add Logging for Long-held NN Read Locks

2016-08-31 Thread Erik Krogen (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10817?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15453654#comment-15453654 ] Erik Krogen commented on HDFS-10817: Great, thanks Zhe! > Add Logging for Long-held NN Read Locks >

[jira] [Commented] (HDFS-10817) Add Logging for Long-held NN Read Locks

2016-08-31 Thread Hudson (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10817?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15453650#comment-15453650 ] Hudson commented on HDFS-10817: --- SUCCESS: Integrated in Jenkins build Hadoop-trunk-Commit #10381 (See

[jira] [Created] (HDFS-10824) MiniDFSCluster#storageCapacities has no effects on real capacity

2016-08-31 Thread Xiaobing Zhou (JIRA)
Xiaobing Zhou created HDFS-10824: Summary: MiniDFSCluster#storageCapacities has no effects on real capacity Key: HDFS-10824 URL: https://issues.apache.org/jira/browse/HDFS-10824 Project: Hadoop HDFS

[jira] [Updated] (HDFS-10824) MiniDFSCluster#storageCapacities has no effects on real capacity

2016-08-31 Thread Xiaobing Zhou (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10824?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiaobing Zhou updated HDFS-10824: - Description: It has been noticed MiniDFSCluster# > MiniDFSCluster#storageCapacities has no

[jira] [Commented] (HDFS-10823) Implement HttpFSFileSystem#listStatusIterator

2016-08-31 Thread Andrew Wang (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10823?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15453852#comment-15453852 ] Andrew Wang commented on HDFS-10823: Thinking about it a little more, what we'd like is a generic way

[jira] [Commented] (HDFS-10817) Add Logging for Long-held NN Read Locks

2016-08-31 Thread Hadoop QA (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10817?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15454032#comment-15454032 ] Hadoop QA commented on HDFS-10817: -- | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote || Subsystem ||

[jira] [Updated] (HDFS-10824) MiniDFSCluster#storageCapacities has no effects on real capacity

2016-08-31 Thread Xiaobing Zhou (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10824?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiaobing Zhou updated HDFS-10824: - Description: It has been noticed MiniDFSCluster#storageCapacities has no effects on real

[jira] [Commented] (HDFS-9847) HDFS configuration without time unit name should accept friendly time units

2016-08-31 Thread Yiqun Lin (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9847?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15454104#comment-15454104 ] Yiqun Lin commented on HDFS-9847: - I have checked the code, it looks good, pending jenkins. > HDFS

[jira] [Commented] (HDFS-7878) API - expose an unique file identifier

2016-08-31 Thread Hadoop QA (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-7878?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15454081#comment-15454081 ] Hadoop QA commented on HDFS-7878: - | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote || Subsystem ||

[jira] [Comment Edited] (HDFS-10713) Throttle FsNameSystem lock warnings

2016-08-31 Thread Jing Zhao (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10713?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15453607#comment-15453607 ] Jing Zhao edited comment on HDFS-10713 at 8/31/16 10:40 PM: bq. we have a look

[jira] [Commented] (HDFS-10823) Implement HttpFSFileSystem#listStatusIterator

2016-08-31 Thread Andrew Wang (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10823?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15453738#comment-15453738 ] Andrew Wang commented on HDFS-10823: Looking at the HttpFS code, it's a bit tricky to do this. HttpFS

[jira] [Updated] (HDFS-7878) API - expose an unique file identifier

2016-08-31 Thread Chris Douglas (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-7878?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chris Douglas updated HDFS-7878: Attachment: HDFS-7878.06.patch Attaching patch for discussion. This rebases v05 with the following

[jira] [Commented] (HDFS-7878) API - expose an unique file identifier

2016-08-31 Thread Sergey Shelukhin (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-7878?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15453828#comment-15453828 ] Sergey Shelukhin commented on HDFS-7878: One idea behind open(long/InodeId) is to be able to open

[jira] [Commented] (HDFS-10713) Throttle FsNameSystem lock warnings

2016-08-31 Thread Hanisha Koneru (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10713?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15453832#comment-15453832 ] Hanisha Koneru commented on HDFS-10713: --- We can cache the thread's stackTrace and lock time and once

[jira] [Comment Edited] (HDFS-7878) API - expose an unique file identifier

2016-08-31 Thread Sergey Shelukhin (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-7878?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15453828#comment-15453828 ] Sergey Shelukhin edited comment on HDFS-7878 at 9/1/16 12:13 AM: - One idea

[jira] [Commented] (HDFS-7878) API - expose an unique file identifier

2016-08-31 Thread Chris Douglas (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-7878?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15453857#comment-15453857 ] Chris Douglas commented on HDFS-7878: - bq. One idea behind open(long/InodeId) is to be able to open

[jira] [Comment Edited] (HDFS-10817) Add Logging for Long-held NN Read Locks

2016-08-31 Thread Erik Krogen (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10817?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15453863#comment-15453863 ] Erik Krogen edited comment on HDFS-10817 at 9/1/16 12:32 AM: - Discovered that

[jira] [Updated] (HDFS-10817) Add Logging for Long-held NN Read Locks

2016-08-31 Thread Erik Krogen (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10817?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Erik Krogen updated HDFS-10817: --- Attachment: HDFS-10817.addendum.000.patch Discovered that the test fails on branch-2 because the logs

[jira] [Commented] (HDFS-10662) Optimize UTF8 string/byte conversions

2016-08-31 Thread Hadoop QA (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10662?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15454066#comment-15454066 ] Hadoop QA commented on HDFS-10662: -- | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote || Subsystem ||

[jira] [Commented] (HDFS-9781) FsDatasetImpl#getBlockReports can occasionally throw NullPointerException

2016-08-31 Thread Manoj Govindassamy (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9781?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15453821#comment-15453821 ] Manoj Govindassamy commented on HDFS-9781: -- Ref: {noformat} @Override public

[jira] [Updated] (HDFS-9847) HDFS configuration without time unit name should accept friendly time units

2016-08-31 Thread Chris Douglas (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9847?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chris Douglas updated HDFS-9847: Attachment: HDFS-9847.007.patch Rebased nothrow.004 as .007 and added a unit test for the warnings

[jira] [Created] (HDFS-10825) Snapshot read can reveal future bytes if snapshotted while writing

2016-08-31 Thread Abhishek Rai (JIRA)
Abhishek Rai created HDFS-10825: --- Summary: Snapshot read can reveal future bytes if snapshotted while writing Key: HDFS-10825 URL: https://issues.apache.org/jira/browse/HDFS-10825 Project: Hadoop HDFS

[jira] [Updated] (HDFS-10825) Snapshot read can reveal future bytes if snapshotted while writing

2016-08-31 Thread Abhishek Rai (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10825?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Abhishek Rai updated HDFS-10825: Attachment: TestSnapshotFileBeingWritten.java > Snapshot read can reveal future bytes if

[jira] [Created] (HDFS-10823) Implement HttpFSFileSystem#listStatusIterator

2016-08-31 Thread Andrew Wang (JIRA)
Andrew Wang created HDFS-10823: -- Summary: Implement HttpFSFileSystem#listStatusIterator Key: HDFS-10823 URL: https://issues.apache.org/jira/browse/HDFS-10823 Project: Hadoop HDFS Issue Type:

[jira] [Updated] (HDFS-10729) Improve log message for edit loading failures caused by FS limit checks.

2016-08-31 Thread Yongjun Zhang (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10729?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yongjun Zhang updated HDFS-10729: - Labels: supportability (was: ) > Improve log message for edit loading failures caused by FS

[jira] [Commented] (HDFS-9847) HDFS configuration without time unit name should accept friendly time units

2016-08-31 Thread Yiqun Lin (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9847?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15454026#comment-15454026 ] Yiqun Lin commented on HDFS-9847: - Thanks a lot for updating the patch, [~chris.douglas]! The latest patch

[jira] [Commented] (HDFS-9847) HDFS configuration without time unit name should accept friendly time units

2016-08-31 Thread Hadoop QA (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9847?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15454185#comment-15454185 ] Hadoop QA commented on HDFS-9847: - | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote || Subsystem ||

[jira] [Commented] (HDFS-10817) Add Logging for Long-held NN Read Locks

2016-08-31 Thread Zhe Zhang (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10817?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15453636#comment-15453636 ] Zhe Zhang commented on HDFS-10817: -- BTW backporting to lower branches hit this issue:

[jira] [Commented] (HDFS-7878) API - expose an unique file identifier

2016-08-31 Thread Sergey Shelukhin (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-7878?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15453830#comment-15453830 ] Sergey Shelukhin commented on HDFS-7878: Thanks for picking this up by the way! > API - expose an

[jira] [Updated] (HDFS-9668) Optimize the locking in FsDatasetImpl

2016-08-31 Thread Jingcheng Du (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9668?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jingcheng Du updated HDFS-9668: --- Status: Patch Available (was: Open) > Optimize the locking in FsDatasetImpl >

[jira] [Updated] (HDFS-9668) Optimize the locking in FsDatasetImpl

2016-08-31 Thread Jingcheng Du (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9668?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jingcheng Du updated HDFS-9668: --- Status: Open (was: Patch Available) > Optimize the locking in FsDatasetImpl >

[jira] [Updated] (HDFS-10800) [SPS]: Storage Policy Satisfier daemon thread in Namenode to find the blocks which were placed in other storages than what NN is expecting.

2016-08-31 Thread Uma Maheswara Rao G (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10800?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Uma Maheswara Rao G updated HDFS-10800: --- Attachment: HDFS-10800-HDFS-10285-01.patch Tests were passed in my local eclipse.

[jira] [Assigned] (HDFS-10825) Snapshot read can reveal future bytes if snapshotted while writing

2016-08-31 Thread Manoj Govindassamy (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10825?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Manoj Govindassamy reassigned HDFS-10825: - Assignee: Manoj Govindassamy > Snapshot read can reveal future bytes if

[jira] [Commented] (HDFS-10609) Uncaught InvalidEncryptionKeyException during pipeline recovery may abort downstream applications

2016-08-31 Thread Wei-Chiu Chuang (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10609?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15452812#comment-15452812 ] Wei-Chiu Chuang commented on HDFS-10609: Thanks [~xiaochen] and [~andrew.wang] for reviewing the

[jira] [Updated] (HDFS-10609) Uncaught InvalidEncryptionKeyException during pipeline recovery may abort downstream applications

2016-08-31 Thread Wei-Chiu Chuang (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10609?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wei-Chiu Chuang updated HDFS-10609: --- Attachment: HDFS-10609.003.patch Oops. Forgot to upload the patch. Here's the patch v03.

[jira] [Commented] (HDFS-10609) Uncaught InvalidEncryptionKeyException during pipeline recovery may abort downstream applications

2016-08-31 Thread Wei-Chiu Chuang (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10609?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15452842#comment-15452842 ] Wei-Chiu Chuang commented on HDFS-10609: To correct myself -- The reason

[jira] [Commented] (HDFS-10713) Throttle FsNameSystem lock warnings

2016-08-31 Thread Arpit Agarwal (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10713?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15452881#comment-15452881 ] Arpit Agarwal commented on HDFS-10713: -- Also we can rename the config key to

[jira] [Commented] (HDFS-10817) Add Logging for Long-held NN Read Locks

2016-08-31 Thread Zhe Zhang (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10817?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15452844#comment-15452844 ] Zhe Zhang commented on HDFS-10817: -- Thanks Erik for the updates. I'm not very familiar with

[jira] [Updated] (HDFS-10638) Modifications to remove the assumption that StorageLocation is associated with java.io.File.

2016-08-31 Thread Virajith Jalaparti (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10638?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Virajith Jalaparti updated HDFS-10638: -- Attachment: HDFS-10638.003.patch Posting an updated patch to work with the most recent

[jira] [Comment Edited] (HDFS-10609) Uncaught InvalidEncryptionKeyException during pipeline recovery may abort downstream applications

2016-08-31 Thread Wei-Chiu Chuang (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10609?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15452812#comment-15452812 ] Wei-Chiu Chuang edited comment on HDFS-10609 at 8/31/16 5:25 PM: - Thanks

[jira] [Commented] (HDFS-10742) Measurement of lock held time in FsDatasetImpl

2016-08-31 Thread Arpit Agarwal (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10742?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15452877#comment-15452877 ] Arpit Agarwal commented on HDFS-10742: -- Thanks [~vagarychen]. The change lgtm. A few minor comments:

[jira] [Updated] (HDFS-10637) Modifications to remove the assumption that FsVolumes are backed by java.io.File.

2016-08-31 Thread Virajith Jalaparti (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10637?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Virajith Jalaparti updated HDFS-10637: -- Attachment: HDFS-10637.006.patch Posting an updated patch (to work with the most recent

[jira] [Commented] (HDFS-10742) Measurement of lock held time in FsDatasetImpl

2016-08-31 Thread Hadoop QA (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10742?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15453065#comment-15453065 ] Hadoop QA commented on HDFS-10742: -- | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote || Subsystem ||

[jira] [Commented] (HDFS-10817) Add Logging for Long-held NN Read Locks

2016-08-31 Thread Hadoop QA (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10817?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15453069#comment-15453069 ] Hadoop QA commented on HDFS-10817: -- | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote || Subsystem ||

[jira] [Updated] (HDFS-10729) Improve log message for edit loading failures caused by FS limit checks.

2016-08-31 Thread Kihwal Lee (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10729?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kihwal Lee updated HDFS-10729: -- Summary: Improve log message for edit loading failures caused by FS limit checks. (was: NameNode

[jira] [Updated] (HDFS-10819) BlockManager fails to store a good block for a datanode storage after it reported a corrupt block — block replication stuck

2016-08-31 Thread Manoj Govindassamy (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10819?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Manoj Govindassamy updated HDFS-10819: -- Attachment: HDFS-10819.001.patch *Problem:* — BlockManager reports incorrect replica

[jira] [Commented] (HDFS-10729) Improve log message for edit loading failures caused by FS limit checks.

2016-08-31 Thread Kihwal Lee (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10729?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15453084#comment-15453084 ] Kihwal Lee commented on HDFS-10729: --- +1 the patch looks good. > Improve log message for edit loading

[jira] [Updated] (HDFS-10819) BlockManager fails to store a good block for a datanode storage after it reported a corrupt block — block replication stuck

2016-08-31 Thread Manoj Govindassamy (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10819?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Manoj Govindassamy updated HDFS-10819: -- Status: Patch Available (was: Open) > BlockManager fails to store a good block for a

[jira] [Updated] (HDFS-10729) Improve log message for edit loading failures caused by FS limit checks.

2016-08-31 Thread Kihwal Lee (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10729?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kihwal Lee updated HDFS-10729: -- Priority: Major (was: Critical) > Improve log message for edit loading failures caused by FS limit

[jira] [Commented] (HDFS-10729) Improve log message for edit loading failures caused by FS limit checks.

2016-08-31 Thread Hudson (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10729?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15453121#comment-15453121 ] Hudson commented on HDFS-10729: --- SUCCESS: Integrated in Jenkins build Hadoop-trunk-Commit #10379 (See

[jira] [Created] (HDFS-10822) Log DataNodes in the write pipeline

2016-08-31 Thread John Zhuge (JIRA)
John Zhuge created HDFS-10822: - Summary: Log DataNodes in the write pipeline Key: HDFS-10822 URL: https://issues.apache.org/jira/browse/HDFS-10822 Project: Hadoop HDFS Issue Type: Improvement

[jira] [Updated] (HDFS-10729) Improve log message for edit loading failures caused by FS limit checks.

2016-08-31 Thread Kihwal Lee (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10729?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kihwal Lee updated HDFS-10729: -- Resolution: Fixed Hadoop Flags: Reviewed Fix Version/s: 3.0.0-alpha2

[jira] [Commented] (HDFS-10729) NameNode crashes when loading edits because max directory items is exceeded

2016-08-31 Thread Kihwal Lee (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10729?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15452990#comment-15452990 ] Kihwal Lee commented on HDFS-10729: --- They pass for me too. {noformat}

[jira] [Updated] (HDFS-10800) [SPS]: Storage Policy Satisfier daemon thread in Namenode to find the blocks which were placed in other storages than what NN is expecting.

2016-08-31 Thread Uma Maheswara Rao G (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10800?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Uma Maheswara Rao G updated HDFS-10800: --- Status: Patch Available (was: Open) > [SPS]: Storage Policy Satisfier daemon thread

[jira] [Commented] (HDFS-10819) BlockManager fails to store a good block for a datanode storage after it reported a corrupt block — block replication stuck

2016-08-31 Thread Manoj Govindassamy (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10819?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15452935#comment-15452935 ] Manoj Govindassamy commented on HDFS-10819: --- Here is the log snippet from a failed test which

[jira] [Commented] (HDFS-10777) DataNode should report volume failures if DU cannot access files

2016-08-31 Thread Wei-Chiu Chuang (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10777?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15452933#comment-15452933 ] Wei-Chiu Chuang commented on HDFS-10777: Interesting observation! [~steve_l] I've not thought

[jira] [Commented] (HDFS-10609) Uncaught InvalidEncryptionKeyException during pipeline recovery may abort downstream applications

2016-08-31 Thread Hadoop QA (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10609?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15453052#comment-15453052 ] Hadoop QA commented on HDFS-10609: -- | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote || Subsystem ||

[jira] [Comment Edited] (HDFS-10820) Reuse closeResponder to reset the response variable in DataStreamer#run

2016-08-31 Thread Xiao Chen (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10820?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15451307#comment-15451307 ] Xiao Chen edited comment on HDFS-10820 at 8/31/16 6:23 AM: --- +1, will commit

[jira] [Updated] (HDFS-10820) DataStreamer#Responder close fails that break down the pipeline recovery

2016-08-31 Thread Yiqun Lin (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10820?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yiqun Lin updated HDFS-10820: - Summary: DataStreamer#Responder close fails that break down the pipeline recovery (was:

[jira] [Updated] (HDFS-10820) Reuse closeResponder to reset the response variable in DataStreamer#run

2016-08-31 Thread Xiao Chen (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10820?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Chen updated HDFS-10820: - Summary: Reuse closeResponder to reset the response variable in DataStreamer#run (was:

[jira] [Updated] (HDFS-9668) Optimize the locking in FsDatasetImpl

2016-08-31 Thread Jingcheng Du (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9668?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jingcheng Du updated HDFS-9668: --- Attachment: HDFS-9668-6.patch Rebase and upload the new patch V6 to fix the merging issues.

[jira] [Commented] (HDFS-10426) TestPendingInvalidateBlock failed in trunk

2016-08-31 Thread Hadoop QA (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10426?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15451268#comment-15451268 ] Hadoop QA commented on HDFS-10426: -- | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote || Subsystem ||

[jira] [Commented] (HDFS-10820) DataStreamer#Responder close fails that break down the pipeline recovery

2016-08-31 Thread Hadoop QA (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10820?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15451293#comment-15451293 ] Hadoop QA commented on HDFS-10820: -- | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote || Subsystem ||

[jira] [Commented] (HDFS-10820) Reuse closeResponder to reset the response variable in DataStreamer#run

2016-08-31 Thread Xiao Chen (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10820?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15451307#comment-15451307 ] Xiao Chen commented on HDFS-10820: -- +1, will commit after 24 hours. > Reuse closeResponder to reset the

  1   2   >