[jira] [Commented] (HDFS-4115) TestHDFSCLI.testAll fails one test due to number format

2016-02-02 Thread Tony Wu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-4115?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15128540#comment-15128540 ] Tony Wu commented on HDFS-4115: --- Hi [~sureshms], [~jingzhao] & [~scurrilous], I ran into this error when

[jira] [Commented] (HDFS-9493) Test o.a.h.hdfs.server.namenode.TestMetaSave fails in trunk

2016-01-11 Thread Tony Wu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9493?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15092146#comment-15092146 ] Tony Wu commented on HDFS-9493: --- Thanks [~liuml07] & [~eddyxu] for the review and comments! > Test

[jira] [Commented] (HDFS-9493) Test o.a.h.hdfs.server.namenode.TestMetaSave fails in trunk

2016-01-04 Thread Tony Wu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9493?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15081284#comment-15081284 ] Tony Wu commented on HDFS-9493: --- Hi [~eddyxu], Could you take a look at the latest patch (v3) and let me

[jira] [Commented] (HDFS-9493) Test o.a.h.hdfs.server.namenode.TestMetaSave fails in trunk

2015-12-29 Thread Tony Wu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9493?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15074430#comment-15074430 ] Tony Wu commented on HDFS-9493: --- The test failures are not related to this patch as this patch only affects

[jira] [Updated] (HDFS-9493) Test o.a.h.hdfs.server.namenode.TestMetaSave fails in trunk

2015-12-18 Thread Tony Wu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9493?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tony Wu updated HDFS-9493: -- Attachment: HDFS-9493.003.patch In v3 patch: * Addressed [~eddyxu]'s review comments: ** Dropped grabbing read

[jira] [Commented] (HDFS-9493) Test o.a.h.hdfs.server.namenode.TestMetaSave fails in trunk

2015-12-18 Thread Tony Wu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9493?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15065039#comment-15065039 ] Tony Wu commented on HDFS-9493: --- Hi [~eddyxu], Thanks a lot for reviewing my patch. I have incorporated your

[jira] [Commented] (HDFS-9493) Test o.a.h.hdfs.server.namenode.TestMetaSave fails in trunk

2015-12-14 Thread Tony Wu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9493?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15057224#comment-15057224 ] Tony Wu commented on HDFS-9493: --- The failed tests are not related to the patch. As the patch only updated

[jira] [Commented] (HDFS-9493) Test o.a.h.hdfs.server.namenode.TestMetaSave fails in trunk

2015-12-14 Thread Tony Wu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9493?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15056768#comment-15056768 ] Tony Wu commented on HDFS-9493: --- Hi [~liuml07], Thanks for your detailed comments. I have confirmed problem

[jira] [Updated] (HDFS-9493) Test o.a.h.hdfs.server.namenode.TestMetaSave fails in trunk

2015-12-14 Thread Tony Wu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9493?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tony Wu updated HDFS-9493: -- Attachment: HDFS-9493.002.patch In v2 patch: * {{Use BlockManagerTestUtil#noticeDeadDatanode()}} to reduce the

[jira] [Assigned] (HDFS-9493) Test o.a.h.hdfs.server.namenode.TestMetaSave fails in trunk

2015-12-11 Thread Tony Wu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9493?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tony Wu reassigned HDFS-9493: - Assignee: Tony Wu > Test o.a.h.hdfs.server.namenode.TestMetaSave fails in trunk >

[jira] [Commented] (HDFS-9493) Test o.a.h.hdfs.server.namenode.TestMetaSave fails in trunk

2015-12-11 Thread Tony Wu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9493?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15053953#comment-15053953 ] Tony Wu commented on HDFS-9493: --- Hi [~liuml07], I would like to work on fixing this test. Did some analysis

[jira] [Updated] (HDFS-9493) Test o.a.h.hdfs.server.namenode.TestMetaSave fails in trunk

2015-12-11 Thread Tony Wu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9493?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tony Wu updated HDFS-9493: -- Attachment: HDFS-9493.001.patch In v1 patch: * Add a helper function

[jira] [Updated] (HDFS-9493) Test o.a.h.hdfs.server.namenode.TestMetaSave fails in trunk

2015-12-11 Thread Tony Wu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9493?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tony Wu updated HDFS-9493: -- Status: Patch Available (was: Open) > Test o.a.h.hdfs.server.namenode.TestMetaSave fails in trunk >

[jira] [Commented] (HDFS-9491) Tests should get the number of pending async delets via FsDatasetTestUtils

2015-12-04 Thread Tony Wu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9491?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15042448#comment-15042448 ] Tony Wu commented on HDFS-9491: --- Hi [~eddyxu], I just ran {{TestDecommission}} with the patch with JDK 8 on

[jira] [Commented] (HDFS-9490) MiniDFSCluster should change block generation stamp via FsDatasetTestUtils

2015-12-04 Thread Tony Wu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9490?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15041906#comment-15041906 ] Tony Wu commented on HDFS-9490: --- Thanks a lot [~eddyxu], for reviewing and providing comments. >

[jira] [Updated] (HDFS-9491) Tests should get the number of pending async delets via FsDatasetTestUtils

2015-12-04 Thread Tony Wu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9491?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tony Wu updated HDFS-9491: -- Attachment: HDFS-9491.002.patch In v2 patch: * Rebased to latest trunk. * Verified the build completes on OSX &

[jira] [Commented] (HDFS-9491) Tests should get the number of pending async delets via FsDatasetTestUtils

2015-12-04 Thread Tony Wu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9491?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15042017#comment-15042017 ] Tony Wu commented on HDFS-9491: --- Hi [~eddyxu], I Just rebased the patch on latest trunk and ran a few tests.

[jira] [Commented] (HDFS-9490) MiniDFSCluster should change block generation stamp via FsDatasetTestUtils

2015-12-02 Thread Tony Wu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9490?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15036109#comment-15036109 ] Tony Wu commented on HDFS-9490: --- Hi [~eddyxu], Thanks a lot for the quick review. I have updated the patch

[jira] [Updated] (HDFS-9490) MiniDFSCluster should change block generation stamp via FsDatasetTestUtils

2015-12-02 Thread Tony Wu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9490?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tony Wu updated HDFS-9490: -- Attachment: HDFS-9490.002.patch In v2 patch: * Addressed [~eddyxu]'s review comments by changing

[jira] [Commented] (HDFS-9490) MiniDFSCluster should change block generation stamp via FsDatasetTestUtils

2015-12-02 Thread Tony Wu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9490?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15036754#comment-15036754 ] Tony Wu commented on HDFS-9490: --- The failed tests are not related to this patch. Only

[jira] [Commented] (HDFS-9491) Tests should get the number of pending async delets via FsDatasetTestUtils

2015-12-01 Thread Tony Wu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9491?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15035142#comment-15035142 ] Tony Wu commented on HDFS-9491: --- Manually ran the failed tests: TestWriteReadStripedFile,

[jira] [Updated] (HDFS-9490) MiniDFSCluster should change block generation stamp via FsDatasetTestUtils

2015-12-01 Thread Tony Wu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9490?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tony Wu updated HDFS-9490: -- Attachment: HDFS-9490.001.patch In v1 patch: * Add {{FsDatasetTestUtils#changeStoredGenerationStamp}}. * Update

[jira] [Created] (HDFS-9490) MiniDFSCluster should change block generation stamp via FsDatasetTestUtils

2015-12-01 Thread Tony Wu (JIRA)
Tony Wu created HDFS-9490: - Summary: MiniDFSCluster should change block generation stamp via FsDatasetTestUtils Key: HDFS-9490 URL: https://issues.apache.org/jira/browse/HDFS-9490 Project: Hadoop HDFS

[jira] [Updated] (HDFS-9490) MiniDFSCluster should change block generation stamp via FsDatasetTestUtils

2015-12-01 Thread Tony Wu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9490?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tony Wu updated HDFS-9490: -- Status: Patch Available (was: Open) > MiniDFSCluster should change block generation stamp via

[jira] [Created] (HDFS-9491) Tests should get the number of pending async delets via FsDatasetTestUtils

2015-12-01 Thread Tony Wu (JIRA)
Tony Wu created HDFS-9491: - Summary: Tests should get the number of pending async delets via FsDatasetTestUtils Key: HDFS-9491 URL: https://issues.apache.org/jira/browse/HDFS-9491 Project: Hadoop HDFS

[jira] [Updated] (HDFS-9491) Tests should get the number of pending async delets via FsDatasetTestUtils

2015-12-01 Thread Tony Wu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9491?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tony Wu updated HDFS-9491: -- Target Version/s: 3.0.0, 2.8.0 > Tests should get the number of pending async delets via FsDatasetTestUtils >

[jira] [Updated] (HDFS-9491) Tests should get the number of pending async delets via FsDatasetTestUtils

2015-12-01 Thread Tony Wu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9491?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tony Wu updated HDFS-9491: -- Attachment: HDFS-9491.001.patch In v1 patch: * Add {{FsDatasetTestUtils#getPendingAsyncDeletions}}. * Update

[jira] [Updated] (HDFS-9491) Tests should get the number of pending async delets via FsDatasetTestUtils

2015-12-01 Thread Tony Wu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9491?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tony Wu updated HDFS-9491: -- Status: Patch Available (was: Open) > Tests should get the number of pending async delets via

[jira] [Commented] (HDFS-9282) Make data directory count and storage raw capacity related tests FsDataset-agnostic

2015-11-05 Thread Tony Wu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9282?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14991879#comment-14991879 ] Tony Wu commented on HDFS-9282: --- The failed test {[TestDeleteRace}} is not relevant to the change. Also

[jira] [Commented] (HDFS-9236) Missing sanity check for block size during block recovery

2015-11-05 Thread Tony Wu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9236?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14992724#comment-14992724 ] Tony Wu commented on HDFS-9236: --- Looked at the failed tests and none are related to block recovery. Also

[jira] [Commented] (HDFS-9282) Make data directory count and storage raw capacity related tests FsDataset-agnostic

2015-11-04 Thread Tony Wu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9282?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14990521#comment-14990521 ] Tony Wu commented on HDFS-9282: --- Thanks [~eddyxu] for catching this. I have incorporated the comments in the

[jira] [Updated] (HDFS-9282) Make data directory count and storage raw capacity related tests FsDataset-agnostic

2015-11-04 Thread Tony Wu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9282?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tony Wu updated HDFS-9282: -- Attachment: HDFS-9282.004.patch In v4 patch: * Addressed [~eddyxu]'s review comments (use {{try-with-resources}}

[jira] [Commented] (HDFS-9236) Missing sanity check for block size during block recovery

2015-11-04 Thread Tony Wu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9236?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14990215#comment-14990215 ] Tony Wu commented on HDFS-9236: --- Thanks a lot [~yzhangal] for your comments. I incorporated them into the new

[jira] [Updated] (HDFS-9236) Missing sanity check for block size during block recovery

2015-11-04 Thread Tony Wu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9236?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tony Wu updated HDFS-9236: -- Attachment: HDFS-9236.007.patch In v7 patch: * Addressed [~yzhangal]'s review comments. * Update the test case.

[jira] [Updated] (HDFS-9236) Missing sanity check for block size during block recovery

2015-11-03 Thread Tony Wu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9236?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tony Wu updated HDFS-9236: -- Attachment: HDFS-9236.006.patch In v6 patch: * Address [~walter.k.su]'s comment by excluding RUR replicas from

[jira] [Created] (HDFS-9363) Add fetchReplica() to FsDatasetTestUtils()

2015-11-02 Thread Tony Wu (JIRA)
Tony Wu created HDFS-9363: - Summary: Add fetchReplica() to FsDatasetTestUtils() Key: HDFS-9363 URL: https://issues.apache.org/jira/browse/HDFS-9363 Project: Hadoop HDFS Issue Type: Improvement

[jira] [Updated] (HDFS-9363) Add fetchReplica() to FsDatasetTestUtils()

2015-11-02 Thread Tony Wu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9363?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tony Wu updated HDFS-9363: -- Attachment: HDFS-9363.001.patch In v1 patch: * Add {{fetchReplica()}} to {{FsDatasetTestUtils()}}. * Modify

[jira] [Updated] (HDFS-9363) Add fetchReplica() to FsDatasetTestUtils()

2015-11-02 Thread Tony Wu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9363?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tony Wu updated HDFS-9363: -- Status: Patch Available (was: Open) > Add fetchReplica() to FsDatasetTestUtils() >

[jira] [Commented] (HDFS-9236) Missing sanity check for block size during block recovery

2015-10-31 Thread Tony Wu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9236?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14984040#comment-14984040 ] Tony Wu commented on HDFS-9236: --- Thanks [~walter.k.su] and [~yzhangal] for your comments. I'll post a new

[jira] [Updated] (HDFS-9308) Add truncateMeta() and deleteMeta() to MiniDFSCluster

2015-10-30 Thread Tony Wu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9308?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tony Wu updated HDFS-9308: -- Attachment: HDFS-9308.003.patch In v3 patch: * Addressed [~eddyxu]'s review comments. *Please note:*

[jira] [Commented] (HDFS-9236) Missing sanity check for block size during block recovery

2015-10-30 Thread Tony Wu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9236?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14982958#comment-14982958 ] Tony Wu commented on HDFS-9236: --- Thanks a lot for [~walter.k.su] and [~zhz]'s comments! [~walter.k.su], DN

[jira] [Commented] (HDFS-9236) Missing sanity check for block size during block recovery

2015-10-29 Thread Tony Wu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9236?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14980912#comment-14980912 ] Tony Wu commented on HDFS-9236: --- [~yzhangal] Thanks a lot for looking at the patch. > Missing sanity check

[jira] [Commented] (HDFS-9236) Missing sanity check for block size during block recovery

2015-10-29 Thread Tony Wu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9236?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14981071#comment-14981071 ] Tony Wu commented on HDFS-9236: --- Hi [~yzhangal], I believe HDFS-9255 has moved block recovery related code to

[jira] [Updated] (HDFS-9236) Missing sanity check for block size during block recovery

2015-10-29 Thread Tony Wu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9236?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tony Wu updated HDFS-9236: -- Attachment: HDFS-9236.004.patch In v4 patch: * Rebased to latest trunk (moved the changes to new file

[jira] [Commented] (HDFS-9236) Missing sanity check for block size during block recovery

2015-10-29 Thread Tony Wu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9236?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14981419#comment-14981419 ] Tony Wu commented on HDFS-9236: --- Hi [~liuml07], Thanks a lot for your comment. I debated about having an

[jira] [Updated] (HDFS-9282) Make data directory count and storage raw capacity related tests FsDataset-agnostic

2015-10-29 Thread Tony Wu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9282?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tony Wu updated HDFS-9282: -- Attachment: HDFS-9282.003.patch In v3 patch: * Rebased patch on latest trunk. > Make data directory count and

[jira] [Commented] (HDFS-9331) Modify TestNameNodeMXBean#testNameNodeMXBeanInfo() to account for filesystem entirely allocated for DFS use

2015-10-29 Thread Tony Wu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9331?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14981671#comment-14981671 ] Tony Wu commented on HDFS-9331: --- The test failures are unrelated as the patch only changed

[jira] [Commented] (HDFS-9236) Missing sanity check for block size during block recovery

2015-10-29 Thread Tony Wu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9236?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14981585#comment-14981585 ] Tony Wu commented on HDFS-9236: --- Thanks for clarifying. I'll post a updated patch shortly. > Missing sanity

[jira] [Updated] (HDFS-9236) Missing sanity check for block size during block recovery

2015-10-29 Thread Tony Wu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9236?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tony Wu updated HDFS-9236: -- Attachment: HDFS-9236.005.patch In v5 patch: * Addressed [~liuml07]'s comment by updating the test case. >

[jira] [Updated] (HDFS-9331) Modify TestNameNodeMXBean#testNameNodeMXBeanInfo() to account for filesystem entirely allocated for DFS use

2015-10-28 Thread Tony Wu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9331?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tony Wu updated HDFS-9331: -- Status: Patch Available (was: Open) > Modify TestNameNodeMXBean#testNameNodeMXBeanInfo() to account for

[jira] [Updated] (HDFS-9331) Modify TestNameNodeMXBean#testNameNodeMXBeanInfo() to account for filesystem entirely allocated for DFS use

2015-10-28 Thread Tony Wu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9331?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tony Wu updated HDFS-9331: -- Attachment: HDFS-9331.001.patch In v1 patch: * Relax the check in {{TestNameNodeMXBean#testNameNodeMXBeanInfo}}

[jira] [Created] (HDFS-9331) Modify TestNameNodeMXBean#testNameNodeMXBeanInfo() to account for filesystem entirely allocated for DFS use

2015-10-28 Thread Tony Wu (JIRA)
Tony Wu created HDFS-9331: - Summary: Modify TestNameNodeMXBean#testNameNodeMXBeanInfo() to account for filesystem entirely allocated for DFS use Key: HDFS-9331 URL: https://issues.apache.org/jira/browse/HDFS-9331

[jira] [Updated] (HDFS-9331) Modify TestNameNodeMXBean#testNameNodeMXBeanInfo() to account for filesystem entirely allocated for DFS use

2015-10-28 Thread Tony Wu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9331?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tony Wu updated HDFS-9331: -- Description: {{TestNameNodeMXBean#testNameNodeMXBeanInfo}} expects a none-zero nonDFS size. The nonDFS size is

[jira] [Updated] (HDFS-9308) Add truncateMeta() and deleteMeta() to MiniDFSCluster

2015-10-27 Thread Tony Wu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9308?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tony Wu updated HDFS-9308: -- Attachment: HDFS-9308.002.patch * Add {{truncateMeta()}} and {{deleteMeta()}} method to truncate/delete meta

[jira] [Updated] (HDFS-9308) Add truncateMeta() and deleteMeta() to MiniDFSCluster

2015-10-27 Thread Tony Wu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9308?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tony Wu updated HDFS-9308: -- Summary: Add truncateMeta() and deleteMeta() to MiniDFSCluster (was: Add truncateMeta() to MiniDFSCluster) >

[jira] [Updated] (HDFS-9308) Add truncateMeta() and deleteMeta() to MiniDFSCluster

2015-10-27 Thread Tony Wu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9308?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tony Wu updated HDFS-9308: -- Description: HDFS-9188 introduced {{corruptMeta()}} method to make corrupting the metadata file filesystem

[jira] [Updated] (HDFS-9308) Add truncateMeta() to MiniDFSCluster

2015-10-26 Thread Tony Wu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9308?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tony Wu updated HDFS-9308: -- Attachment: HDFS-9308.001.patch In this patch: * Add {{truncateMeta()}} method to truncate meta data files on

[jira] [Updated] (HDFS-9308) Add truncateMeta() to MiniDFSCluster

2015-10-26 Thread Tony Wu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9308?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tony Wu updated HDFS-9308: -- Status: Patch Available (was: Open) > Add truncateMeta() to MiniDFSCluster >

[jira] [Created] (HDFS-9308) Add truncateMeta() to MiniDFSCluster

2015-10-26 Thread Tony Wu (JIRA)
Tony Wu created HDFS-9308: - Summary: Add truncateMeta() to MiniDFSCluster Key: HDFS-9308 URL: https://issues.apache.org/jira/browse/HDFS-9308 Project: Hadoop HDFS Issue Type: Improvement

[jira] [Commented] (HDFS-9282) Make data directory count and storage raw capacity related tests FsDataset-agnostic

2015-10-26 Thread Tony Wu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9282?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14975193#comment-14975193 ] Tony Wu commented on HDFS-9282: --- Manually verified hadoop.hdfs.server.datanode.TestDirectoryScanner runs

[jira] [Commented] (HDFS-9290) DFSClient#callAppend() is not backward compatible for slightly older NameNodes

2015-10-26 Thread Tony Wu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9290?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14975231#comment-14975231 ] Tony Wu commented on HDFS-9290: --- HI [~kihwal], Thanks for taking the time to manually run these tests. I

[jira] [Updated] (HDFS-9308) Add truncateMeta() to MiniDFSCluster

2015-10-26 Thread Tony Wu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9308?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tony Wu updated HDFS-9308: -- Description: HDFS-9188 introduced {{corruptMeta()}} method to make corrupting the metadata file filesystem

[jira] [Updated] (HDFS-9282) Make data directory count and storage raw capacity related tests FsDataset-agnostic

2015-10-26 Thread Tony Wu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9282?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tony Wu updated HDFS-9282: -- Attachment: HDFS-9282.002.patch Addressed [~eddyxu]'s review comments. > Make data directory count and storage

[jira] [Commented] (HDFS-9282) Make data directory count and storage raw capacity related tests FsDataset-agnostic

2015-10-26 Thread Tony Wu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9282?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14974469#comment-14974469 ] Tony Wu commented on HDFS-9282: --- Hi [~eddyxu], Thank you very much for the detailed review. I have addressed

[jira] [Commented] (HDFS-9297) Update TestBlockMissingException to use corruptBlockOnDataNodesByDeletingBlockFile()

2015-10-23 Thread Tony Wu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9297?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14971546#comment-14971546 ] Tony Wu commented on HDFS-9297: --- The failed tests are not related to this change. > Update

[jira] [Updated] (HDFS-9297) Update TestBlockMissingException to use corruptBlockOnDataNodesByDeletingBlockFile()

2015-10-23 Thread Tony Wu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9297?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tony Wu updated HDFS-9297: -- Status: Patch Available (was: Open) > Update TestBlockMissingException to use >

[jira] [Updated] (HDFS-9297) Update TestBlockMissingException to use corruptBlockOnDataNodesByDeletingBlockFile()

2015-10-23 Thread Tony Wu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9297?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tony Wu updated HDFS-9297: -- Attachment: HDFS-9297.001.patch In this patch: * Use {{corruptBlockOnDataNodesByDeletingBlockFile()}} to corrupt

[jira] [Created] (HDFS-9297) Update TestBlockMissingException to use corruptBlockOnDataNodesByDeletingBlockFile()

2015-10-23 Thread Tony Wu (JIRA)
Tony Wu created HDFS-9297: - Summary: Update TestBlockMissingException to use corruptBlockOnDataNodesByDeletingBlockFile() Key: HDFS-9297 URL: https://issues.apache.org/jira/browse/HDFS-9297 Project: Hadoop

[jira] [Updated] (HDFS-9290) DFSClient#callAppend() is not backward compatible for slightly older NameNodes

2015-10-23 Thread Tony Wu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9290?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tony Wu updated HDFS-9290: -- Attachment: HDFS-9290.002.patch Hi [~kihwal], Thanks a lot for taking the time to look at the patch! I have

[jira] [Updated] (HDFS-9290) DFSClient#callAppend() is not backward compatible for slightly older NameNodes

2015-10-22 Thread Tony Wu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9290?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tony Wu updated HDFS-9290: -- Status: Patch Available (was: Open) > DFSClient#callAppend() is not backward compatible for slightly older

[jira] [Created] (HDFS-9290) DFSClient#callAppend() is not backward compatible for slightly older NameNodes

2015-10-22 Thread Tony Wu (JIRA)
Tony Wu created HDFS-9290: - Summary: DFSClient#callAppend() is not backward compatible for slightly older NameNodes Key: HDFS-9290 URL: https://issues.apache.org/jira/browse/HDFS-9290 Project: Hadoop HDFS

[jira] [Updated] (HDFS-9290) DFSClient#callAppend() is not backward compatible for slightly older NameNodes

2015-10-22 Thread Tony Wu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9290?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tony Wu updated HDFS-9290: -- Attachment: HDFS-9290.001.patch In this patch: * Detect that DFSClient is talking to an older NameNode and make

[jira] [Updated] (HDFS-9290) DFSClient#callAppend() is not backward compatible for slightly older NameNodes

2015-10-22 Thread Tony Wu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9290?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tony Wu updated HDFS-9290: -- Description: HDFS-7210 combined 2 RPC calls used at file append into a single one. Specifically

[jira] [Commented] (HDFS-9282) Make data directory count and storage raw capacity related tests FsDataset-agnostic

2015-10-22 Thread Tony Wu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9282?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14969354#comment-14969354 ] Tony Wu commented on HDFS-9282: --- Manually reran the failed tests and they both pass without error. Inspected

[jira] [Commented] (HDFS-9236) Missing sanity check for block size during block recovery

2015-10-21 Thread Tony Wu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9236?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14967513#comment-14967513 ] Tony Wu commented on HDFS-9236: --- Hi [~yzhangal], Could you take another look at the updated patch? Thanks,

[jira] [Updated] (HDFS-9282) Make data directory count and storage raw capacity related tests FsDataset-agnostic

2015-10-21 Thread Tony Wu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9282?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tony Wu updated HDFS-9282: -- Status: Patch Available (was: Open) > Make data directory count and storage raw capacity related tests >

[jira] [Updated] (HDFS-9282) Make data directory count and storage raw capacity related tests FsDataset-agnostic

2015-10-21 Thread Tony Wu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9282?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tony Wu updated HDFS-9282: -- Attachment: HDFS-9282.001.patch In this patch: * Add getNumOfDataDirs() and getRawCapacity() interfaces. * Have

[jira] [Created] (HDFS-9282) Make data directory count and storage raw capacity related tests FsDataset-agnostic

2015-10-21 Thread Tony Wu (JIRA)
Tony Wu created HDFS-9282: - Summary: Make data directory count and storage raw capacity related tests FsDataset-agnostic Key: HDFS-9282 URL: https://issues.apache.org/jira/browse/HDFS-9282 Project: Hadoop

[jira] [Commented] (HDFS-9236) Missing sanity check for block size during block recovery

2015-10-16 Thread Tony Wu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9236?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14961543#comment-14961543 ] Tony Wu commented on HDFS-9236: --- checksyle and pre-patch error are not related to this patch. > Missing

[jira] [Updated] (HDFS-9236) Missing sanity check for block size during block recovery

2015-10-15 Thread Tony Wu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9236?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tony Wu updated HDFS-9236: -- Attachment: HDFS-9236.002.patch Removed block size check on NN. > Missing sanity check for block size during

[jira] [Commented] (HDFS-9236) Missing sanity check for block size during block recovery

2015-10-15 Thread Tony Wu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9236?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14959008#comment-14959008 ] Tony Wu commented on HDFS-9236: --- Thanks to [~yzhangal] for offline review and valuable comments! In summary:

[jira] [Commented] (HDFS-9236) Missing sanity check for block size during block recovery

2015-10-15 Thread Tony Wu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9236?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14959253#comment-14959253 ] Tony Wu commented on HDFS-9236: --- Hi [~yzhangal], Thanks a lot for looking at the patch. Regarding your

[jira] [Updated] (HDFS-9236) Missing sanity check for block size during block recovery

2015-10-15 Thread Tony Wu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9236?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tony Wu updated HDFS-9236: -- Attachment: HDFS-9236.003.patch Addressed [~yzhangal]'s review comments. > Missing sanity check for block size

[jira] [Commented] (HDFS-9238) Update TestFileCreation#testLeaseExpireHardLimit() to avoid using DataNodeTestUtils#getFile()

2015-10-14 Thread Tony Wu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9238?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14957068#comment-14957068 ] Tony Wu commented on HDFS-9238: --- The failed tests are not related to this patch as the patch itself only

[jira] [Created] (HDFS-9236) Add sanity check for block size during block recovery

2015-10-13 Thread Tony Wu (JIRA)
Tony Wu created HDFS-9236: - Summary: Add sanity check for block size during block recovery Key: HDFS-9236 URL: https://issues.apache.org/jira/browse/HDFS-9236 Project: Hadoop HDFS Issue Type: Bug

[jira] [Updated] (HDFS-9236) Missing sanity check for block size during block recovery

2015-10-13 Thread Tony Wu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9236?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tony Wu updated HDFS-9236: -- Summary: Missing sanity check for block size during block recovery (was: Add sanity check for block size during

[jira] [Updated] (HDFS-9236) Missing sanity check for block size during block recovery

2015-10-13 Thread Tony Wu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9236?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tony Wu updated HDFS-9236: -- Attachment: HDFS-9236.001.patch First patch. > Missing sanity check for block size during block recovery >

[jira] [Commented] (HDFS-9236) Missing sanity check for block size during block recovery

2015-10-13 Thread Tony Wu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9236?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14955280#comment-14955280 ] Tony Wu commented on HDFS-9236: --- The path does: * Add replica length check in syncBlock() so DN reports error

[jira] [Commented] (HDFS-9236) Missing sanity check for block size during block recovery

2015-10-13 Thread Tony Wu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9236?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14955818#comment-14955818 ] Tony Wu commented on HDFS-9236: --- All tests pass when manually run on OSX and Linux (CentOS 6.4) with latest

[jira] [Created] (HDFS-9238) Update TestFileCreation#testLeaseExpireHardLimit() to avoid using DataNodeTestUtils#getFile()

2015-10-13 Thread Tony Wu (JIRA)
Tony Wu created HDFS-9238: - Summary: Update TestFileCreation#testLeaseExpireHardLimit() to avoid using DataNodeTestUtils#getFile() Key: HDFS-9238 URL: https://issues.apache.org/jira/browse/HDFS-9238 Project:

[jira] [Updated] (HDFS-9238) Update TestFileCreation#testLeaseExpireHardLimit() to avoid using DataNodeTestUtils#getFile()

2015-10-13 Thread Tony Wu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9238?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tony Wu updated HDFS-9238: -- Attachment: HDFS-9238.001.patch Use getBlockInputStream() to read block on DN instead of using getFile() and

[jira] [Updated] (HDFS-9238) Update TestFileCreation#testLeaseExpireHardLimit() to avoid using DataNodeTestUtils#getFile()

2015-10-13 Thread Tony Wu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9238?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tony Wu updated HDFS-9238: -- Status: Patch Available (was: In Progress) > Update TestFileCreation#testLeaseExpireHardLimit() to avoid using

[jira] [Work started] (HDFS-9238) Update TestFileCreation#testLeaseExpireHardLimit() to avoid using DataNodeTestUtils#getFile()

2015-10-13 Thread Tony Wu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9238?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Work on HDFS-9238 started by Tony Wu. - > Update TestFileCreation#testLeaseExpireHardLimit() to avoid using >

[jira] [Updated] (HDFS-9238) Update TestFileCreation#testLeaseExpireHardLimit() to avoid using DataNodeTestUtils#getFile()

2015-10-13 Thread Tony Wu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9238?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tony Wu updated HDFS-9238: -- Component/s: test HDFS > Update TestFileCreation#testLeaseExpireHardLimit() to avoid using >

[jira] [Updated] (HDFS-9236) Missing sanity check for block size during block recovery

2015-10-13 Thread Tony Wu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9236?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tony Wu updated HDFS-9236: -- Component/s: HDFS > Missing sanity check for block size during block recovery >

[jira] [Commented] (HDFS-9148) Incorrect assert message in TestWriteToReplica#testWriteToTemporary

2015-09-28 Thread Tony Wu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9148?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14933910#comment-14933910 ] Tony Wu commented on HDFS-9148: --- I don't think the failed tests (TestDirectoryScanner, TestWebHDFS,

[jira] [Updated] (HDFS-9148) Incorrect assert message in TestWriteToReplica#testWriteToTemporary

2015-09-28 Thread Tony Wu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9148?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tony Wu updated HDFS-9148: -- Status: Patch Available (was: Open) > Incorrect assert message in TestWriteToReplica#testWriteToTemporary >

[jira] [Commented] (HDFS-9148) Incorrect assert message in TestWriteToReplica#testWriteToTemporary

2015-09-28 Thread Tony Wu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9148?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14933386#comment-14933386 ] Tony Wu commented on HDFS-9148: --- Than you Daniel, for looking at it quickly! > Incorrect assert message in

[jira] [Updated] (HDFS-9148) Incorrect assert message in TestWriteToReplica#testWriteToTemporary

2015-09-25 Thread Tony Wu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9148?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tony Wu updated HDFS-9148: -- Attachment: hdfs-9148.patch A pretty trivial change to assert text. > Incorrect assert message in

[jira] [Updated] (HDFS-9148) Incorrect assert message in TestWriteToReplica#testWriteToTemporary

2015-09-25 Thread Tony Wu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9148?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tony Wu updated HDFS-9148: -- Attachment: (was: hdfs-9148.patch) > Incorrect assert message in TestWriteToReplica#testWriteToTemporary >

  1   2   >