[jira] [Updated] (HDFS-7859) Erasure Coding: Persist erasure coding policies in NameNode

2016-08-25 Thread Xinwei Qin (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-7859?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xinwei Qin updated HDFS-7859: -- Attachment: (was: HDFS-7859.008.patch) > Erasure Coding: Persist erasure coding policies in NameNode

[jira] [Commented] (HDFS-7859) Erasure Coding: Persist erasure coding policies in NameNode

2016-08-24 Thread Xinwei Qin (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-7859?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15435188#comment-15435188 ] Xinwei Qin commented on HDFS-7859: --- Sorry for attaching the wrong patch, not the latest one, I will

[jira] [Commented] (HDFS-7859) Erasure Coding: Persist erasure coding policies in NameNode

2016-08-24 Thread Xinwei Qin (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-7859?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15435092#comment-15435092 ] Xinwei Qin commented on HDFS-7859: --- Attach the new patch to fix the only TestOfflineEditsViewer failure.

[jira] [Updated] (HDFS-7859) Erasure Coding: Persist erasure coding policies in NameNode

2016-08-24 Thread Xinwei Qin (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-7859?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xinwei Qin updated HDFS-7859: -- Attachment: HDFS-7859.008.patch > Erasure Coding: Persist erasure coding policies in NameNode >

[jira] [Updated] (HDFS-10786) Erasure Coding: Add removeErasureCodingPolicy API

2016-08-23 Thread Xinwei Qin (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10786?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xinwei Qin updated HDFS-10786: --- Description: HDFS-7859 has developed addErasureCodingPolicy API to add some user-added Erasure Coding

[jira] [Commented] (HDFS-7859) Erasure Coding: Persist erasure coding policies in NameNode

2016-08-23 Thread Xinwei Qin (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-7859?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15432993#comment-15432993 ] Xinwei Qin commented on HDFS-7859: --- Hi, [~zhz], Have updated the patch with your comments and fixed

[jira] [Created] (HDFS-10786) Erasure Coding: Add removeErasureCodingPolicy API

2016-08-23 Thread Xinwei Qin (JIRA)
Xinwei Qin created HDFS-10786: -- Summary: Erasure Coding: Add removeErasureCodingPolicy API Key: HDFS-10786 URL: https://issues.apache.org/jira/browse/HDFS-10786 Project: Hadoop HDFS Issue

[jira] [Updated] (HDFS-7859) Erasure Coding: Persist erasure coding policies in NameNode

2016-08-23 Thread Xinwei Qin (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-7859?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xinwei Qin updated HDFS-7859: -- Attachment: HDFS-7859.007.patch > Erasure Coding: Persist erasure coding policies in NameNode >

[jira] [Commented] (HDFS-7859) Erasure Coding: Persist erasure coding policies in NameNode

2016-08-17 Thread Xinwei Qin (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-7859?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15424776#comment-15424776 ] Xinwei Qin commented on HDFS-7859: --- Hi, [~zhz] I have updated the patch. About your comments, I have

[jira] [Issue Comment Deleted] (HDFS-7859) Erasure Coding: Persist erasure coding policies in NameNode

2016-08-17 Thread Xinwei Qin (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-7859?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xinwei Qin updated HDFS-7859: -- Comment: was deleted (was: Hi, [~zhz] I have updated the patch. About your comments, I have fixed 1, 2,

[jira] [Commented] (HDFS-7859) Erasure Coding: Persist erasure coding policies in NameNode

2016-08-17 Thread Xinwei Qin (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-7859?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15424774#comment-15424774 ] Xinwei Qin commented on HDFS-7859: --- Hi, [~zhz] I have updated the patch. About your comments, I have

[jira] [Updated] (HDFS-7859) Erasure Coding: Persist erasure coding policies in NameNode

2016-08-17 Thread Xinwei Qin (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-7859?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xinwei Qin updated HDFS-7859: -- Attachment: HDFS-7859.006.patch > Erasure Coding: Persist erasure coding policies in NameNode >

[jira] [Commented] (HDFS-7859) Erasure Coding: Persist erasure coding policies in NameNode

2016-08-15 Thread Xinwei Qin (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-7859?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15421090#comment-15421090 ] Xinwei Qin commented on HDFS-7859: --- Thanks for your comments, [~zhz], I will update the patch shortly.

[jira] [Updated] (HDFS-7859) Erasure Coding: Persist erasure coding policies in NameNode

2016-07-04 Thread Xinwei Qin (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-7859?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xinwei Qin updated HDFS-7859: -- Attachment: HDFS-7859.005.patch Rebased the patch(005.patch), and add a unit test case. > Erasure

[jira] [Commented] (HDFS-7859) Erasure Coding: Persist erasure coding policies in NameNode

2016-06-26 Thread Xinwei Qin (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-7859?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15350146#comment-15350146 ] Xinwei Qin commented on HDFS-7859: --- Rebased patch still have some test failure, I'm doing my best to fix

[jira] [Commented] (HDFS-7859) Erasure Coding: Persist erasure coding policies in NameNode

2016-06-21 Thread Xinwei Qin (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-7859?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15341759#comment-15341759 ] Xinwei Qin commented on HDFS-7859: --- [~zhz], it's better to have this in 3.0, rebasing and perfect this

[jira] [Commented] (HDFS-7859) Erasure Coding: Persist erasure coding policies in NameNode

2016-04-22 Thread Xinwei Qin (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-7859?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15253497#comment-15253497 ] Xinwei Qin commented on HDFS-7859: --- Need to update patch to fix the checkstyles and relevant unit test

[jira] [Updated] (HDFS-7859) Erasure Coding: Persist erasure coding policies in NameNode

2016-04-19 Thread Xinwei Qin (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-7859?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xinwei Qin updated HDFS-7859: -- Attachment: HDFS-7859.004.patch Rebase the patch with the latest trunk. > Erasure Coding: Persist

[jira] [Commented] (HDFS-7859) Erasure Coding: Persist erasure coding policies in NameNode

2016-04-18 Thread Xinwei Qin (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-7859?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15245677#comment-15245677 ] Xinwei Qin commented on HDFS-7859: --- [~rakeshr] [~drankye], and [~zhz], thanks for your comments and

[jira] [Commented] (HDFS-7859) Erasure Coding: Persist erasure coding policies in NameNode

2016-01-17 Thread Xinwei Qin (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-7859?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15104111#comment-15104111 ] Xinwei Qin commented on HDFS-7859: --- Hi, [~drankye], I will proceed to fill this jira recently. >

[jira] [Updated] (HDFS-8202) Improve end to end stirpping file test to add erasure recovering test

2015-07-30 Thread Xinwei Qin (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8202?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xinwei Qin updated HDFS-8202: -- Attachment: HDFS-8202-HDFS-7285.006.patch Thanks [~zhz] for the comments. Upload 006 patch for review.

[jira] [Updated] (HDFS-8202) Improve end to end stirpping file test to add erasure recovering test

2015-07-23 Thread Xinwei Qin (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8202?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xinwei Qin updated HDFS-8202: -- Attachment: HDFS-8202-HDFS-7285.005.patch Thanks [~zhz] and [~walter.k.su] for the comments and advice.

[jira] [Updated] (HDFS-8202) Improve end to end stirpping file test to add erasure recovering test

2015-07-22 Thread Xinwei Qin (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8202?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xinwei Qin updated HDFS-8202: -- Attachment: HDFS-8202-HDFS-7285.004.patch Thanks [~zhz] for the comments. Update 004 patch to address

[jira] [Commented] (HDFS-8704) Erasure Coding: client fails to write large file when one datanode fails

2015-07-22 Thread Xinwei Qin (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8704?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14636721#comment-14636721 ] Xinwei Qin commented on HDFS-8704: --- Hi, [~libo-intel], even if the file is smaller than

[jira] [Updated] (HDFS-8202) Improve end to end stirpping file test to add erasure recovering test

2015-07-15 Thread Xinwei Qin (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8202?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xinwei Qin updated HDFS-8202: -- Attachment: HDFS-8202-HDFS-7285.003.patch [~zhz], updated the patch including reading and writing EC

[jira] [Commented] (HDFS-8202) Improve end to end stirpping file test to add erasure recovering test

2015-07-15 Thread Xinwei Qin (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8202?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14629094#comment-14629094 ] Xinwei Qin commented on HDFS-8202: --- Hi, [~zhz], thanks for your clarify. I will move

[jira] [Updated] (HDFS-8260) Erasure Coding: system test of writing EC file

2015-07-14 Thread Xinwei Qin (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8260?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xinwei Qin updated HDFS-8260: -- Attachment: HDFS-8260-HDFS-7285.001.patch Attach the system test patch of writing EC file with some

[jira] [Commented] (HDFS-8260) Erasure Coding: system test of writing EC file

2015-07-14 Thread Xinwei Qin (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8260?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14627488#comment-14627488 ] Xinwei Qin commented on HDFS-8260: --- Hi, [~demongaorui], Maybe I incorrectly understand

[jira] [Commented] (HDFS-8732) Erasure Coding: Fail to read a file with corrupted blocks

2015-07-09 Thread Xinwei Qin (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8732?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14621593#comment-14621593 ] Xinwei Qin commented on HDFS-8732: --- Yes, [~jingzhao], the test passed with patch in

[jira] [Resolved] (HDFS-8732) Erasure Coding: Fail to read a file with corrupted blocks

2015-07-09 Thread Xinwei Qin (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8732?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xinwei Qin resolved HDFS-8732. --- Resolution: Fixed Erasure Coding: Fail to read a file with corrupted blocks

[jira] [Reopened] (HDFS-8732) Erasure Coding: Fail to read a file with corrupted blocks

2015-07-09 Thread Xinwei Qin (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8732?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xinwei Qin reopened HDFS-8732: --- Erasure Coding: Fail to read a file with corrupted blocks

[jira] [Resolved] (HDFS-8732) Erasure Coding: Fail to read a file with corrupted blocks

2015-07-09 Thread Xinwei Qin (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8732?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xinwei Qin resolved HDFS-8732. --- Resolution: Duplicate Erasure Coding: Fail to read a file with corrupted blocks

[jira] [Created] (HDFS-8732) Erasure Coding: Fail to read a file with corrupted blocks

2015-07-08 Thread Xinwei Qin (JIRA)
Xinwei Qin created HDFS-8732: - Summary: Erasure Coding: Fail to read a file with corrupted blocks Key: HDFS-8732 URL: https://issues.apache.org/jira/browse/HDFS-8732 Project: Hadoop HDFS Issue

[jira] [Commented] (HDFS-8732) Erasure Coding: Fail to read a file with corrupted blocks

2015-07-08 Thread Xinwei Qin (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8732?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14618199#comment-14618199 ] Xinwei Qin commented on HDFS-8732: --- Hi, [~hitliuyi], I noticed HDFS-8602 had resolved

[jira] [Updated] (HDFS-8259) Erasure Coding: System Test of reading EC file

2015-07-08 Thread Xinwei Qin (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8259?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xinwei Qin updated HDFS-8259: -- Status: Patch Available (was: Open) Erasure Coding: System Test of reading EC file

[jira] [Commented] (HDFS-8732) Erasure Coding: Fail to read a file with corrupted blocks

2015-07-08 Thread Xinwei Qin (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8732?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14618246#comment-14618246 ] Xinwei Qin commented on HDFS-8732: --- a simple patch Erasure Coding: Fail to read a file

[jira] [Updated] (HDFS-8732) Erasure Coding: Fail to read a file with corrupted blocks

2015-07-08 Thread Xinwei Qin (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8732?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xinwei Qin updated HDFS-8732: -- Attachment: testReadCorruptedData.patch Attach a simple path including a test method to reproduce the

[jira] [Commented] (HDFS-8259) Erasure Coding: System Test of reading EC file

2015-07-08 Thread Xinwei Qin (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8259?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14618181#comment-14618181 ] Xinwei Qin commented on HDFS-8259: --- In this patch, the test methods

[jira] [Commented] (HDFS-8259) Erasure Coding: System Test of reading EC file

2015-07-08 Thread Xinwei Qin (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8259?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14618082#comment-14618082 ] Xinwei Qin commented on HDFS-8259: --- Attached the patch to review. About reading a file

[jira] [Commented] (HDFS-8202) Improve end to end stirpping file test to add erasure recovering test

2015-07-08 Thread Xinwei Qin (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8202?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14618097#comment-14618097 ] Xinwei Qin commented on HDFS-8202: --- Maybe this jira is duplicated to HDFS-8259, I have

[jira] [Updated] (HDFS-8259) Erasure Coding: System Test of reading EC file

2015-07-08 Thread Xinwei Qin (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8259?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xinwei Qin updated HDFS-8259: -- Attachment: HDFS-8259-HDFS-7285.001.patch Erasure Coding: System Test of reading EC file

[jira] [Commented] (HDFS-8710) Always read DU value from the cached dfsUsed file on datanode startup

2015-07-03 Thread Xinwei Qin (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8710?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14613014#comment-14613014 ] Xinwei Qin commented on HDFS-8710: --- [~aw], thanks for your comment. The du value will be

[jira] [Commented] (HDFS-8260) Erasure Coding: test of writing EC file

2015-07-03 Thread Xinwei Qin (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8260?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14612934#comment-14612934 ] Xinwei Qin commented on HDFS-8260: --- Hi [~demongaorui], sorry for busy with other work in

[jira] [Updated] (HDFS-8710) Always read DU value from the cached dfsUsed file on datanode startup

2015-07-02 Thread Xinwei Qin (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8710?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xinwei Qin updated HDFS-8710: -- Attachment: HDFS-8710.001.patch Attached the patch to review. Always read DU value from the cached

[jira] [Created] (HDFS-8710) Always read DU value from the cached dfsUsed file on datanode startup

2015-07-02 Thread Xinwei Qin (JIRA)
Xinwei Qin created HDFS-8710: - Summary: Always read DU value from the cached dfsUsed file on datanode startup Key: HDFS-8710 URL: https://issues.apache.org/jira/browse/HDFS-8710 Project: Hadoop HDFS

[jira] [Updated] (HDFS-8710) Always read DU value from the cached dfsUsed file on datanode startup

2015-07-02 Thread Xinwei Qin (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8710?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xinwei Qin updated HDFS-8710: -- Status: Patch Available (was: Open) Always read DU value from the cached dfsUsed file on datanode

[jira] [Updated] (HDFS-8201) Refactor the end to end test for stripping file writing and reading

2015-05-21 Thread Xinwei Qin (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8201?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xinwei Qin updated HDFS-8201: -- Summary: Refactor the end to end test for stripping file writing and reading (was: Add an end to end

[jira] [Commented] (HDFS-8201) Refactor the end to end test for stripping file writing and reading

2015-05-21 Thread Xinwei Qin (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8201?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14554239#comment-14554239 ] Xinwei Qin commented on HDFS-8201: --- Thanks [~zhz] for the review. 1. The JIRA

[jira] [Updated] (HDFS-8201) Refactor the end to end test for stripping file writing and reading

2015-05-21 Thread Xinwei Qin (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8201?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xinwei Qin updated HDFS-8201: -- Attachment: HDFS-8201-HDFS-7285.005.patch Refactor the end to end test for stripping file writing and

[jira] [Updated] (HDFS-8201) Refactor the end to end test for stripping file writing and reading

2015-05-21 Thread Xinwei Qin (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8201?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xinwei Qin updated HDFS-8201: -- Description: According to off-line discussion with [~zhz] and [~xinwei], we need to implement an end to

[jira] [Updated] (HDFS-8201) Add an end to end test for stripping file writing and reading

2015-05-12 Thread Xinwei Qin (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8201?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xinwei Qin updated HDFS-8201: -- Attachment: HDFS-8201-HDFS-7285.004.patch Update patch with the latest branch-7285. Add an end to end

[jira] [Updated] (HDFS-8202) Improve end to end stirpping file test to add erasure recovering test

2015-05-12 Thread Xinwei Qin (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8202?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xinwei Qin updated HDFS-8202: -- Attachment: HDFS-8202.002.patch Update patch with the latest branch-7285. Improve end to end stirpping

[jira] [Updated] (HDFS-8201) Add an end to end test for stripping file writing and reading

2015-05-12 Thread Xinwei Qin (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8201?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xinwei Qin updated HDFS-8201: -- Status: Patch Available (was: In Progress) Add an end to end test for stripping file writing and

[jira] [Updated] (HDFS-8201) Add an end to end test for stripping file writing and reading

2015-05-10 Thread Xinwei Qin (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8201?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xinwei Qin updated HDFS-8201: -- Attachment: HDFS-8201-HDFS-7285.003.patch Add an end to end test for stripping file writing and reading

[jira] [Commented] (HDFS-8201) Add an end to end test for stripping file writing and reading

2015-05-10 Thread Xinwei Qin (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8201?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14537478#comment-14537478 ] Xinwei Qin commented on HDFS-8201: --- Attached the patch with a name

[jira] [Updated] (HDFS-8201) Add an end to end test for stripping file writing and reading

2015-05-08 Thread Xinwei Qin (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8201?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xinwei Qin updated HDFS-8201: -- Attachment: HDFS-8201.002.patch Add an end to end test for stripping file writing and reading

[jira] [Updated] (HDFS-8202) Improve end to end stirpping file test to add erasure recovering test

2015-05-08 Thread Xinwei Qin (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8202?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xinwei Qin updated HDFS-8202: -- Attachment: HDFS-8202.001.patch Improve end to end stirpping file test to add erasure recovering test

[jira] [Commented] (HDFS-8201) Add an end to end test for stripping file writing and reading

2015-05-08 Thread Xinwei Qin (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8201?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14534230#comment-14534230 ] Xinwei Qin commented on HDFS-8201: --- The {{TestWriteReadStripedFile}} has implemented a

[jira] [Commented] (HDFS-8202) Improve end to end stirpping file test to add erasure recovering test

2015-05-08 Thread Xinwei Qin (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8202?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14534236#comment-14534236 ] Xinwei Qin commented on HDFS-8202: --- Initial patch for review. This patch is based on

[jira] [Updated] (HDFS-7859) Erasure Coding: Persist EC schemas in NameNode

2015-04-30 Thread Xinwei Qin (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-7859?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xinwei Qin updated HDFS-7859: -- Attachment: HDFS-7859-HDFS-7285.003.patch Erasure Coding: Persist EC schemas in NameNode

[jira] [Updated] (HDFS-8295) Add MODIFY and REMOVE ECSchema editlog operations

2015-04-30 Thread Xinwei Qin (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8295?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xinwei Qin updated HDFS-8295: -- Issue Type: Sub-task (was: Task) Parent: HDFS-8031 Add MODIFY and REMOVE ECSchema editlog

[jira] [Commented] (HDFS-7859) Erasure Coding: Persist EC schemas in NameNode

2015-04-30 Thread Xinwei Qin (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-7859?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14521029#comment-14521029 ] Xinwei Qin commented on HDFS-7859: --- The 003 patch removes MODIFY and REMOVE ECSchema

[jira] [Created] (HDFS-8295) Add MODIFY and REMOVE ECSchema editlog operations

2015-04-30 Thread Xinwei Qin (JIRA)
Xinwei Qin created HDFS-8295: - Summary: Add MODIFY and REMOVE ECSchema editlog operations Key: HDFS-8295 URL: https://issues.apache.org/jira/browse/HDFS-8295 Project: Hadoop HDFS Issue Type:

[jira] [Updated] (HDFS-8295) Add MODIFY and REMOVE ECSchema editlog operations

2015-04-30 Thread Xinwei Qin (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8295?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xinwei Qin updated HDFS-8295: -- Attachment: HDFS-8295.001.patch A initial patch based on HDFS-7859. Add MODIFY and REMOVE ECSchema

[jira] [Commented] (HDFS-7836) BlockManager Scalability Improvements

2015-04-29 Thread Xinwei Qin (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-7836?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14518976#comment-14518976 ] Xinwei Qin commented on HDFS-7836: --- Hi [~cmccabe], [~clamb], This is a very meaningful

[jira] [Assigned] (HDFS-8259) Erasure Coding: Test of reading EC file

2015-04-29 Thread Xinwei Qin (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8259?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xinwei Qin reassigned HDFS-8259: - Assignee: Xinwei Qin Erasure Coding: Test of reading EC file

[jira] [Assigned] (HDFS-8262) Erasure Coding: Test of datanode decommission which EC blocks are stored

2015-04-29 Thread Xinwei Qin (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8262?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xinwei Qin reassigned HDFS-8262: - Assignee: Xinwei Qin Erasure Coding: Test of datanode decommission which EC blocks are stored

[jira] [Assigned] (HDFS-8260) Erasure Coding: test of writing EC file

2015-04-29 Thread Xinwei Qin (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8260?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xinwei Qin reassigned HDFS-8260: - Assignee: Xinwei Qin Erasure Coding: test of writing EC file

[jira] [Commented] (HDFS-8201) Add an end to end test for stripping file writing and reading

2015-04-28 Thread Xinwei Qin (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8201?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14518545#comment-14518545 ] Xinwei Qin commented on HDFS-8201: --- [~zhz] I have noticed that JIRA. I think your

[jira] [Updated] (HDFS-7859) Erasure Coding: Persist EC schemas in NameNode

2015-04-27 Thread Xinwei Qin (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-7859?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xinwei Qin updated HDFS-7859: -- Status: Patch Available (was: In Progress) Erasure Coding: Persist EC schemas in NameNode

[jira] [Commented] (HDFS-7060) Avoid taking locks when sending heartbeats from the DataNode

2015-04-27 Thread Xinwei Qin (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-7060?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14516265#comment-14516265 ] Xinwei Qin commented on HDFS-7060: --- [~jnp] Thanks for your comment. This test was done

[jira] [Updated] (HDFS-7859) Erasure Coding: Persist EC schemas in NameNode

2015-04-26 Thread Xinwei Qin (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-7859?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xinwei Qin updated HDFS-7859: -- Attachment: HDFS-7859.002.patch Update the patch based on the latest HDFS-7285 branch and Kai's

[jira] [Work started] (HDFS-8201) Add an end to end test for stripping file writing and reading

2015-04-24 Thread Xinwei Qin (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8201?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Work on HDFS-8201 started by Xinwei Qin . - Add an end to end test for stripping file writing and reading

[jira] [Work started] (HDFS-8202) Improve end to end stirpping file test to add erasure recovering test

2015-04-24 Thread Xinwei Qin (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8202?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Work on HDFS-8202 started by Xinwei Qin . - Improve end to end stirpping file test to add erasure recovering test

[jira] [Updated] (HDFS-8201) Add an end to end test for stripping file writing and reading

2015-04-23 Thread Xinwei Qin (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8201?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xinwei Qin updated HDFS-8201: -- Attachment: HDFS-8201.001.patch Add an end to end test for stripping file writing and reading

[jira] [Updated] (HDFS-8202) Improve end to end stirpping file test to add erasure recovering test

2015-04-23 Thread Xinwei Qin (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8202?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xinwei Qin updated HDFS-8202: -- Description: This to follow on HDFS-8201 to add erasure recovering test in the end to end stripping

[jira] [Commented] (HDFS-8156) Add/implement necessary APIs even we just have the system default schema

2015-04-21 Thread Xinwei Qin (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8156?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14506221#comment-14506221 ] Xinwei Qin commented on HDFS-8156: --- A minor comment: The {{toString()}} method will

[jira] [Commented] (HDFS-8125) Erasure Coding: Expose refreshECSchemas command to reload predefined schemas

2015-04-17 Thread Xinwei Qin (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8125?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14499371#comment-14499371 ] Xinwei Qin commented on HDFS-8125: --- As HDFS-7859 and HDFS-7866 are all moved to

[jira] [Commented] (HDFS-8156) Define some system schemas in codes

2015-04-17 Thread Xinwei Qin (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8156?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14499359#comment-14499359 ] Xinwei Qin commented on HDFS-8156: --- Patch looks fine. Just a minor comment: {code} -

[jira] [Updated] (HDFS-8140) ECSchema supports for offline EditsVistor over an OEV XML file

2015-04-17 Thread Xinwei Qin (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8140?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xinwei Qin updated HDFS-8140: -- Issue Type: Sub-task (was: Task) Parent: HDFS-8031 ECSchema supports for offline EditsVistor

[jira] [Commented] (HDFS-8156) Define some system schemas in codes

2015-04-16 Thread Xinwei Qin (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8156?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14499161#comment-14499161 ] Xinwei Qin commented on HDFS-8156: --- Hi, [~drankye] Does the map field {{options}} of

[jira] [Commented] (HDFS-7859) Erasure Coding: Persist EC schemas in NameNode

2015-04-14 Thread Xinwei Qin (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-7859?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14493702#comment-14493702 ] Xinwei Qin commented on HDFS-7859: --- OK, I will track it. Erasure Coding: Persist EC

[jira] [Commented] (HDFS-7859) Erasure Coding: Persist EC schemas in NameNode

2015-04-14 Thread Xinwei Qin (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-7859?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14493699#comment-14493699 ] Xinwei Qin commented on HDFS-7859: --- [~drankye], thanks for your comments. {quote} 1.

[jira] [Commented] (HDFS-7859) Erasure Coding: Persist EC schemas in NameNode

2015-04-14 Thread Xinwei Qin (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-7859?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14493703#comment-14493703 ] Xinwei Qin commented on HDFS-7859: --- OK, I will track it. Erasure Coding: Persist EC

[jira] [Commented] (HDFS-7859) Erasure Coding: Persist EC schemas in NameNode

2015-04-14 Thread Xinwei Qin (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-7859?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14493704#comment-14493704 ] Xinwei Qin commented on HDFS-7859: --- OK, I will track it. Erasure Coding: Persist EC

[jira] [Created] (HDFS-8140) ECSchema supports for offline EditsVistor over an OEV XML file

2015-04-14 Thread Xinwei Qin (JIRA)
Xinwei Qin created HDFS-8140: - Summary: ECSchema supports for offline EditsVistor over an OEV XML file Key: HDFS-8140 URL: https://issues.apache.org/jira/browse/HDFS-8140 Project: Hadoop HDFS

[jira] [Commented] (HDFS-7866) Erasure coding: NameNode manages EC schemas

2015-04-14 Thread Xinwei Qin (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-7866?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14493727#comment-14493727 ] Xinwei Qin commented on HDFS-7866: --- OK, that sounds good. Erasure coding: NameNode

[jira] [Commented] (HDFS-7866) Erasure coding: NameNode manages EC schemas

2015-04-14 Thread Xinwei Qin (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-7866?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14493720#comment-14493720 ] Xinwei Qin commented on HDFS-7866: --- Hi, [~drankye] {code} +/** + * TODO:

[jira] [Updated] (HDFS-7859) Erasure Coding: Persist EC schemas in NameNode

2015-04-13 Thread Xinwei Qin (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-7859?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xinwei Qin updated HDFS-7859: -- Attachment: HDFS-7859.001.patch Post the patch for review. Erasure Coding: Persist EC schemas in

[jira] [Commented] (HDFS-7859) Erasure Coding: Persist EC schemas in NameNode

2015-04-13 Thread Xinwei Qin (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-7859?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14492394#comment-14492394 ] Xinwei Qin commented on HDFS-7859: --- Hi [~drankye] The patch has been completed, but is

[jira] [Commented] (HDFS-7859) Erasure Coding: Persist EC schemas in NameNode

2015-04-09 Thread Xinwei Qin (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-7859?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14487291#comment-14487291 ] Xinwei Qin commented on HDFS-7859: --- Hi [~drankye], Thanks for your clarification and

[jira] [Work started] (HDFS-7859) Erasure Coding: Persist EC schemas in NameNode

2015-04-08 Thread Xinwei Qin (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-7859?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Work on HDFS-7859 started by Xinwei Qin . - Erasure Coding: Persist EC schemas in NameNode

[jira] [Commented] (HDFS-7859) Erasure Coding: Persist EC schemas in NameNode

2015-04-02 Thread Xinwei Qin (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-7859?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14393895#comment-14393895 ] Xinwei Qin commented on HDFS-7859: --- Hi, [~drankye], I'm interested in this issue, if you

[jira] [Assigned] (HDFS-7859) Erasure Coding: Persist EC schemas in NameNode

2015-04-02 Thread Xinwei Qin (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-7859?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xinwei Qin reassigned HDFS-7859: - Assignee: Xinwei Qin (was: Kai Zheng) Erasure Coding: Persist EC schemas in NameNode

[jira] [Commented] (HDFS-7060) Avoid taking locks when sending heartbeats from the DataNode

2015-04-02 Thread Xinwei Qin (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-7060?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14392672#comment-14392672 ] Xinwei Qin commented on HDFS-7060: --- [~cmccabe] , [~wheat9] Thanks a lot for your

[jira] [Commented] (HDFS-7999) FsDatasetImpl#createTemporary sometimes holds the FSDatasetImpl lock for a very long time

2015-03-31 Thread Xinwei Qin (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-7999?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14388273#comment-14388273 ] Xinwei Qin commented on HDFS-7999: --- Yeah, It's a good and necessary idea to avoid

[jira] [Commented] (HDFS-7999) FsDatasetImpl#createTemporary sometimes holds the FSDatasetImpl lock for a very long time

2015-03-31 Thread Xinwei Qin (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-7999?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14388309#comment-14388309 ] Xinwei Qin commented on HDFS-7999: --- Hi [~cmccabe] Thanks for your comment. {quote} even

[jira] [Commented] (HDFS-7060) Contentions of the monitor of FsDatasetImpl block DN's heartbeat

2015-03-29 Thread Xinwei Qin (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-7060?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14386110#comment-14386110 ] Xinwei Qin commented on HDFS-7060: --- [~brahmareddy] Thanks for your review. bq. some

[jira] [Commented] (HDFS-7060) Contentions of the monitor of FsDatasetImpl block DN's heartbeat

2015-03-29 Thread Xinwei Qin (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-7060?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14386122#comment-14386122 ] Xinwei Qin commented on HDFS-7060: --- 2nd patch Contentions of the monitor of

[jira] [Commented] (HDFS-7060) Contentions of the monitor of FsDatasetImpl block DN's heartbeat

2015-03-29 Thread Xinwei Qin (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-7060?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14386121#comment-14386121 ] Xinwei Qin commented on HDFS-7060: --- Sorry, my mistake. I mean the 2rd patch, which

  1   2   >