[jira] [Created] (HDFS-13762) Support non-volatile memory or storage class memory(SCM) in HDFS cache

2018-07-24 Thread SammiChen (JIRA)
SammiChen created HDFS-13762: Summary: Support non-volatile memory or storage class memory(SCM) in HDFS cache Key: HDFS-13762 URL: https://issues.apache.org/jira/browse/HDFS-13762 Project: Hadoop HDFS

[jira] [Commented] (HDFS-13642) Creating a file with block size smaller than EC policy's cell size should throw

2018-06-08 Thread SammiChen (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-13642?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16505749#comment-16505749 ] SammiChen commented on HDFS-13642: -- [~xiaochen], sorry I didn't notice the second

[jira] [Commented] (HDFS-13642) Creating a file with block size smaller than EC policy's cell size should throw

2018-06-06 Thread SammiChen (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-13642?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16502852#comment-16502852 ] SammiChen commented on HDFS-13642: -- Right, the current {{hasErasureCodingPolicy}} doesn't check if it's

[jira] [Commented] (HDFS-13642) Creating a file with block size smaller than EC policy's cell size should throw

2018-06-04 Thread SammiChen (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-13642?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16499852#comment-16499852 ] SammiChen commented on HDFS-13642: -- Some comments, 1. *private static final int BLOCK_SIZE = 1 << 20;

[jira] [Commented] (HDFS-13642) Creating a file with block size smaller than EC policy's cell size should throw

2018-05-31 Thread SammiChen (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-13642?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16496266#comment-16496266 ] SammiChen commented on HDFS-13642: -- [~xiaochen], agree, NN should reject the request when the block size

[jira] [Updated] (HDFS-13540) DFSStripedInputStream should only allocate new buffers when reading

2018-05-23 Thread SammiChen (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-13540?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] SammiChen updated HDFS-13540: - Resolution: Fixed Target Version/s: 3.0.3 (was: 3.0.4) Status: Resolved (was:

[jira] [Commented] (HDFS-13540) DFSStripedInputStream should only allocate new buffers when reading

2018-05-23 Thread SammiChen (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-13540?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16487136#comment-16487136 ] SammiChen commented on HDFS-13540: -- +1. Thanks [~xiaochen] for the contribution. Committed to trunk,

[jira] [Updated] (HDFS-13540) DFSStripedInputStream should only allocate new buffers when reading

2018-05-23 Thread SammiChen (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-13540?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] SammiChen updated HDFS-13540: - Fix Version/s: 3.0.3 3.1.1 3.2.0 > DFSStripedInputStream should

[jira] [Commented] (HDFS-13540) DFSStripedInputStream should only allocate new buffers when reading

2018-05-22 Thread SammiChen (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-13540?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16483581#comment-16483581 ] SammiChen commented on HDFS-13540: -- Hi Xiao, the overall idea looks good to me. 1.  There are two

[jira] [Commented] (HDFS-13540) DFSStripedInputStream should not allocate new buffers during close / unbuffer

2018-05-17 Thread SammiChen (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-13540?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16478871#comment-16478871 ] SammiChen commented on HDFS-13540: -- [~xiaochen], thanks for the explanation. It makes sense to change the

[jira] [Commented] (HDFS-13540) DFSStripedInputStream should not allocate new buffers during close / unbuffer

2018-05-15 Thread SammiChen (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-13540?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16476804#comment-16476804 ] SammiChen commented on HDFS-13540: -- Hi [~xiaochen], thanks for working on this!

[jira] [Comment Edited] (HDFS-13388) RequestHedgingProxyProvider calls multiple configured NNs all the time

2018-04-10 Thread SammiChen (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-13388?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16431908#comment-16431908 ] SammiChen edited comment on HDFS-13388 at 4/10/18 8:22 AM: --- Hi [~LiJinglun] and

[jira] [Commented] (HDFS-13388) RequestHedgingProxyProvider calls multiple configured NNs all the time

2018-04-10 Thread SammiChen (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-13388?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16431908#comment-16431908 ] SammiChen commented on HDFS-13388: -- Hi [~LiJinglun] and [~elgoiri],  branch-2.9 suffers build failure

[jira] [Updated] (HDFS-11600) Refactor TestDFSStripedOutputStreamWithFailure test classes

2018-04-07 Thread SammiChen (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11600?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] SammiChen updated HDFS-11600: - Component/s: erasure-coding > Refactor TestDFSStripedOutputStreamWithFailure test classes >

[jira] [Updated] (HDFS-10183) Prevent race condition during class initialization

2018-04-02 Thread SammiChen (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10183?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] SammiChen updated HDFS-10183: - Fix Version/s: (was: 2.9.1) > Prevent race condition during class initialization >

[jira] [Updated] (HDFS-13337) Backport HDFS-4275 to branch-2.9

2018-04-02 Thread SammiChen (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-13337?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] SammiChen updated HDFS-13337: - Target Version/s: 2.10.0, 2.9.2 (was: 2.10.0, 2.9.1) > Backport HDFS-4275 to branch-2.9 >

[jira] [Updated] (HDFS-11885) createEncryptionZone should not block on initializing EDEK cache

2018-04-02 Thread SammiChen (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11885?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] SammiChen updated HDFS-11885: - Target Version/s: 2.8.3, 3.2.0, 2.9.2 (was: 2.8.3, 2.9.1, 3.2.0) > createEncryptionZone should not block

[jira] [Updated] (HDFS-12257) Expose getSnapshottableDirListing as a public API in HdfsAdmin

2018-04-02 Thread SammiChen (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12257?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] SammiChen updated HDFS-12257: - Target Version/s: 2.8.3, 3.2.0, 2.9.2 (was: 2.8.3, 2.9.1, 3.2.0) > Expose getSnapshottableDirListing as

[jira] [Updated] (HDFS-13051) dead lock occurs when rolleditlog rpc call happen and editPendingQ is full

2018-04-02 Thread SammiChen (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-13051?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] SammiChen updated HDFS-13051: - Target Version/s: 2.10.0, 2.8.4, 2.7.6, 3.0.2, 2.9.2 (was: 2.10.0, 2.9.1, 2.8.4, 2.7.6, 3.0.2) > dead

[jira] [Updated] (HDFS-13174) hdfs mover -p /path times out after 20 min

2018-04-02 Thread SammiChen (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-13174?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] SammiChen updated HDFS-13174: - Target Version/s: 3.0.1, 2.8.4, 2.7.6, 2.9.2 (was: 2.9.1, 3.0.1, 2.8.4, 2.7.6) > hdfs mover -p /path

[jira] [Updated] (HDFS-11600) Refactor TestDFSStripedOutputStreamWithFailure test classes

2018-03-14 Thread SammiChen (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11600?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] SammiChen updated HDFS-11600: - Fix Version/s: (was: 3.2.0) > Refactor TestDFSStripedOutputStreamWithFailure test classes >

[jira] [Updated] (HDFS-11600) Refactor TestDFSStripedOutputStreamWithFailure test classes

2018-03-14 Thread SammiChen (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11600?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] SammiChen updated HDFS-11600: - Target Version/s: 3.1.0, 3.0.2 (was: 3.1.0) > Refactor TestDFSStripedOutputStreamWithFailure test

[jira] [Updated] (HDFS-11600) Refactor TestDFSStripedOutputStreamWithFailure test classes

2018-03-14 Thread SammiChen (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11600?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] SammiChen updated HDFS-11600: - Resolution: Fixed Fix Version/s: 3.2.0 3.0.2 3.1.0

[jira] [Assigned] (HDFS-11600) Refactor TestDFSStripedOutputStreamWithFailure test classes

2018-03-14 Thread SammiChen (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11600?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] SammiChen reassigned HDFS-11600: Assignee: SammiChen > Refactor TestDFSStripedOutputStreamWithFailure test classes >

[jira] [Commented] (HDFS-11600) Refactor TestDFSStripedOutputStreamWithFailure test classes

2018-03-13 Thread SammiChen (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11600?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16396772#comment-16396772 ] SammiChen commented on HDFS-11600: -- Thanks [~xiaochen] for the review.  I uploaded 007 patch to address

[jira] [Updated] (HDFS-11600) Refactor TestDFSStripedOutputStreamWithFailure test classes

2018-03-13 Thread SammiChen (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11600?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] SammiChen updated HDFS-11600: - Attachment: HDFS-11600.007.patch > Refactor TestDFSStripedOutputStreamWithFailure test classes >

[jira] [Commented] (HDFS-11600) Refactor TestDFSStripedOutputStreamWithFailure test classes

2018-03-12 Thread SammiChen (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11600?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16396493#comment-16396493 ] SammiChen commented on HDFS-11600: -- Thanks [~xiaochen] for the comments. bq. Do you know why this range

[jira] [Updated] (HDFS-11600) Refactor TestDFSStripedOutputStreamWithFailure test classes

2018-03-12 Thread SammiChen (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11600?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] SammiChen updated HDFS-11600: - Attachment: HDFS-11600.006.patch > Refactor TestDFSStripedOutputStreamWithFailure test classes >

[jira] [Updated] (HDFS-11600) Refactor TestDFSStripedOutputStreamWithFailure test classes

2018-03-12 Thread SammiChen (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11600?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] SammiChen updated HDFS-11600: - Attachment: HDFS-11600.005.patch > Refactor TestDFSStripedOutputStreamWithFailure test classes >

[jira] [Comment Edited] (HDFS-11600) Refactor TestDFSStripedOutputStreamWithFailure test classes

2018-03-12 Thread SammiChen (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11600?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16394916#comment-16394916 ] SammiChen edited comment on HDFS-11600 at 3/12/18 8:37 AM: --- Hi [~rakeshr],

[jira] [Commented] (HDFS-11600) Refactor TestDFSStripedOutputStreamWithFailure test classes

2018-03-12 Thread SammiChen (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11600?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16394916#comment-16394916 ] SammiChen commented on HDFS-11600: -- Hi [~rakeshr], thanks for the comments. I will upload a new patch to

[jira] [Commented] (HDFS-11600) Refactor TestDFSStripedOutputStreamWithFailure test classes

2018-03-07 Thread SammiChen (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11600?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16390803#comment-16390803 ] SammiChen commented on HDFS-11600: -- Hi [~rakeshr], do you have time to help review the patch? >

[jira] [Commented] (HDFS-11600) Refactor TestDFSStripedOutputStreamWithFailure test classes

2018-03-07 Thread SammiChen (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11600?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16389328#comment-16389328 ] SammiChen commented on HDFS-11600: -- Handle javac issues and improve the {{testMultipleDatanodeFailure56}}

[jira] [Updated] (HDFS-11600) Refactor TestDFSStripedOutputStreamWithFailure test classes

2018-03-07 Thread SammiChen (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11600?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] SammiChen updated HDFS-11600: - Attachment: HDFS-11600.004.patch > Refactor TestDFSStripedOutputStreamWithFailure test classes >

[jira] [Commented] (HDFS-12654) APPEND API call is different in HTTPFS and NameNode REST

2018-03-05 Thread SammiChen (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12654?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16387350#comment-16387350 ] SammiChen commented on HDFS-12654: -- Hi, [~Nuke] and [~iwasakims], seem it's not a issue after the

[jira] [Commented] (HDFS-11600) Refactor TestDFSStripedOutputStreamWithFailure test classes

2018-03-05 Thread SammiChen (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11600?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16387171#comment-16387171 ] SammiChen commented on HDFS-11600: -- The output links in last build expired. Try to trigger the build

[jira] [Commented] (HDFS-11885) createEncryptionZone should not block on initializing EDEK cache

2018-03-01 Thread SammiChen (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11885?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16383049#comment-16383049 ] SammiChen commented on HDFS-11885: -- Is it still on target for 2.9.1?  If not, can we push this out from

[jira] [Comment Edited] (HDFS-12257) Expose getSnapshottableDirListing as a public API in HdfsAdmin

2018-03-01 Thread SammiChen (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12257?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16383028#comment-16383028 ] SammiChen edited comment on HDFS-12257 at 3/2/18 2:29 AM: -- Hi  [~HuafengWang], 

[jira] [Commented] (HDFS-12257) Expose getSnapshottableDirListing as a public API in HdfsAdmin

2018-03-01 Thread SammiChen (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12257?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16383028#comment-16383028 ] SammiChen commented on HDFS-12257: -- Hi  [~HuafengWang],  does this target for 2.9.1?  If not, can we push

[jira] [Commented] (HDFS-11600) Refactor TestDFSStripedOutputStreamWithFailure test classes

2018-02-11 Thread SammiChen (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11600?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16359850#comment-16359850 ] SammiChen commented on HDFS-11600: -- I uploaded a new patch based on Andrew's 002 patch. The idea is to

[jira] [Updated] (HDFS-11600) Refactor TestDFSStripedOutputStreamWithFailure test classes

2018-02-11 Thread SammiChen (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11600?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] SammiChen updated HDFS-11600: - Attachment: HDFS-11600.003.patch > Refactor TestDFSStripedOutputStreamWithFailure test classes >

[jira] [Updated] (HDFS-11600) Refactor TestDFSStripedOutputStreamWithFailure test classes

2018-02-11 Thread SammiChen (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11600?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] SammiChen updated HDFS-11600: - Status: Patch Available (was: Open) > Refactor TestDFSStripedOutputStreamWithFailure test classes >

[jira] [Commented] (HDFS-11600) Refactor TestDFSStripedOutputStreamWithFailure test classes

2018-02-09 Thread SammiChen (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11600?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16358077#comment-16358077 ] SammiChen commented on HDFS-11600: -- Hi [~andrew.wang],  the idea of using JUnit parameterize to really

[jira] [Commented] (HDFS-12462) Erasure coding policy extra options should be sorted by key value

2018-01-07 Thread SammiChen (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12462?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16315629#comment-16315629 ] SammiChen commented on HDFS-12462: -- Sure. Thanks [~liaoyuxiangqin] for be interested in this task.

[jira] [Commented] (HDFS-12860) StripedBlockUtil#getRangesInternalBlocks throws exception for the block group size larger than 2GB

2018-01-04 Thread SammiChen (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12860?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16311064#comment-16311064 ] SammiChen commented on HDFS-12860: -- Just come back from a long vacation. Sorry for the late response.

[jira] [Commented] (HDFS-12860) StripedBlockUtil#getRangesInternalBlocks throws exception for the block group size larger than 2GB

2017-12-21 Thread SammiChen (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12860?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16300987#comment-16300987 ] SammiChen commented on HDFS-12860: -- Hi [~eddyxu], thanks for report the issue and work on it. 1. It's

[jira] [Commented] (HDFS-12915) Fix findbugs warning in INodeFile$HeaderFormat.getBlockLayoutRedundancy

2017-12-21 Thread SammiChen (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12915?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16300908#comment-16300908 ] SammiChen commented on HDFS-12915: -- bq. IIRC the unit test failed because %02x was printing as a literal,

[jira] [Commented] (HDFS-12915) Fix findbugs warning in INodeFile$HeaderFormat.getBlockLayoutRedundancy

2017-12-17 Thread SammiChen (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12915?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16294575#comment-16294575 ] SammiChen commented on HDFS-12915: -- Thanks [~jojochuang] for reporting and working on it. Back to the

[jira] [Commented] (HDFS-12840) Creating a file with non-default EC policy in a EC zone is not correctly serialized in the editlog

2017-12-06 Thread SammiChen (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12840?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16281379#comment-16281379 ] SammiChen commented on HDFS-12840: -- Thanks [~eddyxu] for the contribution! The latest patch LGTM and +1.

[jira] [Commented] (HDFS-12896) when set replicate EC policy for a directory or file,it's EC policy cannot be querying by getPolicy command.

2017-12-06 Thread SammiChen (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12896?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16281229#comment-16281229 ] SammiChen commented on HDFS-12896: -- Hi [~candychencan], HDFS-12308 is about to provide a new API other

[jira] [Commented] (HDFS-12896) when set replicate EC policy for a directory or file,it's EC policy cannot be querying by getPolicy command.

2017-12-05 Thread SammiChen (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12896?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16279723#comment-16279723 ] SammiChen commented on HDFS-12896: -- Hi [~candychencan], sure, welcome to contribute to the community!

[jira] [Commented] (HDFS-12840) Creating a file with non-default EC policy in a EC zone is not correctly serialized in the editlog

2017-12-05 Thread SammiChen (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12840?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16279711#comment-16279711 ] SammiChen commented on HDFS-12840: -- Hi [~eddyxu], TestOfflineEditsViewer with editsStored.03 can be

[jira] [Commented] (HDFS-12896) when set replicate EC policy for a directory or file,it's EC policy cannot be querying by getPolicy command.

2017-12-05 Thread SammiChen (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12896?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16279618#comment-16279618 ] SammiChen commented on HDFS-12896: -- Hi [~candychencan], thanks for reporting this! Currently it's the

[jira] [Commented] (HDFS-12840) Creating a file with non-default EC policy in a EC zone is not correctly serialized in the editlog

2017-12-04 Thread SammiChen (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12840?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16278150#comment-16278150 ] SammiChen commented on HDFS-12840: -- Thanks [~eddyxu] ! The latest patch looks overall good. 1.

[jira] [Commented] (HDFS-12840) Creating a replicated file in a EC zone does not correctly serialized in EditLogs

2017-11-29 Thread SammiChen (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12840?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16272270#comment-16272270 ] SammiChen commented on HDFS-12840: -- Hi Eddy, thanks for working on it. some comments here, 1.

[jira] [Commented] (HDFS-12840) Creating a replicated file in a EC zone does not correctly serialized in EditLogs

2017-11-23 Thread SammiChen (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12840?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16264935#comment-16264935 ] SammiChen commented on HDFS-12840: -- Hi [~eddyxu], thanks for reporting and fix this. I'm not able to

[jira] [Commented] (HDFS-12682) ECAdmin -listPolicies will always show SystemErasureCodingPolicies state as DISABLED

2017-10-31 Thread SammiChen (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12682?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16226598#comment-16226598 ] SammiChen commented on HDFS-12682: -- HI [~xiaochen], thanks for the update. The patch looks overall good.

[jira] [Commented] (HDFS-12686) Erasure coding system policy state is not correctly saved and loaded during real cluster restart

2017-10-24 Thread SammiChen (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12686?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16217973#comment-16217973 ] SammiChen commented on HDFS-12686: -- Hi [~jojochuang], this JIRA is closely related with HDFS-12682. I was

[jira] [Assigned] (HDFS-12046) Hadoop CRC implementation using Intel ISA-L library

2017-10-24 Thread SammiChen (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12046?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] SammiChen reassigned HDFS-12046: Assignee: SammiChen (was: luhuichun) > Hadoop CRC implementation using Intel ISA-L library >

[jira] [Commented] (HDFS-12682) ECAdmin -listPolicies will always show policy state as DISABLED

2017-10-19 Thread SammiChen (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12682?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16210790#comment-16210790 ] SammiChen commented on HDFS-12682: -- Hi [~xiaochen], thanks for reporting this issue. Inspired by your

[jira] [Created] (HDFS-12686) Erasure coding system policy state is not correctly saved and loaded during real cluster restart

2017-10-19 Thread SammiChen (JIRA)
SammiChen created HDFS-12686: Summary: Erasure coding system policy state is not correctly saved and loaded during real cluster restart Key: HDFS-12686 URL: https://issues.apache.org/jira/browse/HDFS-12686

[jira] [Commented] (HDFS-7337) Configurable and pluggable erasure codec and policy

2017-10-19 Thread SammiChen (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-7337?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16210640#comment-16210640 ] SammiChen commented on HDFS-7337: - Hi [~xiaochen], HDFS-7859 is for persist EC policy in protobuffer

[jira] [Commented] (HDFS-12613) Native EC coder should implement release() as idempotent function.

2017-10-16 Thread SammiChen (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12613?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16206917#comment-16206917 ] SammiChen commented on HDFS-12613: -- HDFS-12672 is fired for track. Will prepare a Window environment

[jira] [Assigned] (HDFS-12672) Verify erasure coding native code on Windows platform

2017-10-16 Thread SammiChen (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12672?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] SammiChen reassigned HDFS-12672: Assignee: SammiChen > Verify erasure coding native code on Windows platform >

[jira] [Created] (HDFS-12672) Verify erasure coding native code on Windows platform

2017-10-16 Thread SammiChen (JIRA)
SammiChen created HDFS-12672: Summary: Verify erasure coding native code on Windows platform Key: HDFS-12672 URL: https://issues.apache.org/jira/browse/HDFS-12672 Project: Hadoop HDFS Issue

[jira] [Commented] (HDFS-11467) Support ErasureCoding section in OIV XML/ReverseXML

2017-10-15 Thread SammiChen (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11467?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16205451#comment-16205451 ] SammiChen commented on HDFS-11467: -- Thanks [~HuafengWang] for working on it! 1. Suggest move the new

[jira] [Commented] (HDFS-12575) Improve test coverage for EC related edit logs ops

2017-10-12 Thread SammiChen (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12575?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16201668#comment-16201668 ] SammiChen commented on HDFS-12575: -- Hi [~eddyxu], sure, I will work on it. I'm not clear about the detail

[jira] [Commented] (HDFS-12613) Native EC coder should implement release() as idempotent function.

2017-10-11 Thread SammiChen (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12613?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16201395#comment-16201395 ] SammiChen commented on HDFS-12613: -- Hi [~eddyxu], agree. Check NULL pointer in native code is a

[jira] [Commented] (HDFS-12613) Native EC coder should implement release() as idempotent function.

2017-10-11 Thread SammiChen (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12613?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16199928#comment-16199928 ] SammiChen commented on HDFS-12613: -- Hi [~eddyxu], thanks for reporting and working on it. 1. Apart

[jira] [Commented] (HDFS-12569) Unset EC policy logs empty payload in edit log

2017-09-29 Thread SammiChen (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12569?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16186826#comment-16186826 ] SammiChen commented on HDFS-12569: -- Hi Andrew, Next week is PRC National Day. We will all take 1 week

[jira] [Commented] (HDFS-7337) Configurable and pluggable erasure codec and policy

2017-09-28 Thread SammiChen (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-7337?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16185280#comment-16185280 ] SammiChen commented on HDFS-7337: - Hi [~andrew.wang], the release note is ready. Is there anything I need

[jira] [Commented] (HDFS-12497) Re-enable TestDFSStripedOutputStreamWithFailure tests

2017-09-27 Thread SammiChen (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12497?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16183560#comment-16183560 ] SammiChen commented on HDFS-12497: -- Thanks Huafeng for taking over the task! > Re-enable

[jira] [Commented] (HDFS-12399) Improve erasure coding codec framework adding more unit tests

2017-09-26 Thread SammiChen (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12399?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16182027#comment-16182027 ] SammiChen commented on HDFS-12399: -- I double checked test case failures. They are not relevant. Ping

[jira] [Commented] (HDFS-12497) Re-enable TestDFSStripedOutputStreamWithFailure tests

2017-09-21 Thread SammiChen (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12497?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16174379#comment-16174379 ] SammiChen commented on HDFS-12497: -- I see the {{testMultipleDatanodeFailure56}} failure too. By reducing

[jira] [Commented] (HDFS-12449) TestReconstructStripedFile.testNNSendsErasureCodingTasks randomly cannot finish in 60s

2017-09-19 Thread SammiChen (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12449?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16172611#comment-16172611 ] SammiChen commented on HDFS-12449: -- Thanks [~eddyxu] for review and commit the patch! >

[jira] [Assigned] (HDFS-12497) Re-enable TestDFSStripedOutputStreamWithFailure tests

2017-09-19 Thread SammiChen (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12497?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] SammiChen reassigned HDFS-12497: Assignee: SammiChen > Re-enable TestDFSStripedOutputStreamWithFailure tests >

[jira] [Commented] (HDFS-12447) Refactor addErasureCodingPolicy

2017-09-19 Thread SammiChen (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12447?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16171525#comment-16171525 ] SammiChen commented on HDFS-12447: -- Build failed at step "Apache Hadoop Client Packaging Invariants" and

[jira] [Updated] (HDFS-12399) Improve erasure coding codec framework adding more unit tests

2017-09-19 Thread SammiChen (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12399?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] SammiChen updated HDFS-12399: - Attachment: HDFS-12399.000.patch Initial patch. > Improve erasure coding codec framework adding more

[jira] [Commented] (HDFS-12449) TestReconstructStripedFile.testNNSendsErasureCodingTasks randomly cannot finish in 60s

2017-09-19 Thread SammiChen (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12449?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16171236#comment-16171236 ] SammiChen commented on HDFS-12449: -- Hi [~eddyxu], do you have time to take a look at the patch? I double

[jira] [Updated] (HDFS-12447) Refactor addErasureCodingPolicy

2017-09-19 Thread SammiChen (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12447?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] SammiChen updated HDFS-12447: - Attachment: HDFS-12447.004.patch Rebase the patch against trunk. > Refactor addErasureCodingPolicy >

[jira] [Commented] (HDFS-12395) Support erasure coding policy operations in namenode edit log

2017-09-19 Thread SammiChen (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12395?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16171232#comment-16171232 ] SammiChen commented on HDFS-12395: -- Thanks [~kihwal] for the reminder of taking care the NN layout

[jira] [Commented] (HDFS-12395) Support erasure coding policy operations in namenode edit log

2017-09-18 Thread SammiChen (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12395?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16169666#comment-16169666 ] SammiChen commented on HDFS-12395: -- Hi, [~brahmareddy], thanks for the reminder. I will address these two

[jira] [Updated] (HDFS-12447) Refactor addErasureCodingPolicy

2017-09-15 Thread SammiChen (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12447?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] SammiChen updated HDFS-12447: - Attachment: HDFS-12447.002.patch Improve the patch after offline discussion with Kai. > Refactor

[jira] [Updated] (HDFS-12460) make addErasureCodingPolicy an idempotent operation

2017-09-15 Thread SammiChen (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12460?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] SammiChen updated HDFS-12460: - Attachment: HDFS-12460.001.patch Initial patch. > make addErasureCodingPolicy an idempotent operation >

[jira] [Updated] (HDFS-12460) make addErasureCodingPolicy an idempotent operation

2017-09-15 Thread SammiChen (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12460?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] SammiChen updated HDFS-12460: - Description: Make addErasureCodingPolicy an idempotent operation to guarantee after HA switch,

[jira] [Updated] (HDFS-12460) make addErasureCodingPolicy an idempotent operation

2017-09-15 Thread SammiChen (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12460?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] SammiChen updated HDFS-12460: - Description: Make addErasureCodingPolicy an idempotent operation to guarantee after HA switch, all edit

[jira] [Updated] (HDFS-12460) make addErasureCodingPolicy an idempotent operation

2017-09-15 Thread SammiChen (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12460?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] SammiChen updated HDFS-12460: - Issue Type: Improvement (was: Bug) > make addErasureCodingPolicy an idempotent operation >

[jira] [Updated] (HDFS-12460) make addErasureCodingPolicy an idempotent operation

2017-09-15 Thread SammiChen (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12460?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] SammiChen updated HDFS-12460: - Summary: make addErasureCodingPolicy an idempotent operation (was: addErasureCodingPolicy should) >

[jira] [Updated] (HDFS-12460) addErasureCodingPolicy should

2017-09-15 Thread SammiChen (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12460?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] SammiChen updated HDFS-12460: - Summary: addErasureCodingPolicy should (was: TestNamenodeRetryCache.testRetryCacheRebuild unit test case

[jira] [Updated] (HDFS-12462) Erasure coding policy extra options should be sorted by key value

2017-09-14 Thread SammiChen (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12462?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] SammiChen updated HDFS-12462: - Labels: hdfs-ec-3.0-nice-to-have (was: ) > Erasure coding policy extra options should be sorted by key

[jira] [Updated] (HDFS-12462) Erasure coding policy extra options should be sorted by key value

2017-09-14 Thread SammiChen (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12462?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] SammiChen updated HDFS-12462: - Component/s: erasure-coding > Erasure coding policy extra options should be sorted by key value >

[jira] [Created] (HDFS-12462) Erasure coding policy extra options should be sorted by key value

2017-09-14 Thread SammiChen (JIRA)
SammiChen created HDFS-12462: Summary: Erasure coding policy extra options should be sorted by key value Key: HDFS-12462 URL: https://issues.apache.org/jira/browse/HDFS-12462 Project: Hadoop HDFS

[jira] [Updated] (HDFS-12447) Refactor addErasureCodingPolicy

2017-09-14 Thread SammiChen (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12447?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] SammiChen updated HDFS-12447: - Attachment: HDFS-12447.001.patch > Refactor addErasureCodingPolicy > --- > >

[jira] [Updated] (HDFS-12447) Refactor addErasureCodingPolicy

2017-09-14 Thread SammiChen (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12447?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] SammiChen updated HDFS-12447: - Status: Patch Available (was: Open) > Refactor addErasureCodingPolicy > ---

[jira] [Created] (HDFS-12460) TestNamenodeRetryCache.testRetryCacheRebuild unit test case failure

2017-09-14 Thread SammiChen (JIRA)
SammiChen created HDFS-12460: Summary: TestNamenodeRetryCache.testRetryCacheRebuild unit test case failure Key: HDFS-12460 URL: https://issues.apache.org/jira/browse/HDFS-12460 Project: Hadoop HDFS

[jira] [Commented] (HDFS-7859) Erasure Coding: Persist erasure coding policies in NameNode

2017-09-14 Thread SammiChen (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-7859?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16167239#comment-16167239 ] SammiChen commented on HDFS-7859: - Sure. Release note is ready. Thanks [~drankye], [~eddyxu],

[jira] [Updated] (HDFS-7859) Erasure Coding: Persist erasure coding policies in NameNode

2017-09-14 Thread SammiChen (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-7859?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] SammiChen updated HDFS-7859: Release Note: Persist all built-in erasure coding policies and user defined erasure coding policies into

[jira] [Updated] (HDFS-12449) TestReconstructStripedFile.testNNSendsErasureCodingTasks randomly cannot finish in 60s

2017-09-14 Thread SammiChen (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12449?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] SammiChen updated HDFS-12449: - Description: TestReconstructStripedFile.testNNSendsErasureCodingTasks randomly cannot finish in 60s

[jira] [Updated] (HDFS-12449) TestReconstructStripedFile.testNNSendsErasureCodingTasks randomly cannot finish in 60s

2017-09-14 Thread SammiChen (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12449?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] SammiChen updated HDFS-12449: - Status: Patch Available (was: Open) > TestReconstructStripedFile.testNNSendsErasureCodingTasks randomly

[jira] [Updated] (HDFS-12449) TestReconstructStripedFile.testNNSendsErasureCodingTasks randomly cannot finish in 60s

2017-09-14 Thread SammiChen (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12449?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] SammiChen updated HDFS-12449: - Attachment: HDFS-12449.001.patch > TestReconstructStripedFile.testNNSendsErasureCodingTasks randomly

  1   2   3   4   5   6   7   >