[jira] [Updated] (HDFS-10718) Prefer direct ByteBuffer in native RS encoder and decoder

2016-08-03 Thread SammiChen (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10718?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] SammiChen updated HDFS-10718: - Attachment: HDFS-10718-v1.patch > Prefer direct ByteBuffer in native RS encoder and decoder >

[jira] [Updated] (HDFS-10718) Prefer direct ByteBuffer in native RS encoder and decoder

2016-08-03 Thread SammiChen (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10718?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] SammiChen updated HDFS-10718: - Status: Patch Available (was: Open) > Prefer direct ByteBuffer in native RS encoder and decoder >

[jira] [Created] (HDFS-10718) Prefer direct ByteBuffer in native RS encoder and decoder

2016-08-03 Thread SammiChen (JIRA)
SammiChen created HDFS-10718: Summary: Prefer direct ByteBuffer in native RS encoder and decoder Key: HDFS-10718 URL: https://issues.apache.org/jira/browse/HDFS-10718 Project: Hadoop HDFS Issue

[jira] [Updated] (HDFS-8668) Erasure Coding: revisit buffer used for encoding and decoding.

2016-08-10 Thread SammiChen (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8668?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] SammiChen updated HDFS-8668: Attachment: HDFS-8668-v6.patch > Erasure Coding: revisit buffer used for encoding and decoding. >

[jira] [Commented] (HDFS-8668) Erasure Coding: revisit buffer used for encoding and decoding.

2016-08-10 Thread SammiChen (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8668?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15415276#comment-15415276 ] SammiChen commented on HDFS-8668: - 1. fix checkstyle 2. allocate direct ByteBuffer for checksumBuf per

[jira] [Updated] (HDFS-8668) Erasure Coding: revisit buffer used for encoding and decoding.

2016-08-11 Thread SammiChen (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8668?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] SammiChen updated HDFS-8668: Attachment: HDFS-8668-v8.patch > Erasure Coding: revisit buffer used for encoding and decoding. >

[jira] [Commented] (HDFS-8668) Erasure Coding: revisit buffer used for encoding and decoding.

2016-08-11 Thread SammiChen (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8668?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15417006#comment-15417006 ] SammiChen commented on HDFS-8668: - fix checkstyle issues > Erasure Coding: revisit buffer used for

[jira] [Commented] (HDFS-8668) Erasure Coding: revisit buffer used for encoding and decoding.

2016-08-11 Thread SammiChen (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8668?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15418280#comment-15418280 ] SammiChen commented on HDFS-8668: - Thanks for Kai's review. A new patch is uploaded, fix all mentioned

[jira] [Updated] (HDFS-8668) Erasure Coding: revisit buffer used for encoding and decoding.

2016-08-04 Thread SammiChen (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8668?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] SammiChen updated HDFS-8668: Attachment: HDFS-8668-v3.patch > Erasure Coding: revisit buffer used for encoding and decoding. >

[jira] [Updated] (HDFS-8668) Erasure Coding: revisit buffer used for encoding and decoding.

2016-08-12 Thread SammiChen (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8668?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] SammiChen updated HDFS-8668: Attachment: HDFS-8668-v9.patch > Erasure Coding: revisit buffer used for encoding and decoding. >

[jira] [Updated] (HDFS-8668) Erasure Coding: revisit buffer used for encoding and decoding.

2016-08-10 Thread SammiChen (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8668?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] SammiChen updated HDFS-8668: Attachment: HDFS-8668-v7.patch > Erasure Coding: revisit buffer used for encoding and decoding. >

[jira] [Commented] (HDFS-8668) Erasure Coding: revisit buffer used for encoding and decoding.

2016-08-11 Thread SammiChen (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8668?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15416613#comment-15416613 ] SammiChen commented on HDFS-8668: - Update test file TestReconstructStripedFile,

[jira] [Updated] (HDFS-10718) Prefer direct ByteBuffer in native RS encoder and decoder

2016-08-08 Thread SammiChen (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10718?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] SammiChen updated HDFS-10718: - Attachment: (was: HDFS-8668-v4.patch) > Prefer direct ByteBuffer in native RS encoder and decoder >

[jira] [Commented] (HDFS-8668) Erasure Coding: revisit buffer used for encoding and decoding.

2016-08-08 Thread SammiChen (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8668?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15412992#comment-15412992 ] SammiChen commented on HDFS-8668: - Thanks for Kai's suggestion. The heap buffer convert process is removed

[jira] [Updated] (HDFS-10718) Prefer direct ByteBuffer in native RS encoder and decoder

2016-08-08 Thread SammiChen (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10718?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] SammiChen updated HDFS-10718: - Attachment: HDFS-8668-v4.patch > Prefer direct ByteBuffer in native RS encoder and decoder >

[jira] [Updated] (HDFS-8668) Erasure Coding: revisit buffer used for encoding and decoding.

2016-08-08 Thread SammiChen (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8668?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] SammiChen updated HDFS-8668: Attachment: HDFS-8668-v4.patch > Erasure Coding: revisit buffer used for encoding and decoding. >

[jira] [Updated] (HDFS-8668) Erasure Coding: revisit buffer used for encoding and decoding.

2016-08-09 Thread SammiChen (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8668?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] SammiChen updated HDFS-8668: Attachment: HDFS-8668-v5.patch > Erasure Coding: revisit buffer used for encoding and decoding. >

[jira] [Commented] (HDFS-11152) Start erasure coding policy ID number from 1 instead of 0 to void potential unexpected errors

2017-01-19 Thread SammiChen (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11152?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15829676#comment-15829676 ] SammiChen commented on HDFS-11152: -- Hi [~andrew.wang]] I did some investigation. It seems EC policy ID is

[jira] [Comment Edited] (HDFS-11268) Correctly reconstruct erasure coding file from FSImage

2017-01-18 Thread SammiChen (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11268?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15829394#comment-15829394 ] SammiChen edited comment on HDFS-11268 at 1/19/17 6:31 AM: --- Hi [~andrew.wang],

[jira] [Updated] (HDFS-11268) Correctly reconstruct erasure coding file from FSImage

2017-01-18 Thread SammiChen (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11268?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] SammiChen updated HDFS-11268: - Attachment: HDFS-11268-002.patch Hi [~andrew.wang]], thanks for reviewing the patch! I rebased it. >

[jira] [Updated] (HDFS-11268) Correctly reconstruct erasure coding file from FSImage

2017-01-16 Thread SammiChen (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11268?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] SammiChen updated HDFS-11268: - Description: Currently, FSImageFormatProtoBuf has the information about whether the file is striped or

[jira] [Commented] (HDFS-11268) Correctly reconstruct erasure coding file from FSImage

2017-01-16 Thread SammiChen (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11268?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15823810#comment-15823810 ] SammiChen commented on HDFS-11268: -- Hi [~jojochuang]], you're right, EC policy ID is stored using

[jira] [Updated] (HDFS-11268) Correctly reconstruct erasure coding file from FSImage

2017-01-16 Thread SammiChen (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11268?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] SammiChen updated HDFS-11268: - Summary: Correctly reconstruct erasure coding file from FSImage (was: Persist erasure coding policy ID

[jira] [Updated] (HDFS-11268) Correctly reconstruct erasure coding file from FSImage

2017-01-16 Thread SammiChen (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11268?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] SammiChen updated HDFS-11268: - Attachment: HDFS-11268-001.patch Initial patch > Correctly reconstruct erasure coding file from FSImage

[jira] [Updated] (HDFS-11268) Correctly reconstruct erasure coding file from FSImage

2017-01-16 Thread SammiChen (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11268?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] SammiChen updated HDFS-11268: - Status: Patch Available (was: Open) > Correctly reconstruct erasure coding file from FSImage >

[jira] [Commented] (HDFS-11268) Correctly reconstruct erasure coding file from FSImage

2017-01-19 Thread SammiChen (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11268?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15831073#comment-15831073 ] SammiChen commented on HDFS-11268: -- Thanks Andrew! > Correctly reconstruct erasure coding file from

[jira] [Updated] (HDFS-11152) Start erasure coding policy ID number from 1 instead of 0 to void potential unexpected errors

2017-01-20 Thread SammiChen (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-11152?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] SammiChen updated HDFS-11152: - Attachment: HDFS-11152-v2.patch 1. Add EC policy ID check when loading file from fsImage 2. Add EC policy

[jira] [Updated] (HDFS-8905) Refactor DFSInputStream#ReaderStrategy

2016-08-21 Thread SammiChen (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8905?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] SammiChen updated HDFS-8905: Attachment: HDFS-8905-v13.patch remove the unnecessary whitespace > Refactor DFSInputStream#ReaderStrategy

[jira] [Issue Comment Deleted] (HDFS-8905) Refactor DFSInputStream#ReaderStrategy

2016-08-19 Thread SammiChen (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8905?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] SammiChen updated HDFS-8905: Comment: was deleted (was: rebase the patch on latest trunk code ) > Refactor DFSInputStream#ReaderStrategy

[jira] [Updated] (HDFS-8905) Refactor DFSInputStream#ReaderStrategy

2016-08-19 Thread SammiChen (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8905?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] SammiChen updated HDFS-8905: Attachment: HDFS-8905-v10.patch rebase the patch on latest trunk code > Refactor

[jira] [Updated] (HDFS-8905) Refactor DFSInputStream#ReaderStrategy

2016-08-19 Thread SammiChen (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8905?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] SammiChen updated HDFS-8905: Attachment: (was: HDFS-8905-v10.patch) > Refactor DFSInputStream#ReaderStrategy >

[jira] [Updated] (HDFS-8905) Refactor DFSInputStream#ReaderStrategy

2016-08-19 Thread SammiChen (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8905?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] SammiChen updated HDFS-8905: Attachment: HDFS-8905-v12.patch fix check style reported issues > Refactor DFSInputStream#ReaderStrategy >

[jira] [Commented] (HDFS-8905) Refactor DFSInputStream#ReaderStrategy

2016-08-19 Thread SammiChen (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8905?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15427992#comment-15427992 ] SammiChen commented on HDFS-8905: - fix the issue found in unit test failure > Refactor

[jira] [Updated] (HDFS-8905) Refactor DFSInputStream#ReaderStrategy

2016-08-19 Thread SammiChen (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8905?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] SammiChen updated HDFS-8905: Attachment: HDFS-8905-v11.patch > Refactor DFSInputStream#ReaderStrategy >

[jira] [Updated] (HDFS-8905) Refactor DFSInputStream#ReaderStrategy

2016-08-24 Thread SammiChen (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8905?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] SammiChen updated HDFS-8905: Attachment: HDFS-8905-v15.patch Rebased against the latest trunk code > Refactor

[jira] [Created] (HDFS-10795) Refactor ReaderStrategy#ByteBufferStrategy

2016-08-25 Thread SammiChen (JIRA)
SammiChen created HDFS-10795: Summary: Refactor ReaderStrategy#ByteBufferStrategy Key: HDFS-10795 URL: https://issues.apache.org/jira/browse/HDFS-10795 Project: Hadoop HDFS Issue Type:

[jira] [Updated] (HDFS-10795) Refactor ReaderStrategy#ByteBufferStrategy

2016-08-25 Thread SammiChen (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10795?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] SammiChen updated HDFS-10795: - Assignee: SammiChen Status: Patch Available (was: Open) > Refactor

[jira] [Updated] (HDFS-10795) Refactor ReaderStrategy#ByteBufferStrategy

2016-08-25 Thread SammiChen (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10795?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] SammiChen updated HDFS-10795: - Attachment: HDFS-10795-v1.patch upload the patch > Refactor ReaderStrategy#ByteBufferStrategy >

[jira] [Updated] (HDFS-8901) Use ByteBuffer in striping positional read

2016-09-02 Thread SammiChen (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8901?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] SammiChen updated HDFS-8901: Attachment: HDFS-8901-v19.patch 1. fix TestSnapshotFileLength test failure 2. fix 2 checkstyle issues. the

[jira] [Updated] (HDFS-8901) Use ByteBuffer in striping positional read

2016-09-05 Thread SammiChen (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8901?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] SammiChen updated HDFS-8901: Attachment: HDFS-8901-v20.patch 1. remove "//buffer.position(buffer.position() + bytesToRead)" 2. use

[jira] [Commented] (HDFS-8901) Use ByteBuffer in striping positional read

2016-09-05 Thread SammiChen (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8901?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15464454#comment-15464454 ] SammiChen commented on HDFS-8901: - Thanks Zhe! Thanks Kai! bq. Why do we need to flip? In the previous

[jira] [Commented] (HDFS-8901) Use ByteBuffer in striping positional read

2016-09-05 Thread SammiChen (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8901?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15466024#comment-15466024 ] SammiChen commented on HDFS-8901: - 1. Run {{hadoop.net.TestDNS}} locally, it passed without any failure.

[jira] [Commented] (HDFS-8901) Use ByteBuffer in striping positional read

2016-09-01 Thread SammiChen (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8901?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15455103#comment-15455103 ] SammiChen commented on HDFS-8901: - Thanks Zhe for take a lot of time review patch and give so many good

[jira] [Commented] (HDFS-8901) Use ByteBuffer in striping positional read

2016-09-01 Thread SammiChen (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8901?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15455110#comment-15455110 ] SammiChen commented on HDFS-8901: - attach patch v17, improve the code based on Zhe's good suggestions >

[jira] [Updated] (HDFS-8901) Use ByteBuffer in striping positional read

2016-09-01 Thread SammiChen (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8901?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] SammiChen updated HDFS-8901: Attachment: HDFS-8901-v17.patch > Use ByteBuffer in striping positional read >

[jira] [Updated] (HDFS-8901) Use ByteBuffer in striping positional read

2016-09-02 Thread SammiChen (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8901?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] SammiChen updated HDFS-8901: Attachment: HDFS-8901-v18.patch > Use ByteBuffer in striping positional read >

[jira] [Commented] (HDFS-8905) Refactor DFSInputStream#ReaderStrategy

2016-09-08 Thread SammiChen (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8905?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15473069#comment-15473069 ] SammiChen commented on HDFS-8905: - Thanks, Akira Ajisaka! Sure, I will look into it. > Refactor

[jira] [Commented] (HDFS-8905) Refactor DFSInputStream#ReaderStrategy

2016-09-08 Thread SammiChen (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8905?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15473070#comment-15473070 ] SammiChen commented on HDFS-8905: - Thanks, Akira Ajisaka! Sure, I will look into it. > Refactor

[jira] [Issue Comment Deleted] (HDFS-8905) Refactor DFSInputStream#ReaderStrategy

2016-09-08 Thread SammiChen (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8905?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] SammiChen updated HDFS-8905: Comment: was deleted (was: Thanks, Akira Ajisaka! Sure, I will look into it.) > Refactor

[jira] [Commented] (HDFS-10844) test_libhdfs_threaded_hdfs_static and test_libhdfs_zerocopy_hdfs_static are failing

2016-09-08 Thread SammiChen (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10844?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15473572#comment-15473572 ] SammiChen commented on HDFS-10844: -- Thanks Akira Ajisaka and Allen Wittenauer for all the effort to

[jira] [Updated] (HDFS-10839) Add new BlockReaderl#readAll(ByteBuffer buf, int len) API

2016-09-04 Thread SammiChen (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10839?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] SammiChen updated HDFS-10839: - Attachment: HDFS-10839-v1.patch implement the new readAll API > Add new BlockReaderl#readAll(ByteBuffer

[jira] [Updated] (HDFS-10839) Add new BlockReaderl#readAll(ByteBuffer buf, int len) API

2016-09-04 Thread SammiChen (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10839?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] SammiChen updated HDFS-10839: - Status: Patch Available (was: Open) > Add new BlockReaderl#readAll(ByteBuffer buf, int len) API >

[jira] [Updated] (HDFS-10839) Add new BlockReaderl#readAll(ByteBuffer buf, int len) API

2016-09-04 Thread SammiChen (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10839?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] SammiChen updated HDFS-10839: - Status: Open (was: Patch Available) > Add new BlockReaderl#readAll(ByteBuffer buf, int len) API >

[jira] [Updated] (HDFS-10839) Add new BlockReaderl#readAll(ByteBuffer buf, int len) API

2016-09-04 Thread SammiChen (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10839?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] SammiChen updated HDFS-10839: - Attachment: HDFS-10839-v1.patch > Add new BlockReaderl#readAll(ByteBuffer buf, int len) API >

[jira] [Updated] (HDFS-10839) Add new BlockReaderl#readAll(ByteBuffer buf, int len) API

2016-09-04 Thread SammiChen (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10839?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] SammiChen updated HDFS-10839: - Attachment: (was: HDFS-10839-v1.patch) > Add new BlockReaderl#readAll(ByteBuffer buf, int len) API >

[jira] [Updated] (HDFS-10839) Add new BlockReaderl#readAll(ByteBuffer buf, int len) API

2016-09-04 Thread SammiChen (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10839?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] SammiChen updated HDFS-10839: - Status: Patch Available (was: Open) > Add new BlockReaderl#readAll(ByteBuffer buf, int len) API >

[jira] [Updated] (HDFS-10839) Add new BlockReader#readAll(ByteBuffer buf, int len) API

2016-09-04 Thread SammiChen (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10839?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] SammiChen updated HDFS-10839: - Description: Add a ByteBuffer version readAll API in BlockReader interface, to facilitate read data from

[jira] [Created] (HDFS-10839) Add new BlockReaderl#readAll(ByteBuffer buf, int len) API

2016-09-04 Thread SammiChen (JIRA)
SammiChen created HDFS-10839: Summary: Add new BlockReaderl#readAll(ByteBuffer buf, int len) API Key: HDFS-10839 URL: https://issues.apache.org/jira/browse/HDFS-10839 Project: Hadoop HDFS Issue

[jira] [Commented] (HDFS-8901) Use ByteBuffer in striping positional read

2016-09-02 Thread SammiChen (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8901?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15458196#comment-15458196 ] SammiChen commented on HDFS-8901: - 1. {{TestPread}} failure is fixed. 2. {quote} Looking back at the

[jira] [Commented] (HDFS-8901) Use ByteBuffer in striping positional read

2016-09-08 Thread SammiChen (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8901?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15475782#comment-15475782 ] SammiChen commented on HDFS-8901: - Thanks Zhe for contribute so much effort to review the patch and give

[jira] [Commented] (HDFS-8957) Consolidate client striping input stream codes for stateful read and positional read

2016-09-13 Thread SammiChen (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8957?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15489137#comment-15489137 ] SammiChen commented on HDFS-8957: - For easy review, I created " HDFS-10861 Refactor StatefulStripeReader

[jira] [Updated] (HDFS-10861) Refactor StatefulStripeReader and PositionStripeReader, use ECChunk version decode API

2016-09-14 Thread SammiChen (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10861?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] SammiChen updated HDFS-10861: - Attachment: HDFS-10861-v2.patch Fixed 3 checkstyle issues. Other 2 checkstyle issues are not necessary.

[jira] [Updated] (HDFS-10861) Refactor StatefulStripeReader and PositionStripeReader, use ECChunk version decode API

2016-09-13 Thread SammiChen (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10861?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] SammiChen updated HDFS-10861: - Attachment: HDFS-10861-v1.patch > Refactor StatefulStripeReader and PositionStripeReader, use ECChunk

[jira] [Updated] (HDFS-10861) Refactor StatefulStripeReader and PositionStripeReader, use ECChunk version decode API

2016-09-13 Thread SammiChen (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10861?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] SammiChen updated HDFS-10861: - Status: Patch Available (was: Open) > Refactor StatefulStripeReader and PositionStripeReader, use

[jira] [Created] (HDFS-10861) Refactor StatefulStripeReader and PositionStripeReader, use ECChunk version decode API

2016-09-13 Thread SammiChen (JIRA)
SammiChen created HDFS-10861: Summary: Refactor StatefulStripeReader and PositionStripeReader, use ECChunk version decode API Key: HDFS-10861 URL: https://issues.apache.org/jira/browse/HDFS-10861

[jira] [Updated] (HDFS-10895) Update HDFS Erasure Coding document about how to enable ISA-L library for erasure code codec

2016-10-09 Thread SammiChen (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10895?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] SammiChen updated HDFS-10895: - Attachment: HDFS-10895-v4.patch remove whitespaces > Update HDFS Erasure Coding document about how to

[jira] [Updated] (HDFS-10895) Update HDFS Erasure Coding document about how to enable ISA-L library for erasure code codec

2016-10-08 Thread SammiChen (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10895?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] SammiChen updated HDFS-10895: - Attachment: HDFS-10895-v2.patch Rebase patch against latest trunk doc > Update HDFS Erasure Coding

[jira] [Updated] (HDFS-10895) Update HDFS Erasure Coding document about how to enable ISA-L library for erasure code codec

2016-10-09 Thread SammiChen (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10895?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] SammiChen updated HDFS-10895: - Attachment: HDFS-10895-v3.patch Improve the document by leverage "Build instructions for

[jira] [Updated] (HDFS-10861) Refactor StripeReaders and use ECChunk version decode API

2016-09-19 Thread SammiChen (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10861?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] SammiChen updated HDFS-10861: - Attachment: HDFS-10861-v4.patch After discussion with Kai, I re-factored the patch. Here is the

[jira] [Updated] (HDFS-10861) Refactor StripeReaders and use ECChunk version decode API

2016-09-18 Thread SammiChen (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10861?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] SammiChen updated HDFS-10861: - Attachment: HDFS-10861-v3.patch extract StripeReader from DfsStripedInputStream class > Refactor

[jira] [Updated] (HDFS-10861) Refactor StripeReaders and use ECChunk version decode API

2016-09-19 Thread SammiChen (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10861?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] SammiChen updated HDFS-10861: - Attachment: (was: HDFS-10861-v6.patch) > Refactor StripeReaders and use ECChunk version decode API >

[jira] [Updated] (HDFS-10861) Refactor StripeReaders and use ECChunk version decode API

2016-09-19 Thread SammiChen (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10861?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] SammiChen updated HDFS-10861: - Attachment: HDFS-10861-v6.patch The broken link in {{StripedBlockUtil}} is also fixed. > Refactor

[jira] [Updated] (HDFS-10861) Refactor StripeReaders and use ECChunk version decode API

2016-09-19 Thread SammiChen (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10861?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] SammiChen updated HDFS-10861: - Attachment: HDFS-10861-v6.patch Fix 2 failed test cases > Refactor StripeReaders and use ECChunk version

[jira] [Commented] (HDFS-10861) Refactor StripeReaders and use ECChunk version decode API

2016-09-20 Thread SammiChen (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10861?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15505957#comment-15505957 ] SammiChen commented on HDFS-10861: -- Hadoop QA run the test with the old v5 patch > Refactor

[jira] [Updated] (HDFS-10861) Refactor StripeReaders and use ECChunk version decode API

2016-09-20 Thread SammiChen (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10861?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] SammiChen updated HDFS-10861: - Attachment: HDFS-10861-v6.patch Resubmit patch v6 to trigger Hadoop QA run the test against v6 >

[jira] [Updated] (HDFS-10861) Refactor StripeReaders and use ECChunk version decode API

2016-09-20 Thread SammiChen (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10861?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] SammiChen updated HDFS-10861: - Attachment: (was: HDFS-10861-v6.patch) > Refactor StripeReaders and use ECChunk version decode API >

[jira] [Created] (HDFS-10895) Update HDFS Erasure Coding document about how to enable ISA-L library for erasure code codec

2016-09-23 Thread SammiChen (JIRA)
SammiChen created HDFS-10895: Summary: Update HDFS Erasure Coding document about how to enable ISA-L library for erasure code codec Key: HDFS-10895 URL: https://issues.apache.org/jira/browse/HDFS-10895

[jira] [Commented] (HDFS-10861) Refactor StripeReaders and use ECChunk version decode API

2016-09-20 Thread SammiChen (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10861?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15508390#comment-15508390 ] SammiChen commented on HDFS-10861: -- Thanks Kai for take time review the patch and commit it! > Refactor

[jira] [Updated] (HDFS-10861) Refactor StripeReaders and use ECChunk version decode API

2016-09-19 Thread SammiChen (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10861?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] SammiChen updated HDFS-10861: - Attachment: HDFS-10861-v5.patch Update the patch, 1. fix 1 code style 2. refine code, import protected

[jira] [Commented] (HDFS-8901) Use ByteBuffer in striping positional read

2016-08-26 Thread SammiChen (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8901?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15438885#comment-15438885 ] SammiChen commented on HDFS-8901: - For question 4, why flip is not needed before? The reason is

[jira] [Updated] (HDFS-8901) Use ByteBuffer in striping positional read

2016-08-26 Thread SammiChen (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8901?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] SammiChen updated HDFS-8901: Attachment: HDFS-8901.v15.patch changes based on v14 patch, 1. rebase the patch 2. fix item 1,2,5 from

[jira] [Commented] (HDFS-8905) Refactor DFSInputStream#ReaderStrategy

2016-08-22 Thread SammiChen (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8905?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15432179#comment-15432179 ] SammiChen commented on HDFS-8905: - I run the failed unit cases locally with the latest HDFS trunk + this

[jira] [Updated] (HDFS-8905) Refactor DFSInputStream#ReaderStrategy

2016-08-22 Thread SammiChen (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8905?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] SammiChen updated HDFS-8905: Attachment: HDFS-8905-v14.patch Thanks for Kai's review and suggestion. I remove the unnecessary

[jira] [Updated] (HDFS-8901) Use ByteBuffer in striping positional read

2016-08-28 Thread SammiChen (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8901?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] SammiChen updated HDFS-8901: Attachment: HDFS-8901.v16.patch Fix one style checker reported issue. Another reported issue is a false

[jira] [Commented] (HDFS-8901) Use ByteBuffer in striping positional read

2016-08-29 Thread SammiChen (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8901?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15445461#comment-15445461 ] SammiChen commented on HDFS-8901: - I rerun the unit test TestErasureCodeBenchmarkThroughput locally with

[jira] [Commented] (HDFS-8901) Use ByteBuffer in striping positional read

2016-08-25 Thread SammiChen (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8901?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15438489#comment-15438489 ] SammiChen commented on HDFS-8901: - Hi Zhe, Thanks for your great effort to review the patch. Very good

[jira] [Updated] (HDFS-10926) Erasure coding background recovery work configuration parameters name is stale in ErasureCode doc

2016-09-29 Thread SammiChen (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10926?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] SammiChen updated HDFS-10926: - Description: The Erasure coding background recovery work configuration parameter names are stale in doc.

[jira] [Updated] (HDFS-10926) Erasure coding background recovery work configuration parameters name is stale in ErasureCode doc

2016-09-29 Thread SammiChen (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10926?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] SammiChen updated HDFS-10926: - Description: The Erasure coding background recovery work configuration parameter names are stale in doc.

[jira] [Assigned] (HDFS-10926) Erasure coding background recovery work configuration parameters name is stale in ErasureCode doc

2016-09-29 Thread SammiChen (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10926?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] SammiChen reassigned HDFS-10926: Assignee: SammiChen > Erasure coding background recovery work configuration parameters name is >

[jira] [Created] (HDFS-10926) Erasure coding background recovery work configuration parameters name is stale in ErasureCode doc

2016-09-29 Thread SammiChen (JIRA)
SammiChen created HDFS-10926: Summary: Erasure coding background recovery work configuration parameters name is stale in ErasureCode doc Key: HDFS-10926 URL: https://issues.apache.org/jira/browse/HDFS-10926

[jira] [Commented] (HDFS-10910) HDFS Erasure Coding doc should state its currently supported erasure coding policies

2016-09-28 Thread SammiChen (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10910?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15531781#comment-15531781 ] SammiChen commented on HDFS-10910: -- Thanks Yiqun for update the patch. The v2 patch seems good to me. >

[jira] [Commented] (HDFS-10910) HDFS Erasure Coding doc should state its currently supported erasure coding policies

2016-09-28 Thread SammiChen (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10910?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15528702#comment-15528702 ] SammiChen commented on HDFS-10910: -- It's great to add currently supported erasure coding policies in the

[jira] [Updated] (HDFS-10895) Update HDFS Erasure Coding document about how to enable ISA-L library for erasure code codec

2016-09-28 Thread SammiChen (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10895?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] SammiChen updated HDFS-10895: - Attachment: HDFS-10895-v1.patch Initial patch > Update HDFS Erasure Coding document about how to enable

[jira] [Updated] (HDFS-10895) Update HDFS Erasure Coding document about how to enable ISA-L library for erasure code codec

2016-09-28 Thread SammiChen (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10895?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] SammiChen updated HDFS-10895: - Status: Patch Available (was: Open) > Update HDFS Erasure Coding document about how to enable ISA-L

[jira] [Commented] (HDFS-10910) HDFS Erasure Coding doc should state its currently supported erasure coding policies

2016-09-30 Thread SammiChen (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10910?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15537537#comment-15537537 ] SammiChen commented on HDFS-10910: -- Holiday from Oct. 1 to Oct. 7, please expect no response. Thanks.

[jira] [Updated] (HDFS-10895) Update HDFS Erasure Coding doc to add how to use ISA-L based coder

2016-10-09 Thread SammiChen (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10895?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] SammiChen updated HDFS-10895: - Attachment: HDFS-10895-v5.patch Improve the patch after offline discussion with Kai. > Update HDFS

[jira] [Issue Comment Deleted] (HDFS-10910) HDFS Erasure Coding doc should state its currently supported erasure coding policies

2016-10-09 Thread SammiChen (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10910?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] SammiChen updated HDFS-10910: - Comment: was deleted (was: Holiday from Oct. 1 to Oct. 7, please expect no response. Thanks. ) > HDFS

[jira] [Created] (HDFS-10994) Support "XOR-2-1-64k" policy in "hdfs erasurecode" command

2016-10-10 Thread SammiChen (JIRA)
SammiChen created HDFS-10994: Summary: Support "XOR-2-1-64k" policy in "hdfs erasurecode" command Key: HDFS-10994 URL: https://issues.apache.org/jira/browse/HDFS-10994 Project: Hadoop HDFS Issue

[jira] [Updated] (HDFS-10996) Ability to specify per-file EC policy at create time

2016-11-09 Thread SammiChen (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10996?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] SammiChen updated HDFS-10996: - Attachment: HDFS-10996-v2.patch 1. Fix checkstyle issues 2. Improve code based on Rakesh's suggestions >

[jira] [Commented] (HDFS-10996) Ability to specify per-file EC policy at create time

2016-11-09 Thread SammiChen (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10996?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15653005#comment-15653005 ] SammiChen commented on HDFS-10996: -- Hi Rakesh, thank you very much for helping review the patch and give

  1   2   3   4   5   6   7   >