[jira] [Commented] (HDDS-2169) Avoid buffer copies while submitting client requests in Ratis

2019-09-30 Thread Tsz-wo Sze (Jira)
[ https://issues.apache.org/jira/browse/HDDS-2169?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16941365#comment-16941365 ] Tsz-wo Sze commented on HDDS-2169: -- Please ignore the previous Jenkins build. It was testing an old

[jira] [Updated] (HDDS-2169) Avoid buffer copies while submitting client requests in Ratis

2019-09-30 Thread Tsz-wo Sze (Jira)
[ https://issues.apache.org/jira/browse/HDDS-2169?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tsz-wo Sze updated HDDS-2169: - Attachment: (was: o2169_20190923.patch) > Avoid buffer copies while submitting client requests in

[jira] [Updated] (HDDS-2222) Add a method to update ByteBuffer in PureJavaCrc32/PureJavaCrc32C

2019-10-01 Thread Tsz-wo Sze (Jira)
[ https://issues.apache.org/jira/browse/HDDS-?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tsz-wo Sze updated HDDS-: - Status: Patch Available (was: Open) > Add a method to update ByteBuffer in PureJavaCrc32/PureJavaCrc32C

[jira] [Updated] (HDDS-2222) Add a method to update ByteBuffer in PureJavaCrc32/PureJavaCrc32C

2019-10-01 Thread Tsz-wo Sze (Jira)
[ https://issues.apache.org/jira/browse/HDDS-?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tsz-wo Sze updated HDDS-: - Attachment: (was: o_20191001.patch) > Add a method to update ByteBuffer in

[jira] [Commented] (HDDS-2222) Add a method to update ByteBuffer in PureJavaCrc32/PureJavaCrc32C

2019-10-01 Thread Tsz-wo Sze (Jira)
[ https://issues.apache.org/jira/browse/HDDS-?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16941922#comment-16941922 ] Tsz-wo Sze commented on HDDS-: -- o_20191001.patch: adds PureJavaCrc32ByteBuffer and

[jira] [Updated] (HDDS-2222) Add a method to update ByteBuffer in PureJavaCrc32/PureJavaCrc32C

2019-10-01 Thread Tsz-wo Sze (Jira)
[ https://issues.apache.org/jira/browse/HDDS-?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tsz-wo Sze updated HDDS-: - Attachment: o_20191001.patch > Add a method to update ByteBuffer in PureJavaCrc32/PureJavaCrc32C >

[jira] [Updated] (HDDS-2222) Add a method to update ByteBuffer in PureJavaCrc32/PureJavaCrc32C

2019-10-01 Thread Tsz-wo Sze (Jira)
[ https://issues.apache.org/jira/browse/HDDS-?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tsz-wo Sze updated HDDS-: - Attachment: (was: o_20191001.patch) > Add a method to update ByteBuffer in

[jira] [Updated] (HDDS-2222) Add a method to update ByteBuffer in PureJavaCrc32/PureJavaCrc32C

2019-10-01 Thread Tsz-wo Sze (Jira)
[ https://issues.apache.org/jira/browse/HDDS-?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tsz-wo Sze updated HDDS-: - Attachment: o_20191001.patch > Add a method to update ByteBuffer in PureJavaCrc32/PureJavaCrc32C >

[jira] [Updated] (HDDS-2222) Add a method to update ByteBuffer in PureJavaCrc32/PureJavaCrc32C

2019-10-01 Thread Tsz-wo Sze (Jira)
[ https://issues.apache.org/jira/browse/HDDS-?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tsz-wo Sze updated HDDS-: - Attachment: (was: o_20191001.patch) > Add a method to update ByteBuffer in

[jira] [Updated] (HDDS-2222) Add a method to update ByteBuffer in PureJavaCrc32/PureJavaCrc32C

2019-10-01 Thread Tsz-wo Sze (Jira)
[ https://issues.apache.org/jira/browse/HDDS-?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tsz-wo Sze updated HDDS-: - Attachment: o_20191001.patch > Add a method to update ByteBuffer in PureJavaCrc32/PureJavaCrc32C >

[jira] [Updated] (HDDS-2222) Add a method to update ByteBuffer in PureJavaCrc32/PureJavaCrc32C

2019-10-01 Thread Tsz-wo Sze (Jira)
[ https://issues.apache.org/jira/browse/HDDS-?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tsz-wo Sze updated HDDS-: - Attachment: (was: o_20191001.patch) > Add a method to update ByteBuffer in

[jira] [Commented] (HDDS-2204) Avoid buffer coping in checksum verification

2019-10-01 Thread Tsz-wo Sze (Jira)
[ https://issues.apache.org/jira/browse/HDDS-2204?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16942055#comment-16942055 ] Tsz-wo Sze commented on HDDS-2204: -- o2204_20191001.patch: moved some code to HDDS-. > Avoid buffer

[jira] [Updated] (HDDS-2222) Add a method to update ByteBuffer in PureJavaCrc32/PureJavaCrc32C

2019-10-01 Thread Tsz-wo Sze (Jira)
[ https://issues.apache.org/jira/browse/HDDS-?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tsz-wo Sze updated HDDS-: - Attachment: o_20191001.patch > Add a method to update ByteBuffer in PureJavaCrc32/PureJavaCrc32C >

[jira] [Created] (HDDS-2222) Add a method to update ByteBuffer in PureJavaCrc32/PureJavaCrc32C

2019-10-01 Thread Tsz-wo Sze (Jira)
Tsz-wo Sze created HDDS-: Summary: Add a method to update ByteBuffer in PureJavaCrc32/PureJavaCrc32C Key: HDDS- URL: https://issues.apache.org/jira/browse/HDDS- Project: Hadoop Distributed

[jira] [Updated] (HDDS-2222) Add a method to update ByteBuffer in PureJavaCrc32/PureJavaCrc32C

2019-10-01 Thread Tsz-wo Sze (Jira)
[ https://issues.apache.org/jira/browse/HDDS-?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tsz-wo Sze updated HDDS-: - Attachment: (was: o_20191001.patch) > Add a method to update ByteBuffer in

[jira] [Updated] (HDDS-2222) Add a method to update ByteBuffer in PureJavaCrc32/PureJavaCrc32C

2019-10-01 Thread Tsz-wo Sze (Jira)
[ https://issues.apache.org/jira/browse/HDDS-?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tsz-wo Sze updated HDDS-: - Attachment: o_20191001.patch > Add a method to update ByteBuffer in PureJavaCrc32/PureJavaCrc32C >

[jira] [Commented] (HDDS-2204) Avoid buffer coping in checksum verification

2019-10-01 Thread Tsz-wo Sze (Jira)
[ https://issues.apache.org/jira/browse/HDDS-2204?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16942017#comment-16942017 ] Tsz-wo Sze commented on HDDS-2204: -- Since the patch is quite big, the new ChecksumByteBuffer interface

[jira] [Updated] (HDDS-2222) Add a method to update ByteBuffer in PureJavaCrc32/PureJavaCrc32C

2019-10-01 Thread Tsz-wo Sze (Jira)
[ https://issues.apache.org/jira/browse/HDDS-?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tsz-wo Sze updated HDDS-: - Attachment: o_20191001.patch > Add a method to update ByteBuffer in PureJavaCrc32/PureJavaCrc32C >

[jira] [Updated] (HDDS-2204) Avoid buffer coping in checksum verification

2019-10-01 Thread Tsz-wo Sze (Jira)
[ https://issues.apache.org/jira/browse/HDDS-2204?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tsz-wo Sze updated HDDS-2204: - Status: Open (was: Patch Available) > Avoid buffer coping in checksum verification >

[jira] [Updated] (HDDS-2204) Avoid buffer coping in checksum verification

2019-10-01 Thread Tsz-wo Sze (Jira)
[ https://issues.apache.org/jira/browse/HDDS-2204?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tsz-wo Sze updated HDDS-2204: - Attachment: o2204_20191001.patch > Avoid buffer coping in checksum verification >

[jira] [Commented] (HDDS-2169) Avoid buffer copies while submitting client requests in Ratis

2019-09-26 Thread Tsz-wo Sze (Jira)
[ https://issues.apache.org/jira/browse/HDDS-2169?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16939098#comment-16939098 ] Tsz-wo Sze commented on HDDS-2169: -- The failure does not seem related to the patch; see

[jira] [Updated] (HDDS-2222) Add a method to update ByteBuffer in PureJavaCrc32/PureJavaCrc32C

2019-10-01 Thread Tsz-wo Sze (Jira)
[ https://issues.apache.org/jira/browse/HDDS-?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tsz-wo Sze updated HDDS-: - Attachment: o_20191002.patch > Add a method to update ByteBuffer in PureJavaCrc32/PureJavaCrc32C >

[jira] [Commented] (HDDS-2222) Add a method to update ByteBuffer in PureJavaCrc32/PureJavaCrc32C

2019-10-01 Thread Tsz-wo Sze (Jira)
[ https://issues.apache.org/jira/browse/HDDS-?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16942450#comment-16942450 ] Tsz-wo Sze commented on HDDS-: -- o_20191002.patch: fixes checkstyle warnings. The warnings in

[jira] [Created] (HDDS-2204) Avoid buffer coping in checksum verification

2019-09-29 Thread Tsz-wo Sze (Jira)
Tsz-wo Sze created HDDS-2204: Summary: Avoid buffer coping in checksum verification Key: HDDS-2204 URL: https://issues.apache.org/jira/browse/HDDS-2204 Project: Hadoop Distributed Data Store

[jira] [Updated] (HDDS-2204) Avoid buffer coping in checksum verification

2019-09-29 Thread Tsz-wo Sze (Jira)
[ https://issues.apache.org/jira/browse/HDDS-2204?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tsz-wo Sze updated HDDS-2204: - Status: Patch Available (was: Open) > Avoid buffer coping in checksum verification >

[jira] [Updated] (HDDS-2204) Avoid buffer coping in checksum verification

2019-09-29 Thread Tsz-wo Sze (Jira)
[ https://issues.apache.org/jira/browse/HDDS-2204?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tsz-wo Sze updated HDDS-2204: - Attachment: o2204_20190930.patch > Avoid buffer coping in checksum verification >

[jira] [Updated] (HDDS-2204) Avoid buffer coping in checksum verification

2019-09-30 Thread Tsz-wo Sze (Jira)
[ https://issues.apache.org/jira/browse/HDDS-2204?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tsz-wo Sze updated HDDS-2204: - Attachment: o2204_20190930b.patch > Avoid buffer coping in checksum verification >

[jira] [Commented] (HDDS-2204) Avoid buffer coping in checksum verification

2019-09-30 Thread Tsz-wo Sze (Jira)
[ https://issues.apache.org/jira/browse/HDDS-2204?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16940756#comment-16940756 ] Tsz-wo Sze commented on HDDS-2204: -- o2204_20190930b.patch: compute checksum from a ByteBuffer. Will add

[jira] [Updated] (HDDS-2222) Add a method to update ByteBuffer in PureJavaCrc32/PureJavaCrc32C

2019-10-04 Thread Tsz-wo Sze (Jira)
[ https://issues.apache.org/jira/browse/HDDS-?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tsz-wo Sze updated HDDS-: - Fix Version/s: 0.4.1 Resolution: Fixed Status: Resolved (was: Patch Available) The

[jira] [Commented] (HDFS-14529) NPE while Loading the Editlogs

2019-11-04 Thread Tsz-wo Sze (Jira)
[ https://issues.apache.org/jira/browse/HDFS-14529?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16967079#comment-16967079 ] Tsz-wo Sze commented on HDFS-14529: --- Is there a way to reproduce the NPE? Or, could you describe how

[jira] [Commented] (HDFS-14529) NPE while Loading the Editlogs

2019-11-04 Thread Tsz-wo Sze (Jira)
[ https://issues.apache.org/jira/browse/HDFS-14529?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16967072#comment-16967072 ] Tsz-wo Sze commented on HDFS-14529: --- [~Harsha1206], I am wonder if this is snapshot related. Were you

[jira] [Comment Edited] (HDFS-14529) NPE while Loading the Editlogs

2019-11-04 Thread Tsz-wo Sze (Jira)
[ https://issues.apache.org/jira/browse/HDFS-14529?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16967079#comment-16967079 ] Tsz-wo Sze edited comment on HDFS-14529 at 11/4/19 11:03 PM: - Is there a way

[jira] [Commented] (HDDS-2274) Avoid buffer copying in Codec

2019-11-08 Thread Tsz-wo Sze (Jira)
[ https://issues.apache.org/jira/browse/HDDS-2274?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16970495#comment-16970495 ] Tsz-wo Sze commented on HDDS-2274: -- You are right. The improvement may not be possible since RocksDB API

[jira] [Issue Comment Deleted] (HDDS-2392) Fix TestScmSafeMode#testSCMSafeModeRestrictedOp

2019-11-08 Thread Tsz-wo Sze (Jira)
[ https://issues.apache.org/jira/browse/HDDS-2392?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tsz-wo Sze updated HDDS-2392: - Comment: was deleted (was: [~avijayan], thanks for working on this. - In

[jira] [Commented] (HDDS-2392) Fix TestScmSafeMode#testSCMSafeModeRestrictedOp

2019-11-08 Thread Tsz-wo Sze (Jira)
[ https://issues.apache.org/jira/browse/HDDS-2392?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16970445#comment-16970445 ] Tsz-wo Sze commented on HDDS-2392: -- [~avijayan], thanks for working on this. - In

[jira] [Commented] (HDDS-2372) Datanode pipeline is failing with NoSuchFileException

2019-11-05 Thread Tsz-wo Sze (Jira)
[ https://issues.apache.org/jira/browse/HDDS-2372?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16967873#comment-16967873 ] Tsz-wo Sze commented on HDDS-2372: -- It makes sense to check the chunk file again after temporary chunk

[jira] [Updated] (HDDS-2375) Refactor BlockOutputStream to allow flexible buffering

2019-10-31 Thread Tsz-wo Sze (Jira)
[ https://issues.apache.org/jira/browse/HDDS-2375?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tsz-wo Sze updated HDDS-2375: - Description: In HDDS-2331, we found that Ozone client allocates a ByteBuffer with chunk size (e.g. 16MB

[jira] [Updated] (HDDS-2386) Implement incremental ChunkBuffer

2019-10-31 Thread Tsz-wo Sze (Jira)
[ https://issues.apache.org/jira/browse/HDDS-2386?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tsz-wo Sze updated HDDS-2386: - Attachment: o2386_20191031b.patch > Implement incremental ChunkBuffer > -

[jira] [Commented] (HDDS-2386) Implement incremental ChunkBuffer

2019-10-31 Thread Tsz-wo Sze (Jira)
[ https://issues.apache.org/jira/browse/HDDS-2386?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16964298#comment-16964298 ] Tsz-wo Sze commented on HDDS-2386: -- o2386_20191031b.patch: fixes test failures. > Implement incremental

[jira] [Commented] (HDDS-2386) Implement incremental ChunkBuffer

2019-10-30 Thread Tsz-wo Sze (Jira)
[ https://issues.apache.org/jira/browse/HDDS-2386?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16963460#comment-16963460 ] Tsz-wo Sze commented on HDDS-2386: -- o2386_20191030.patch: requires HDDS-2375. Also, need to fix test

[jira] [Updated] (HDDS-2386) Implement incremental ChunkBuffer

2019-10-30 Thread Tsz-wo Sze (Jira)
[ https://issues.apache.org/jira/browse/HDDS-2386?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tsz-wo Sze updated HDDS-2386: - Attachment: o2386_20191030.patch > Implement incremental ChunkBuffer > -

[jira] [Commented] (HDDS-2331) Client OOME due to buffer retention

2019-10-30 Thread Tsz-wo Sze (Jira)
[ https://issues.apache.org/jira/browse/HDDS-2331?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16963458#comment-16963458 ] Tsz-wo Sze commented on HDDS-2331: -- Filed HDDS-2386 for incremental buffering. > Shall we resolve it,

[jira] [Created] (HDDS-2386) Implement incremental ChunkBuffer

2019-10-30 Thread Tsz-wo Sze (Jira)
Tsz-wo Sze created HDDS-2386: Summary: Implement incremental ChunkBuffer Key: HDDS-2386 URL: https://issues.apache.org/jira/browse/HDDS-2386 Project: Hadoop Distributed Data Store Issue Type:

[jira] [Commented] (HDDS-2386) Implement incremental ChunkBuffer

2019-11-16 Thread Tsz-wo Sze (Jira)
[ https://issues.apache.org/jira/browse/HDDS-2386?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16975862#comment-16975862 ] Tsz-wo Sze commented on HDDS-2386: -- Created https://github.com/apache/hadoop-ozone/pull/205 > Implement

[jira] [Commented] (HDDS-2375) Refactor BlockOutputStream to allow flexible buffering

2019-11-16 Thread Tsz-wo Sze (Jira)
[ https://issues.apache.org/jira/browse/HDDS-2375?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16975861#comment-16975861 ] Tsz-wo Sze commented on HDDS-2375: -- [~xyao], thanks a lot for reviewing and merging the pull-request. >

[jira] [Updated] (HDDS-2523) BufferPool.releaseBuffer may release a buffer different than the head of the list

2019-11-17 Thread Tsz-wo Sze (Jira)
[ https://issues.apache.org/jira/browse/HDDS-2523?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tsz-wo Sze updated HDDS-2523: - Description: {code} //BufferPool public void releaseBuffer(ByteBuffer byteBuffer) { // always

[jira] [Updated] (HDDS-2523) BufferPool.releaseBuffer may release a buffer different than the head of the list

2019-11-17 Thread Tsz-wo Sze (Jira)
[ https://issues.apache.org/jira/browse/HDDS-2523?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tsz-wo Sze updated HDDS-2523: - Description: {code} //BufferPool public void releaseBuffer(ByteBuffer byteBuffer) { // always

[jira] [Updated] (HDDS-2523) BufferPool.releaseBuffer may release a buffer different than the head of the list

2019-11-17 Thread Tsz-wo Sze (Jira)
[ https://issues.apache.org/jira/browse/HDDS-2523?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tsz-wo Sze updated HDDS-2523: - Description: {code} public void releaseBuffer(ByteBuffer byteBuffer) { // always remove from head

[jira] [Updated] (HDDS-2523) BufferPool.releaseBuffer may release a buffer different than the head of the list

2019-11-17 Thread Tsz-wo Sze (Jira)
[ https://issues.apache.org/jira/browse/HDDS-2523?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tsz-wo Sze updated HDDS-2523: - Description: {code} //BufferPool public void releaseBuffer(ByteBuffer byteBuffer) { // always

[jira] [Commented] (HDDS-2523) BufferPool.releaseBuffer may release a buffer different than the head of the list

2019-11-17 Thread Tsz-wo Sze (Jira)
[ https://issues.apache.org/jira/browse/HDDS-2523?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16976206#comment-16976206 ] Tsz-wo Sze commented on HDDS-2523: -- If we change the precondition to check == as below,

[jira] [Created] (HDDS-2523) BufferPool.releaseBuffer may release a buffer different than the head of the list

2019-11-17 Thread Tsz-wo Sze (Jira)
Tsz-wo Sze created HDDS-2523: Summary: BufferPool.releaseBuffer may release a buffer different than the head of the list Key: HDDS-2523 URL: https://issues.apache.org/jira/browse/HDDS-2523 Project:

[jira] [Commented] (HDDS-2523) BufferPool.releaseBuffer may release a buffer different than the head of the list

2019-11-17 Thread Tsz-wo Sze (Jira)
[ https://issues.apache.org/jira/browse/HDDS-2523?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16976229#comment-16976229 ] Tsz-wo Sze commented on HDDS-2523: -- I have checked the code before HDDS-2375. It has the same problem.

[jira] [Commented] (HDFS-14529) NPE while Loading the Editlogs

2019-11-07 Thread Tsz-wo Sze (Jira)
[ https://issues.apache.org/jira/browse/HDFS-14529?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16969659#comment-16969659 ] Tsz-wo Sze commented on HDFS-14529: --- Not sure if HDFS-14807 could fix the NPE. > NPE while Loading the

[jira] [Updated] (HDDS-2273) Avoid buffer copying in GrpcReplicationService

2019-11-08 Thread Tsz-wo Sze (Jira)
[ https://issues.apache.org/jira/browse/HDDS-2273?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tsz-wo Sze updated HDDS-2273: - Fix Version/s: 0.5.0 Resolution: Fixed Status: Resolved (was: Patch Available) I just

[jira] [Updated] (HDDS-2523) BufferPool.releaseBuffer may release a buffer different than the head of the list

2019-11-18 Thread Tsz-wo Sze (Jira)
[ https://issues.apache.org/jira/browse/HDDS-2523?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tsz-wo Sze updated HDDS-2523: - Attachment: a.patch > BufferPool.releaseBuffer may release a buffer different than the head of the >

[jira] [Commented] (HDDS-2523) BufferPool.releaseBuffer may release a buffer different than the head of the list

2019-11-18 Thread Tsz-wo Sze (Jira)
[ https://issues.apache.org/jira/browse/HDDS-2523?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16976756#comment-16976756 ] Tsz-wo Sze commented on HDDS-2523: -- [~adoroszlai], thanks for working on this. I could post some debug I

[jira] [Updated] (HDDS-2523) BufferPool.releaseBuffer may release a buffer different than the head of the list

2019-11-18 Thread Tsz-wo Sze (Jira)
[ https://issues.apache.org/jira/browse/HDDS-2523?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tsz-wo Sze updated HDDS-2523: - Component/s: Ozone Client > BufferPool.releaseBuffer may release a buffer different than the head of the

[jira] [Commented] (HDDS-2523) BufferPool.releaseBuffer may release a buffer different than the head of the list

2019-11-18 Thread Tsz-wo Sze (Jira)
[ https://issues.apache.org/jira/browse/HDDS-2523?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16976771#comment-16976771 ] Tsz-wo Sze commented on HDDS-2523: -- a.patch: for debug It expected ChunkBufferImplWithByteBuffer3 but

[jira] [Commented] (HDFS-15012) NN fails to parse Edit logs after applying HDFS-13101

2019-12-09 Thread Tsz-wo Sze (Jira)
[ https://issues.apache.org/jira/browse/HDFS-15012?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16991695#comment-16991695 ] Tsz-wo Sze commented on HDFS-15012: --- +1 the 000 patch looks good. > NN fails to parse Edit logs after

[jira] [Commented] (HDDS-2331) Client OOME due to buffer retention

2019-10-21 Thread Tsz-wo Sze (Jira)
[ https://issues.apache.org/jira/browse/HDDS-2331?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16955983#comment-16955983 ] Tsz-wo Sze commented on HDDS-2331: -- Chunk size and array size may not be necessarily the same. Chunk

[jira] [Updated] (HDDS-2275) In BatchOperation.SingleOperation, do not clone byte[]

2019-10-15 Thread Tsz-wo Sze (Jira)
[ https://issues.apache.org/jira/browse/HDDS-2275?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tsz-wo Sze updated HDDS-2275: - Status: Patch Available (was: Open) o2275_20191015.patch: removes clone() > In

[jira] [Updated] (HDDS-2275) In BatchOperation.SingleOperation, do not clone byte[]

2019-10-15 Thread Tsz-wo Sze (Jira)
[ https://issues.apache.org/jira/browse/HDDS-2275?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tsz-wo Sze updated HDDS-2275: - Attachment: o2275_20191015.patch > In BatchOperation.SingleOperation, do not clone byte[] >

[jira] [Updated] (HDDS-2275) In BatchOperation.SingleOperation, do not clone byte[]

2019-10-15 Thread Tsz-wo Sze (Jira)
[ https://issues.apache.org/jira/browse/HDDS-2275?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tsz-wo Sze updated HDDS-2275: - Attachment: (was: o2275_20191015.patch) > In BatchOperation.SingleOperation, do not clone byte[] >

[jira] [Updated] (HDDS-2275) In BatchOperation.SingleOperation, do not clone byte[]

2019-10-15 Thread Tsz-wo Sze (Jira)
[ https://issues.apache.org/jira/browse/HDDS-2275?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tsz-wo Sze updated HDDS-2275: - Attachment: o2275_20191015.patch > In BatchOperation.SingleOperation, do not clone byte[] >

[jira] [Updated] (HDDS-2275) In BatchOperation.SingleOperation, do not clone byte[]

2019-10-15 Thread Tsz-wo Sze (Jira)
[ https://issues.apache.org/jira/browse/HDDS-2275?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tsz-wo Sze updated HDDS-2275: - Attachment: o2275_20191015.patch > In BatchOperation.SingleOperation, do not clone byte[] >

[jira] [Updated] (HDDS-2271) Avoid buffer copying in KeyValueHandler

2019-10-15 Thread Tsz-wo Sze (Jira)
[ https://issues.apache.org/jira/browse/HDDS-2271?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tsz-wo Sze updated HDDS-2271: - Attachment: o2271_20191015.patch > Avoid buffer copying in KeyValueHandler >

[jira] [Commented] (HDDS-2271) Avoid buffer copying in KeyValueHandler

2019-10-15 Thread Tsz-wo Sze (Jira)
[ https://issues.apache.org/jira/browse/HDDS-2271?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16951774#comment-16951774 ] Tsz-wo Sze commented on HDDS-2271: -- o2271_20191015.patch: from pull request #1625 > Avoid buffer copying

[jira] [Updated] (HDDS-2271) Avoid buffer copying in KeyValueHandler

2019-10-15 Thread Tsz-wo Sze (Jira)
[ https://issues.apache.org/jira/browse/HDDS-2271?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tsz-wo Sze updated HDDS-2271: - Status: Patch Available (was: Open) > Avoid buffer copying in KeyValueHandler >

[jira] [Updated] (HDDS-2275) In BatchOperation.SingleOperation, do not clone byte[]

2019-10-15 Thread Tsz-wo Sze (Jira)
[ https://issues.apache.org/jira/browse/HDDS-2275?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tsz-wo Sze updated HDDS-2275: - Attachment: (was: o2275_20191015.patch) > In BatchOperation.SingleOperation, do not clone byte[] >

[jira] [Updated] (HDDS-2275) In BatchOperation.SingleOperation, do not clone byte[]

2019-10-15 Thread Tsz-wo Sze (Jira)
[ https://issues.apache.org/jira/browse/HDDS-2275?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tsz-wo Sze updated HDDS-2275: - Attachment: (was: o2275_20191015.patch) > In BatchOperation.SingleOperation, do not clone byte[] >

[jira] [Updated] (HDDS-2275) In BatchOperation.SingleOperation, do not clone byte[]

2019-10-15 Thread Tsz-wo Sze (Jira)
[ https://issues.apache.org/jira/browse/HDDS-2275?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tsz-wo Sze updated HDDS-2275: - Attachment: o2275_20191015.patch > In BatchOperation.SingleOperation, do not clone byte[] >

[jira] [Created] (HDDS-2375) Refactor BlockOutputStream to allow flexible buffering

2019-10-28 Thread Tsz-wo Sze (Jira)
Tsz-wo Sze created HDDS-2375: Summary: Refactor BlockOutputStream to allow flexible buffering Key: HDDS-2375 URL: https://issues.apache.org/jira/browse/HDDS-2375 Project: Hadoop Distributed Data Store

[jira] [Commented] (HDFS-13671) Namenode deletes large dir slowly caused by FoldedTreeSet#removeAndGet

2019-10-28 Thread Tsz-wo Sze (Jira)
[ https://issues.apache.org/jira/browse/HDFS-13671?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16961507#comment-16961507 ] Tsz-wo Sze commented on HDFS-13671: --- There may be some ways to improve the performance of

[jira] [Commented] (HDDS-2331) Client OOME due to buffer retention

2019-10-28 Thread Tsz-wo Sze (Jira)
[ https://issues.apache.org/jira/browse/HDDS-2331?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16961613#comment-16961613 ] Tsz-wo Sze commented on HDDS-2331: -- Created HDDS-2375 to refactor the code. Will implement chunk buffer

[jira] [Commented] (HDDS-2372) Datanode pipeline is failing with NoSuchFileException

2019-10-29 Thread Tsz-wo Sze (Jira)
[ https://issues.apache.org/jira/browse/HDDS-2372?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16962112#comment-16962112 ] Tsz-wo Sze commented on HDDS-2372: -- Some questions (sorry that I don't understand the test): - Did the

[jira] [Commented] (HDDS-2331) Client OOME due to buffer retention

2019-10-18 Thread Tsz-wo Sze (Jira)
[ https://issues.apache.org/jira/browse/HDDS-2331?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16955011#comment-16955011 ] Tsz-wo Sze commented on HDDS-2331: -- >From

[jira] [Commented] (HDDS-2331) Client OOME due to buffer retention

2019-10-18 Thread Tsz-wo Sze (Jira)
[ https://issues.apache.org/jira/browse/HDDS-2331?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16955013#comment-16955013 ] Tsz-wo Sze commented on HDDS-2331: -- [~adoroszlai], thanks for discovering the problem. The finding

[jira] [Updated] (HDDS-2271) Avoid buffer copying in KeyValueHandler

2019-10-16 Thread Tsz-wo Sze (Jira)
[ https://issues.apache.org/jira/browse/HDDS-2271?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tsz-wo Sze updated HDDS-2271: - Fix Version/s: 0.6.0 Resolution: Fixed Status: Resolved (was: Patch Available) I have

[jira] [Commented] (HDDS-2271) Avoid buffer copying in KeyValueHandler

2019-10-16 Thread Tsz-wo Sze (Jira)
[ https://issues.apache.org/jira/browse/HDDS-2271?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16953378#comment-16953378 ] Tsz-wo Sze commented on HDDS-2271: -- Thanks [~shashikant] and [~hanishakoneru] for reviewing this. >

[jira] [Commented] (HDDS-2275) In BatchOperation.SingleOperation, do not clone byte[]

2019-10-17 Thread Tsz-wo Sze (Jira)
[ https://issues.apache.org/jira/browse/HDDS-2275?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16953544#comment-16953544 ] Tsz-wo Sze commented on HDDS-2275: -- Thanks [~hanishakoneru] for reviewing the patch. I have committed

[jira] [Updated] (HDDS-2275) In BatchOperation.SingleOperation, do not clone byte[]

2019-10-17 Thread Tsz-wo Sze (Jira)
[ https://issues.apache.org/jira/browse/HDDS-2275?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tsz-wo Sze updated HDDS-2275: - Fix Version/s: 0.6.0 Resolution: Fixed Status: Resolved (was: Patch Available) > In

[jira] [Created] (HDDS-2273) Avoid buffer copying in GrpcReplicationService

2019-10-09 Thread Tsz-wo Sze (Jira)
Tsz-wo Sze created HDDS-2273: Summary: Avoid buffer copying in GrpcReplicationService Key: HDDS-2273 URL: https://issues.apache.org/jira/browse/HDDS-2273 Project: Hadoop Distributed Data Store

[jira] [Created] (HDDS-2274) Avoid buffer copying in Codec

2019-10-09 Thread Tsz-wo Sze (Jira)
Tsz-wo Sze created HDDS-2274: Summary: Avoid buffer copying in Codec Key: HDDS-2274 URL: https://issues.apache.org/jira/browse/HDDS-2274 Project: Hadoop Distributed Data Store Issue Type:

[jira] [Created] (HDDS-2271) Avoid buffer copying in KeyValueHandler

2019-10-09 Thread Tsz-wo Sze (Jira)
Tsz-wo Sze created HDDS-2271: Summary: Avoid buffer copying in KeyValueHandler Key: HDDS-2271 URL: https://issues.apache.org/jira/browse/HDDS-2271 Project: Hadoop Distributed Data Store Issue

[jira] [Created] (HDDS-2270) Avoid buffer copying in ContainerStateMachine.loadSnapshot/persistContainerSet

2019-10-08 Thread Tsz-wo Sze (Jira)
Tsz-wo Sze created HDDS-2270: Summary: Avoid buffer copying in ContainerStateMachine.loadSnapshot/persistContainerSet Key: HDDS-2270 URL: https://issues.apache.org/jira/browse/HDDS-2270 Project: Hadoop

[jira] [Created] (HDDS-2272) Avoid buffer copying in GrpcReplicationClient

2019-10-09 Thread Tsz-wo Sze (Jira)
Tsz-wo Sze created HDDS-2272: Summary: Avoid buffer copying in GrpcReplicationClient Key: HDDS-2272 URL: https://issues.apache.org/jira/browse/HDDS-2272 Project: Hadoop Distributed Data Store

[jira] [Commented] (HDDS-2271) Avoid buffer copying in KeyValueHandler

2019-10-09 Thread Tsz-wo Sze (Jira)
[ https://issues.apache.org/jira/browse/HDDS-2271?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16947443#comment-16947443 ] Tsz-wo Sze commented on HDDS-2271: -- > In handleGetSmallFile, it first reads chunk data to a byte[] and

[jira] [Created] (HDDS-2275) In BatchOperation.SingleOperation, do not clone byte[]

2019-10-09 Thread Tsz-wo Sze (Jira)
Tsz-wo Sze created HDDS-2275: Summary: In BatchOperation.SingleOperation, do not clone byte[] Key: HDDS-2275 URL: https://issues.apache.org/jira/browse/HDDS-2275 Project: Hadoop Distributed Data Store

[jira] [Commented] (HDDS-2222) Add a method to update ByteBuffer in PureJavaCrc32/PureJavaCrc32C

2019-10-03 Thread Tsz-wo Sze (Jira)
[ https://issues.apache.org/jira/browse/HDDS-?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16944220#comment-16944220 ] Tsz-wo Sze commented on HDDS-: -- > o_20191001.patch: adds PureJavaCrc32ByteBuffer and >

[jira] [Commented] (HDDS-2257) Fix checkstyle issues in ChecksumByteBuffer

2019-10-04 Thread Tsz-wo Sze (Jira)
[ https://issues.apache.org/jira/browse/HDDS-2257?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16944922#comment-16944922 ] Tsz-wo Sze commented on HDDS-2257: -- [~vivekratnavel], thanks for fixing the checkstyle warnings from

[jira] [Resolved] (HDDS-2222) Add a method to update ByteBuffer in PureJavaCrc32/PureJavaCrc32C

2019-10-04 Thread Tsz-wo Sze (Jira)
[ https://issues.apache.org/jira/browse/HDDS-?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tsz-wo Sze resolved HDDS-. -- Resolution: Fixed Merged Pull Request #1595. > Add a method to update ByteBuffer in

[jira] [Updated] (HDDS-2222) Add a method to update ByteBuffer in PureJavaCrc32/PureJavaCrc32C

2019-10-04 Thread Tsz-wo Sze (Jira)
[ https://issues.apache.org/jira/browse/HDDS-?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tsz-wo Sze updated HDDS-: - Status: Reopened (was: Reopened) > Add a method to update ByteBuffer in PureJavaCrc32/PureJavaCrc32C >

[jira] [Updated] (HDDS-2222) Add a method to update ByteBuffer in PureJavaCrc32/PureJavaCrc32C

2019-10-04 Thread Tsz-wo Sze (Jira)
[ https://issues.apache.org/jira/browse/HDDS-?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tsz-wo Sze updated HDDS-: - Status: Reopened (was: Reopened) > Add a method to update ByteBuffer in PureJavaCrc32/PureJavaCrc32C >

[jira] [Reopened] (HDDS-2222) Add a method to update ByteBuffer in PureJavaCrc32/PureJavaCrc32C

2019-10-04 Thread Tsz-wo Sze (Jira)
[ https://issues.apache.org/jira/browse/HDDS-?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tsz-wo Sze reopened HDDS-: -- Reopen to change findbugsExcludeFile.xml > Add a method to update ByteBuffer in

[jira] [Commented] (HDFS-15313) Ensure inodes in active filesytem are not deleted during snapshot delete

2020-04-30 Thread Tsz-wo Sze (Jira)
[ https://issues.apache.org/jira/browse/HDFS-15313?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17097056#comment-17097056 ] Tsz-wo Sze commented on HDFS-15313: --- {code} + // TODO: Is it a bug to return true? + //

[jira] [Commented] (HDFS-15319) Fix INode#isInLatestSnapshot() API

2020-05-01 Thread Tsz-wo Sze (Jira)
[ https://issues.apache.org/jira/browse/HDFS-15319?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17097760#comment-17097760 ] Tsz-wo Sze commented on HDFS-15319: --- The test failures seem related. When parent.isReference() ==

[jira] [Resolved] (HDFS-15519) Check inaccessible INodes in FsImageValidation

2020-08-14 Thread Tsz-wo Sze (Jira)
[ https://issues.apache.org/jira/browse/HDFS-15519?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tsz-wo Sze resolved HDFS-15519. --- Fix Version/s: 3.4.0 Hadoop Flags: Reviewed Resolution: Fixed Thanks, [~shashikant] for

[jira] [Updated] (HDFS-15500) In-order deletion of snapshots: Diff lists must be update only in the last snapshot

2020-08-14 Thread Tsz-wo Sze (Jira)
[ https://issues.apache.org/jira/browse/HDFS-15500?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tsz-wo Sze updated HDFS-15500: -- Component/s: snapshots > In-order deletion of snapshots: Diff lists must be update only in the last >

[jira] [Updated] (HDFS-15463) Add a tool to validate FsImage

2020-07-15 Thread Tsz-wo Sze (Jira)
[ https://issues.apache.org/jira/browse/HDFS-15463?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tsz-wo Sze updated HDFS-15463: -- Attachment: (was: FsImageValidation20200715b.patch) > Add a tool to validate FsImage >

[jira] [Updated] (HDFS-15463) Add a tool to validate FsImage

2020-07-15 Thread Tsz-wo Sze (Jira)
[ https://issues.apache.org/jira/browse/HDFS-15463?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tsz-wo Sze updated HDFS-15463: -- Attachment: FsImageValidation20200715b.patch > Add a tool to validate FsImage >

  1   2   3   >