[jira] [Comment Edited] (HBASE-28390) WAL value compression fails for cells with large values
[
https://issues.apache.org/jira/browse/HBASE-28390?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17819431#comment-17819431
]
Bryan Beaudreault edited comment on HBASE-28390 at 2/22/24 1:05 AM:
I had an idea for how we can workaround this using
WALDecompressionBoundedDelegatingInputStream. Please give it a look when you
have a chance [~apurtell]. I'm also open to other ideas.
https://github.com/apache/hbase/pull/5696
was (Author: bbeaudreault):
I had an idea for how we can workaround this using
WALDecompressionBoundedDelegatingInputStream. Please give it a look when you
have a chance [~apurtell]. I'm also open to other ideas.
> WAL value compression fails for cells with large values
> ---
>
> Key: HBASE-28390
> URL: https://issues.apache.org/jira/browse/HBASE-28390
> Project: HBase
> Issue Type: Bug
>Reporter: Bryan Beaudreault
>Priority: Major
> Labels: pull-request-available
>
> We are testing out WAL compression and noticed that it fails for large values
> when both features (wal compression and wal value compression) are enabled.
> It works fine with either feature independently, but not when combined. It
> seems to fail for all of the value compressor types, and the failure is in
> the LRUDictionary of wal key compression:
>
> {code:java}
> java.io.IOException: Error while reading 2 WAL KVs; started reading at 230
> and read up to 396
> at
> org.apache.hadoop.hbase.regionserver.wal.ProtobufWALStreamReader.next(ProtobufWALStreamReader.java:94)
> ~[classes/:?]
> at
> org.apache.hadoop.hbase.wal.CompressedWALTestBase.doTest(CompressedWALTestBase.java:181)
> ~[test-classes/:?]
> at
> org.apache.hadoop.hbase.wal.CompressedWALTestBase.testForSize(CompressedWALTestBase.java:129)
> ~[test-classes/:?]
> at
> org.apache.hadoop.hbase.wal.CompressedWALTestBase.testLarge(CompressedWALTestBase.java:94)
> ~[test-classes/:?]
> at jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> ~[?:?]
> at
> jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> ~[?:?]
> at
> jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> ~[?:?]
> at java.lang.reflect.Method.invoke(Method.java:566) ~[?:?]
> at
> org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:59)
> ~[junit-4.13.2.jar:4.13.2]
> at
> org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
> ~[junit-4.13.2.jar:4.13.2]
> at
> org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:56)
> ~[junit-4.13.2.jar:4.13.2]
> at
> org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
> ~[junit-4.13.2.jar:4.13.2]
> at org.junit.runners.ParentRunner$3.evaluate(ParentRunner.java:306)
> ~[junit-4.13.2.jar:4.13.2]
> at
> org.junit.runners.BlockJUnit4ClassRunner$1.evaluate(BlockJUnit4ClassRunner.java:100)
> ~[junit-4.13.2.jar:4.13.2]
> at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:366)
> ~[junit-4.13.2.jar:4.13.2]
> at
> org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:103)
> ~[junit-4.13.2.jar:4.13.2]
> at
> org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:63)
> ~[junit-4.13.2.jar:4.13.2]
> at org.junit.runners.ParentRunner$4.run(ParentRunner.java:331)
> ~[junit-4.13.2.jar:4.13.2]
> at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:79)
> ~[junit-4.13.2.jar:4.13.2]
> at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:329)
> ~[junit-4.13.2.jar:4.13.2]
> at org.junit.runners.ParentRunner.access$100(ParentRunner.java:66)
> ~[junit-4.13.2.jar:4.13.2]
> at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:293)
> ~[junit-4.13.2.jar:4.13.2]
> at
> org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26)
> ~[junit-4.13.2.jar:4.13.2]
> at
> org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27)
> ~[junit-4.13.2.jar:4.13.2]
> at
> org.junit.internal.runners.statements.FailOnTimeout$CallableStatement.call(FailOnTimeout.java:299)
> ~[junit-4.13.2.jar:4.13.2]
> at
> org.junit.internal.runners.statements.FailOnTimeout$CallableStatement.call(FailOnTimeout.java:293)
> ~[junit-4.13.2.jar:4.13.2]
> at java.util.concurrent.FutureTask.run(FutureTask.java:264) ~[?:?]
> at java.lang.Thread.run(Thread.java:829) ~[?:?]
> Caused by: java.lang.IndexOutOfBoundsException: index (21) must be less than
> size (1)
> at
> org.apache.hbase.thirdparty.com.google.common.base.Preconditions.checkElementIndex(Preconditions.java:1371)
>
[jira] [Comment Edited] (HBASE-28390) WAL value compression fails for cells with large values
[
https://issues.apache.org/jira/browse/HBASE-28390?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17819403#comment-17819403
]
Andrew Kyle Purtell edited comment on HBASE-28390 at 2/21/24 10:40 PM:
---
IntegrationTestLoadCommonCrawl loads and verifies the data. It does not read
the WAL per se but I had replication enabled and was verifying on the far side
at least some of the time.
was (Author: apurtell):
IntegrationTestLoadCommonCrawl loads and verifies the data. It does not read
the WAL per se but I had replication enabled and was verifying on the far side.
> WAL value compression fails for cells with large values
> ---
>
> Key: HBASE-28390
> URL: https://issues.apache.org/jira/browse/HBASE-28390
> Project: HBase
> Issue Type: Bug
>Reporter: Bryan Beaudreault
>Priority: Major
>
> We are testing out WAL compression and noticed that it fails for large values
> when both features (wal compression and wal value compression) are enabled.
> It works fine with either feature independently, but not when combined. It
> seems to fail for all of the value compressor types, and the failure is in
> the LRUDictionary of wal key compression:
>
> {code:java}
> java.io.IOException: Error while reading 2 WAL KVs; started reading at 230
> and read up to 396
> at
> org.apache.hadoop.hbase.regionserver.wal.ProtobufWALStreamReader.next(ProtobufWALStreamReader.java:94)
> ~[classes/:?]
> at
> org.apache.hadoop.hbase.wal.CompressedWALTestBase.doTest(CompressedWALTestBase.java:181)
> ~[test-classes/:?]
> at
> org.apache.hadoop.hbase.wal.CompressedWALTestBase.testForSize(CompressedWALTestBase.java:129)
> ~[test-classes/:?]
> at
> org.apache.hadoop.hbase.wal.CompressedWALTestBase.testLarge(CompressedWALTestBase.java:94)
> ~[test-classes/:?]
> at jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> ~[?:?]
> at
> jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> ~[?:?]
> at
> jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> ~[?:?]
> at java.lang.reflect.Method.invoke(Method.java:566) ~[?:?]
> at
> org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:59)
> ~[junit-4.13.2.jar:4.13.2]
> at
> org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
> ~[junit-4.13.2.jar:4.13.2]
> at
> org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:56)
> ~[junit-4.13.2.jar:4.13.2]
> at
> org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
> ~[junit-4.13.2.jar:4.13.2]
> at org.junit.runners.ParentRunner$3.evaluate(ParentRunner.java:306)
> ~[junit-4.13.2.jar:4.13.2]
> at
> org.junit.runners.BlockJUnit4ClassRunner$1.evaluate(BlockJUnit4ClassRunner.java:100)
> ~[junit-4.13.2.jar:4.13.2]
> at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:366)
> ~[junit-4.13.2.jar:4.13.2]
> at
> org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:103)
> ~[junit-4.13.2.jar:4.13.2]
> at
> org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:63)
> ~[junit-4.13.2.jar:4.13.2]
> at org.junit.runners.ParentRunner$4.run(ParentRunner.java:331)
> ~[junit-4.13.2.jar:4.13.2]
> at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:79)
> ~[junit-4.13.2.jar:4.13.2]
> at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:329)
> ~[junit-4.13.2.jar:4.13.2]
> at org.junit.runners.ParentRunner.access$100(ParentRunner.java:66)
> ~[junit-4.13.2.jar:4.13.2]
> at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:293)
> ~[junit-4.13.2.jar:4.13.2]
> at
> org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26)
> ~[junit-4.13.2.jar:4.13.2]
> at
> org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27)
> ~[junit-4.13.2.jar:4.13.2]
> at
> org.junit.internal.runners.statements.FailOnTimeout$CallableStatement.call(FailOnTimeout.java:299)
> ~[junit-4.13.2.jar:4.13.2]
> at
> org.junit.internal.runners.statements.FailOnTimeout$CallableStatement.call(FailOnTimeout.java:293)
> ~[junit-4.13.2.jar:4.13.2]
> at java.util.concurrent.FutureTask.run(FutureTask.java:264) ~[?:?]
> at java.lang.Thread.run(Thread.java:829) ~[?:?]
> Caused by: java.lang.IndexOutOfBoundsException: index (21) must be less than
> size (1)
> at
> org.apache.hbase.thirdparty.com.google.common.base.Preconditions.checkElementIndex(Preconditions.java:1371)
> ~[hbase-shaded-miscellaneous-4.1.5.jar:4.1.5]
> at
>
[jira] [Comment Edited] (HBASE-28390) WAL value compression fails for cells with large values
[
https://issues.apache.org/jira/browse/HBASE-28390?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17819399#comment-17819399
]
Andrew Kyle Purtell edited comment on HBASE-28390 at 2/21/24 10:15 PM:
---
I find the failure interesting because I tested WAL compression using
IntegrationTestLoadCommonCrawl which would have value payloads well in excess
of ~200kb, up to > 1MB. Let me go back and look at my test plan because perhaps
somehow I failed to always read back the written WALs...
bq. But I guess I'm not sure how it's working to begin with, since
BlockDecompressorStream does not know to read an extra 0 int. So maybe not
breaking to fix after all?
I don't understand this either.
Maybe {{len > MAX_INPUT_SIZE}} is rare?
bq. Or maybe we can work around this in hbase.
We can certainly do that. We could even import BlockCompressorStream and
BlockDecompressorStream and bug fix in our tree. Unfortunately I worry about
cases where people have used Hadoop compression codecs for compressing WALs and
HFiles, and then upgrade to a version where we have our fixed compressor stream
implementations, and they are not compatible unless we add special handling.
Maintaining compatibility would hinge on knowing under what circumstances we
should read that extra 0. I have not thought about this in detail so do not
know how challenging that might be.
was (Author: apurtell):
I find the failure interesting because I tested WAL compression using
IntegrationTestLoadCommonCrawl which would have value payloads well in excess
of ~200kb, up to > 1MB. Let me go back and look at my test plan because perhaps
somehow I failed to always read back the written WALs...
bq. But I guess I'm not sure how it's working to begin with, since
BlockDecompressorStream does not know to read an extra 0 int. So maybe not
breaking to fix after all?
I don't understand this either.
bq. Or maybe we can work around this in hbase.
We can certainly do that. We could even import BlockCompressorStream and
BlockDecompressorStream and bug fix in our tree. Unfortunately I worry about
cases where people have used Hadoop compression codecs for compressing WALs and
HFiles, and then upgrade to a version where we have our fixed compressor stream
implementations, and they are not compatible unless we add special handling.
Maintaining compatibility would hinge on knowing under what circumstances we
should read that extra 0. I have not thought about this in detail so do not
know how challenging that might be.
> WAL value compression fails for cells with large values
> ---
>
> Key: HBASE-28390
> URL: https://issues.apache.org/jira/browse/HBASE-28390
> Project: HBase
> Issue Type: Bug
>Reporter: Bryan Beaudreault
>Priority: Major
>
> We are testing out WAL compression and noticed that it fails for large values
> when both features (wal compression and wal value compression) are enabled.
> It works fine with either feature independently, but not when combined. It
> seems to fail for all of the value compressor types, and the failure is in
> the LRUDictionary of wal key compression:
>
> {code:java}
> java.io.IOException: Error while reading 2 WAL KVs; started reading at 230
> and read up to 396
> at
> org.apache.hadoop.hbase.regionserver.wal.ProtobufWALStreamReader.next(ProtobufWALStreamReader.java:94)
> ~[classes/:?]
> at
> org.apache.hadoop.hbase.wal.CompressedWALTestBase.doTest(CompressedWALTestBase.java:181)
> ~[test-classes/:?]
> at
> org.apache.hadoop.hbase.wal.CompressedWALTestBase.testForSize(CompressedWALTestBase.java:129)
> ~[test-classes/:?]
> at
> org.apache.hadoop.hbase.wal.CompressedWALTestBase.testLarge(CompressedWALTestBase.java:94)
> ~[test-classes/:?]
> at jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> ~[?:?]
> at
> jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> ~[?:?]
> at
> jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> ~[?:?]
> at java.lang.reflect.Method.invoke(Method.java:566) ~[?:?]
> at
> org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:59)
> ~[junit-4.13.2.jar:4.13.2]
> at
> org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
> ~[junit-4.13.2.jar:4.13.2]
> at
> org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:56)
> ~[junit-4.13.2.jar:4.13.2]
> at
> org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
> ~[junit-4.13.2.jar:4.13.2]
> at org.junit.runners.ParentRunner$3.evaluate(ParentRunner.java:306)
> ~[junit-4.13.2.jar:4.13.2]
> at
>
[jira] [Comment Edited] (HBASE-28390) WAL value compression fails for cells with large values
[
https://issues.apache.org/jira/browse/HBASE-28390?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17819399#comment-17819399
]
Andrew Kyle Purtell edited comment on HBASE-28390 at 2/21/24 10:13 PM:
---
I find the failure interesting because I tested WAL compression using
IntegrationTestLoadCommonCrawl which would have value payloads well in excess
of ~200kb, up to > 1MB. Let me go back and look at my test plan because perhaps
somehow I failed to always read back the written WALs...
bq. But I guess I'm not sure how it's working to begin with, since
BlockDecompressorStream does not know to read an extra 0 int. So maybe not
breaking to fix after all?
I don't understand this either.
bq. Or maybe we can work around this in hbase.
We can certainly do that. We could even import BlockCompressorStream and
BlockDecompressorStream and bug fix in our tree. Unfortunately I worry about
cases where people have used Hadoop compression codecs for compressing WALs and
HFiles, and then upgrade to a version where we have our fixed compressor stream
implementations, and they are not compatible unless we add special handling.
Maintaining compatibility would hinge on knowing under what circumstances we
should read that extra 0. I have not thought about this in detail so do not
know how challenging that might be.
was (Author: apurtell):
I find the failure interesting because I tested WAL compression using
IntegrationTestLoadCommonCrawl which would have value payloads well in excess
of ~200kb, up to > 1MB. Let me go back and look at my test plan because perhaps
somehow I failed to always read back the written WALs...
bq. But I guess I'm not sure how it's working to begin with, since
BlockDecompressorStream does not know to read an extra 0 int. So maybe not
breaking to fix after all?
I don't understand this either.
bq. Or maybe we can work around this in hbase.
We can certainly do that. We could even import BlockCompressorStream and
BlockDecompressorStream and bug fix in our tree. Unfortunately I worry about
cases where people have used Hadoop compression codecs for compressing WALs and
HFiles, and then upgrade to a version where we have our fixed compressor stream
implementations, and they are not compatibility. Maintaining compatibility
would hinge on knowing under what circumstances we should read that extra 0. I
have not thought about this in detail so do not know how challenging that might
be.
> WAL value compression fails for cells with large values
> ---
>
> Key: HBASE-28390
> URL: https://issues.apache.org/jira/browse/HBASE-28390
> Project: HBase
> Issue Type: Bug
>Reporter: Bryan Beaudreault
>Priority: Major
>
> We are testing out WAL compression and noticed that it fails for large values
> when both features (wal compression and wal value compression) are enabled.
> It works fine with either feature independently, but not when combined. It
> seems to fail for all of the value compressor types, and the failure is in
> the LRUDictionary of wal key compression:
>
> {code:java}
> java.io.IOException: Error while reading 2 WAL KVs; started reading at 230
> and read up to 396
> at
> org.apache.hadoop.hbase.regionserver.wal.ProtobufWALStreamReader.next(ProtobufWALStreamReader.java:94)
> ~[classes/:?]
> at
> org.apache.hadoop.hbase.wal.CompressedWALTestBase.doTest(CompressedWALTestBase.java:181)
> ~[test-classes/:?]
> at
> org.apache.hadoop.hbase.wal.CompressedWALTestBase.testForSize(CompressedWALTestBase.java:129)
> ~[test-classes/:?]
> at
> org.apache.hadoop.hbase.wal.CompressedWALTestBase.testLarge(CompressedWALTestBase.java:94)
> ~[test-classes/:?]
> at jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> ~[?:?]
> at
> jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> ~[?:?]
> at
> jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> ~[?:?]
> at java.lang.reflect.Method.invoke(Method.java:566) ~[?:?]
> at
> org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:59)
> ~[junit-4.13.2.jar:4.13.2]
> at
> org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
> ~[junit-4.13.2.jar:4.13.2]
> at
> org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:56)
> ~[junit-4.13.2.jar:4.13.2]
> at
> org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
> ~[junit-4.13.2.jar:4.13.2]
> at org.junit.runners.ParentRunner$3.evaluate(ParentRunner.java:306)
> ~[junit-4.13.2.jar:4.13.2]
> at
> org.junit.runners.BlockJUnit4ClassRunner$1.evaluate(BlockJUnit4ClassRunner.java:100)
>
[jira] [Comment Edited] (HBASE-28390) WAL value compression fails for cells with large values
[
https://issues.apache.org/jira/browse/HBASE-28390?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17819399#comment-17819399
]
Andrew Kyle Purtell edited comment on HBASE-28390 at 2/21/24 10:12 PM:
---
I find the failure interesting because I tested WAL compression using
IntegrationTestLoadCommonCrawl which would have value payloads well in excess
of ~200kb, up to > 1MB. Let me go back and look at my test plan because perhaps
somehow I failed to always read back the written WALs...
bq. But I guess I'm not sure how it's working to begin with, since
BlockDecompressorStream does not know to read an extra 0 int. So maybe not
breaking to fix after all?
I don't understand this either.
bq. Or maybe we can work around this in hbase.
We can certainly do that. We could even import BlockCompressorStream and
BlockDecompressorStream and bug fix in our tree. Unfortunately I worry about
cases where people have used Hadoop compression codecs for compressing WALs and
HFiles, and then upgrade to a version where we have our fixed compressor stream
implementations, and they are not compatibility. Maintaining compatibility
would hinge on knowing under what circumstances we should read that extra 0. I
have not thought about this in detail so do not know how challenging that might
be.
was (Author: apurtell):
I find the failure interesting because I tested WAL compression using
IntegrationTestLoadCommonCrawl which would have value payloads well in excess
of ~200kb, up to > 1MB.
bq. But I guess I'm not sure how it's working to begin with, since
BlockDecompressorStream does not know to read an extra 0 int. So maybe not
breaking to fix after all?
I don't understand this either.
bq. Or maybe we can work around this in hbase.
We can certainly do that. We could even import BlockCompressorStream and
BlockDecompressorStream and bug fix in our tree. Unfortunately I worry about
cases where people have used Hadoop compression codecs for compressing WALs and
HFiles, and then upgrade to a version where we have our fixed compressor stream
implementations, and they are not compatibility. Maintaining compatibility
would hinge on knowing under what circumstances we should read that extra 0. I
have not thought about this in detail so do not know how challenging that might
be.
> WAL value compression fails for cells with large values
> ---
>
> Key: HBASE-28390
> URL: https://issues.apache.org/jira/browse/HBASE-28390
> Project: HBase
> Issue Type: Bug
>Reporter: Bryan Beaudreault
>Priority: Major
>
> We are testing out WAL compression and noticed that it fails for large values
> when both features (wal compression and wal value compression) are enabled.
> It works fine with either feature independently, but not when combined. It
> seems to fail for all of the value compressor types, and the failure is in
> the LRUDictionary of wal key compression:
>
> {code:java}
> java.io.IOException: Error while reading 2 WAL KVs; started reading at 230
> and read up to 396
> at
> org.apache.hadoop.hbase.regionserver.wal.ProtobufWALStreamReader.next(ProtobufWALStreamReader.java:94)
> ~[classes/:?]
> at
> org.apache.hadoop.hbase.wal.CompressedWALTestBase.doTest(CompressedWALTestBase.java:181)
> ~[test-classes/:?]
> at
> org.apache.hadoop.hbase.wal.CompressedWALTestBase.testForSize(CompressedWALTestBase.java:129)
> ~[test-classes/:?]
> at
> org.apache.hadoop.hbase.wal.CompressedWALTestBase.testLarge(CompressedWALTestBase.java:94)
> ~[test-classes/:?]
> at jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> ~[?:?]
> at
> jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> ~[?:?]
> at
> jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> ~[?:?]
> at java.lang.reflect.Method.invoke(Method.java:566) ~[?:?]
> at
> org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:59)
> ~[junit-4.13.2.jar:4.13.2]
> at
> org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
> ~[junit-4.13.2.jar:4.13.2]
> at
> org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:56)
> ~[junit-4.13.2.jar:4.13.2]
> at
> org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
> ~[junit-4.13.2.jar:4.13.2]
> at org.junit.runners.ParentRunner$3.evaluate(ParentRunner.java:306)
> ~[junit-4.13.2.jar:4.13.2]
> at
> org.junit.runners.BlockJUnit4ClassRunner$1.evaluate(BlockJUnit4ClassRunner.java:100)
> ~[junit-4.13.2.jar:4.13.2]
> at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:366)
> ~[junit-4.13.2.jar:4.13.2]
> at
>
[jira] [Comment Edited] (HBASE-28390) WAL value compression fails for cells with large values
[
https://issues.apache.org/jira/browse/HBASE-28390?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17819369#comment-17819369
]
Bryan Beaudreault edited comment on HBASE-28390 at 2/21/24 7:45 PM:
I think the bug is related to exceeding the
[MAX_INPUT_SIZE|https://github.com/apache/hadoop/blob/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/io/compress/BlockCompressorStream.java#L43].
The value for zstd is 261120, and we see it fail for inputs of size >= 261121.
It looks like this case should be handled, breaking the input into smaller
chunks. I'm thinking we are missing reading an int field somewhere, because the
bytes read is consistently off by 4 bytes from the expected compressedLen
retrieved from readCompressedValue
was (Author: bbeaudreault):
I think the bug is related to exceeding the
[MAX_INPUT_SIZE|https://github.com/apache/hadoop/blob/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/io/compress/BlockCompressorStream.java#L43].
The value for zstd is 261120, and we see it fail for inputs of size 261121. It
looks like this case should be handled, breaking the input into smaller chunks.
I'm thinking we are missing reading an int field somewhere, because the bytes
read is consistently off by 4 bytes from the expected compressedLen retrieved
from readCompressedValue
> WAL value compression fails for cells with large values
> ---
>
> Key: HBASE-28390
> URL: https://issues.apache.org/jira/browse/HBASE-28390
> Project: HBase
> Issue Type: Bug
>Reporter: Bryan Beaudreault
>Priority: Major
>
> We are testing out WAL compression and noticed that it fails for large values
> when both features (wal compression and wal value compression) are enabled.
> It works fine with either feature independently, but not when combined. It
> seems to fail for all of the value compressor types, and the failure is in
> the LRUDictionary of wal key compression:
>
> {code:java}
> java.io.IOException: Error while reading 2 WAL KVs; started reading at 230
> and read up to 396
> at
> org.apache.hadoop.hbase.regionserver.wal.ProtobufWALStreamReader.next(ProtobufWALStreamReader.java:94)
> ~[classes/:?]
> at
> org.apache.hadoop.hbase.wal.CompressedWALTestBase.doTest(CompressedWALTestBase.java:181)
> ~[test-classes/:?]
> at
> org.apache.hadoop.hbase.wal.CompressedWALTestBase.testForSize(CompressedWALTestBase.java:129)
> ~[test-classes/:?]
> at
> org.apache.hadoop.hbase.wal.CompressedWALTestBase.testLarge(CompressedWALTestBase.java:94)
> ~[test-classes/:?]
> at jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> ~[?:?]
> at
> jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> ~[?:?]
> at
> jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> ~[?:?]
> at java.lang.reflect.Method.invoke(Method.java:566) ~[?:?]
> at
> org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:59)
> ~[junit-4.13.2.jar:4.13.2]
> at
> org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
> ~[junit-4.13.2.jar:4.13.2]
> at
> org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:56)
> ~[junit-4.13.2.jar:4.13.2]
> at
> org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
> ~[junit-4.13.2.jar:4.13.2]
> at org.junit.runners.ParentRunner$3.evaluate(ParentRunner.java:306)
> ~[junit-4.13.2.jar:4.13.2]
> at
> org.junit.runners.BlockJUnit4ClassRunner$1.evaluate(BlockJUnit4ClassRunner.java:100)
> ~[junit-4.13.2.jar:4.13.2]
> at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:366)
> ~[junit-4.13.2.jar:4.13.2]
> at
> org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:103)
> ~[junit-4.13.2.jar:4.13.2]
> at
> org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:63)
> ~[junit-4.13.2.jar:4.13.2]
> at org.junit.runners.ParentRunner$4.run(ParentRunner.java:331)
> ~[junit-4.13.2.jar:4.13.2]
> at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:79)
> ~[junit-4.13.2.jar:4.13.2]
> at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:329)
> ~[junit-4.13.2.jar:4.13.2]
> at org.junit.runners.ParentRunner.access$100(ParentRunner.java:66)
> ~[junit-4.13.2.jar:4.13.2]
> at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:293)
> ~[junit-4.13.2.jar:4.13.2]
> at
> org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26)
> ~[junit-4.13.2.jar:4.13.2]
> at
> org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27)
>
