[
https://issues.apache.org/jira/browse/HDFS-16544?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
qinyuren updated HDFS-16544:
----------------------------
Description:
In [HDFS-16538|http://https//issues.apache.org/jira/browse/HDFS-16538] , we
found a ec decoding bug if more than one data block read failed.
Currently, we found another bug trigger by #StatefulStripeReader.decode.
If we read an EC file which {*}length more than one stripe{*}, and this file
have *one data block* and *the first parity block* corrupted, this error will
happen.
{code:java}
org.apache.hadoop.HadoopIllegalArgumentException: Invalid buffer found, not
allowing null at
org.apache.hadoop.io.erasurecode.rawcoder.ByteBufferDecodingState.checkOutputBuffers(ByteBufferDecodingState.java:132)
at
org.apache.hadoop.io.erasurecode.rawcoder.ByteBufferDecodingState.<init>(ByteBufferDecodingState.java:48)
at
org.apache.hadoop.io.erasurecode.rawcoder.RawErasureDecoder.decode(RawErasureDecoder.java:86)
at
org.apache.hadoop.io.erasurecode.rawcoder.RawErasureDecoder.decode(RawErasureDecoder.java:170)
at
org.apache.hadoop.hdfs.StripeReader.decodeAndFillBuffer(StripeReader.java:435)
at
org.apache.hadoop.hdfs.StatefulStripeReader.decode(StatefulStripeReader.java:94)
at org.apache.hadoop.hdfs.StripeReader.readStripe(StripeReader.java:392)
at
org.apache.hadoop.hdfs.DFSStripedInputStream.readOneStripe(DFSStripedInputStream.java:315)
at
org.apache.hadoop.hdfs.DFSStripedInputStream.readWithStrategy(DFSStripedInputStream.java:408)
at org.apache.hadoop.hdfs.DFSInputStream.read(DFSInputStream.java:918)
{code}
was:
In [HDFS-16538|http://https://issues.apache.org/jira/browse/HDFS-16538] , we
found a ec decoding bug if more than one data block read failed.
Currently, we found another bug trigger by #StatefulStripeReader.decode.
If we read an EC file which length more than one stripe, and this file have one
data block and the first parity block corrupted, this error will happen.
{code:java}
org.apache.hadoop.HadoopIllegalArgumentException: Invalid buffer found, not
allowing null at
org.apache.hadoop.io.erasurecode.rawcoder.ByteBufferDecodingState.checkOutputBuffers(ByteBufferDecodingState.java:132)
at
org.apache.hadoop.io.erasurecode.rawcoder.ByteBufferDecodingState.<init>(ByteBufferDecodingState.java:48)
at
org.apache.hadoop.io.erasurecode.rawcoder.RawErasureDecoder.decode(RawErasureDecoder.java:86)
at
org.apache.hadoop.io.erasurecode.rawcoder.RawErasureDecoder.decode(RawErasureDecoder.java:170)
at
org.apache.hadoop.hdfs.StripeReader.decodeAndFillBuffer(StripeReader.java:435)
at
org.apache.hadoop.hdfs.StatefulStripeReader.decode(StatefulStripeReader.java:94)
at org.apache.hadoop.hdfs.StripeReader.readStripe(StripeReader.java:392)
at
org.apache.hadoop.hdfs.DFSStripedInputStream.readOneStripe(DFSStripedInputStream.java:315)
at
org.apache.hadoop.hdfs.DFSStripedInputStream.readWithStrategy(DFSStripedInputStream.java:408)
at org.apache.hadoop.hdfs.DFSInputStream.read(DFSInputStream.java:918)
{code}
> EC decoding failed due to invalid buffer
> ----------------------------------------
>
> Key: HDFS-16544
> URL: https://issues.apache.org/jira/browse/HDFS-16544
> Project: Hadoop HDFS
> Issue Type: Bug
> Reporter: qinyuren
> Priority: Major
>
> In [HDFS-16538|http://https//issues.apache.org/jira/browse/HDFS-16538] , we
> found a ec decoding bug if more than one data block read failed.
> Currently, we found another bug trigger by #StatefulStripeReader.decode.
> If we read an EC file which {*}length more than one stripe{*}, and this file
> have *one data block* and *the first parity block* corrupted, this error will
> happen.
> {code:java}
> org.apache.hadoop.HadoopIllegalArgumentException: Invalid buffer found, not
> allowing null at
> org.apache.hadoop.io.erasurecode.rawcoder.ByteBufferDecodingState.checkOutputBuffers(ByteBufferDecodingState.java:132)
> at
> org.apache.hadoop.io.erasurecode.rawcoder.ByteBufferDecodingState.<init>(ByteBufferDecodingState.java:48)
> at
> org.apache.hadoop.io.erasurecode.rawcoder.RawErasureDecoder.decode(RawErasureDecoder.java:86)
> at
> org.apache.hadoop.io.erasurecode.rawcoder.RawErasureDecoder.decode(RawErasureDecoder.java:170)
> at
> org.apache.hadoop.hdfs.StripeReader.decodeAndFillBuffer(StripeReader.java:435)
> at
> org.apache.hadoop.hdfs.StatefulStripeReader.decode(StatefulStripeReader.java:94)
> at org.apache.hadoop.hdfs.StripeReader.readStripe(StripeReader.java:392)
> at
> org.apache.hadoop.hdfs.DFSStripedInputStream.readOneStripe(DFSStripedInputStream.java:315)
> at
> org.apache.hadoop.hdfs.DFSStripedInputStream.readWithStrategy(DFSStripedInputStream.java:408)
> at org.apache.hadoop.hdfs.DFSInputStream.read(DFSInputStream.java:918)
> {code}
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
