[
https://issues.apache.org/jira/browse/HADOOP-16194?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16793620#comment-16793620
]
Surendra Singh Lilhore commented on HADOOP-16194:
-------------------------------------------------
1. Last incomplete cell read cause EOS
{noformat}
java.io.IOException: Unexpected EOS from the reader
at org.apache.hadoop.hdfs.StripeReader.readToBuffer(StripeReader.java:241)
at org.apache.hadoop.hdfs.StripeReader.lambda$readCells$0(StripeReader.java:281)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748){noformat}
2. Buffer length is mismatched because of EOS
{noformat}
Caused by: org.apache.hadoop.HadoopIllegalArgumentException: Invalid buffer,
not of length 214405
at
org.apache.hadoop.io.erasurecode.rawcoder.ByteBufferDecodingState.checkOutputBuffers(ByteBufferDecodingState.java:137)
at
org.apache.hadoop.io.erasurecode.rawcoder.ByteBufferDecodingState.<init>(ByteBufferDecodingState.java:48)
at
org.apache.hadoop.io.erasurecode.rawcoder.RawErasureDecoder.decode(RawErasureDecoder.java:86)
at
org.apache.hadoop.io.erasurecode.rawcoder.RawErasureDecoder.decode(RawErasureDecoder.java:170)
at
org.apache.hadoop.hdfs.StripeReader.decodeAndFillBuffer(StripeReader.java:430)
at
org.apache.hadoop.hdfs.PositionStripeReader.decode(PositionStripeReader.java:74)
at org.apache.hadoop.hdfs.StripeReader.readStripe(StripeReader.java:389)
at
org.apache.hadoop.hdfs.DFSStripedInputStream.fetchBlockByteRange(DFSStripedInputStream.java:503)
at org.apache.hadoop.hdfs.DFSInputStream.pread(DFSInputStream.java:1357)
at org.apache.hadoop.hdfs.DFSInputStream.read(DFSInputStream.java:1321)
{noformat}
3. Missing chunk log
{noformat}
2019-03-11 08:36:02,263 DEBUG [main] org.apache.hadoop.hdfs.DFSClient: Read
task returned: (index=1, state =1, readStats
=bytesRead=214405,isShortCircuit=false,networkDistance=0), for stripe
AlignedStripe(Offset=9830567, length=214405, fetchedChunksNum=0,
missingChunksNum=1)
2019-03-11 08:36:02,303 DEBUG [main] org.apache.hadoop.hdfs.DFSClient: Read
task returned: (index=6, state =1, readStats
=bytesRead=214405,isShortCircuit=false,networkDistance=2147483647), for stripe
AlignedStripe(Offset=9830567, length=214405, fetchedChunksNum=1,
missingChunksNum=1)
2019-03-11 08:36:02,305 DEBUG [main] org.apache.hadoop.hdfs.DFSClient: Read
task returned: (index=2, state =2, readStats =null), for stripe
AlignedStripe(Offset=9830567, length=214405, fetchedChunksNum=2,
missingChunksNum=1)
2019-03-11 08:36:02,365 DEBUG [main] org.apache.hadoop.hdfs.DFSClient: Read
task returned: (index=7, state =1, readStats
=bytesRead=214405,isShortCircuit=false,networkDistance=2147483647), for stripe
AlignedStripe(Offset=9830567, length=214405, fetchedChunksNum=2,
missingChunksNum=2)
{noformat}
> EC : Decoding is failing when block group last incomplete cell fall in to
> AlignedStripe
> ---------------------------------------------------------------------------------------
>
> Key: HADOOP-16194
> URL: https://issues.apache.org/jira/browse/HADOOP-16194
> Project: Hadoop Common
> Issue Type: Bug
> Components: hdfs-client
> Affects Versions: 3.1.1
> Reporter: Surendra Singh Lilhore
> Assignee: Surendra Singh Lilhore
> Priority: Major
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
