[
https://issues.apache.org/jira/browse/CASSANDRA-20022?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Niket Vilas Bagwe updated CASSANDRA-20022:
------------------------------------------
Fix Version/s: 4.0.x
4.1.x
> ChannelProxy read might not read complete buffer causing "Checksum didn't
> match" error for sstableloader
> --------------------------------------------------------------------------------------------------------
>
> Key: CASSANDRA-20022
> URL: https://issues.apache.org/jira/browse/CASSANDRA-20022
> Project: Cassandra
> Issue Type: Bug
> Components: Tool/bulk load
> Reporter: Niket Vilas Bagwe
> Priority: Normal
> Fix For: 4.0.x, 4.1.x
>
>
> CassandraCompressedStreamWriter expects the outbuffer to be filled completely
> while issuing the read call for ChannelProxy. Please refer the code snippet
> below from CassandraCompressedStreamWriter.
> {code:java}
> out.writeToChannel(bufferSupplier -> {
> ByteBuffer outBuffer = bufferSupplier.get(toTransfer);
> long read = fc.read(outBuffer, position);
> assert read == toTransfer : String.format("could not read required
> number of bytes from file to be streamed: read %d bytes, wanted %d bytes",
> read, toTransfer);
> outBuffer.flip();
> }, limiter); {code}
> This is problematic since the underlying FileChannel implementation for
> ChannelProxy can have reads less than the buffer size causing the file upload
> operation to fail.
> Also, to catch the above problem an assertion is used instead of using an
> Exception. Assertions have to enabled explicitly by passing -ea flag to the
> java runtime.
> If assertions are disabled, the above partial read operation can cause an
> invalid buffer to get transferred to the cassandra peer causing "Checksum
> didn't match" error on the peer as shown below.
> {code:java}
> org.apache.cassandra.streaming.StreamReceiveException: java.io.IOException:
> Checksum didn't match (expected: 1967481976, actual: 2124622197)
> at
> org.apache.cassandra.streaming.messages.IncomingStreamMessage$1.deserialize(IncomingStreamMessage.java:60)
> at
> org.apache.cassandra.streaming.messages.IncomingStreamMessage$1.deserialize(IncomingStreamMessage.java:38)
> at
> org.apache.cassandra.streaming.messages.StreamMessage.deserialize(StreamMessage.java:53)
> at
> org.apache.cassandra.streaming.async.StreamingInboundHandler$StreamDeserializingTask.run(StreamingInboundHandler.java:172)
> at
> io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
> at java.base/java.lang.Thread.run(Thread.java:834)
> Caused by: java.io.IOException: Checksum didn't match (expected: 1967481976,
> actual: 2124622197)
> at
> org.apache.cassandra.db.streaming.CompressedInputStream.maybeValidateChecksum(CompressedInputStream.java:206)
> at
> org.apache.cassandra.db.streaming.CompressedInputStream.loadNextChunk(CompressedInputStream.java:157)
> at
> org.apache.cassandra.db.streaming.CompressedInputStream.reBuffer(CompressedInputStream.java:121)
> at
> org.apache.cassandra.io.util.RebufferingInputStream.read(RebufferingInputStream.java:90)
> at
> org.apache.cassandra.io.util.RebufferingInputStream.readFully(RebufferingInputStream.java:68)
> at
> org.apache.cassandra.io.util.RebufferingInputStream.readFully(RebufferingInputStream.java:62)
> at
> org.apache.cassandra.io.util.TrackedDataInputPlus.readFully(TrackedDataInputPlus.java:93)
> at
> org.apache.cassandra.db.marshal.ByteArrayAccessor.read(ByteArrayAccessor.java:101)
> at
> org.apache.cassandra.db.marshal.ByteArrayAccessor.read(ByteArrayAccessor.java:38)
> at
> org.apache.cassandra.db.marshal.AbstractType.read(AbstractType.java:500)
> at
> org.apache.cassandra.db.marshal.AbstractType.readArray(AbstractType.java:480)
> at
> org.apache.cassandra.db.ClusteringPrefix$Serializer.deserializeValuesWithoutSize(ClusteringPrefix.java:441)
> at
> org.apache.cassandra.db.Clustering$Serializer.deserialize(Clustering.java:164)
> at
> org.apache.cassandra.db.rows.UnfilteredSerializer.deserializeOne(UnfilteredSerializer.java:477)
> at
> org.apache.cassandra.db.rows.UnfilteredSerializer.deserialize(UnfilteredSerializer.java:434)
> at
> org.apache.cassandra.io.sstable.SSTableSimpleIterator$CurrentFormatIterator.computeNext(SSTableSimpleIterator.java:84)
> at
> org.apache.cassandra.io.sstable.SSTableSimpleIterator$CurrentFormatIterator.computeNext(SSTableSimpleIterator.java:62)
> at
> org.apache.cassandra.utils.AbstractIterator.hasNext(AbstractIterator.java:47)
> at
> org.apache.cassandra.db.streaming.CassandraStreamReader$StreamDeserializer.hasNext(CassandraStreamReader.java:249)
> at
> org.apache.cassandra.db.rows.UnfilteredRowIterator.isEmpty(UnfilteredRowIterator.java:67)
> at
> org.apache.cassandra.io.sstable.format.big.BigTableWriter.append(BigTableWriter.java:205)
> at
> org.apache.cassandra.io.sstable.SimpleSSTableMultiWriter.append(SimpleSSTableMultiWriter.java:48)
> at
> org.apache.cassandra.io.sstable.format.RangeAwareSSTableWriter.append(RangeAwareSSTableWriter.java:107)
> at
> org.apache.cassandra.db.streaming.CassandraStreamReader.writePartition(CassandraStreamReader.java:175)
> at
> org.apache.cassandra.db.streaming.CassandraCompressedStreamReader.read(CassandraCompressedStreamReader.java:98)
> at
> org.apache.cassandra.db.streaming.CassandraIncomingFile.read(CassandraIncomingFile.java:84)
> at
> org.apache.cassandra.streaming.messages.IncomingStreamMessage$1.deserialize(IncomingStreamMessage.java:53)
> ... 5 common frames omitted {code}
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
