Francisco Guerrero created CASSANDRA-19727:
----------------------------------------------
Summary: [Analytics] Bulk writer fails validation stage when
writing to a cluster using RandomPartitioner
Key: CASSANDRA-19727
URL: https://issues.apache.org/jira/browse/CASSANDRA-19727
Project: Cassandra
Issue Type: Bug
Components: Analytics Library
Reporter: Francisco Guerrero
Assignee: Francisco Guerrero
In bulk writer after writing SSTables, written data to SSTables is read back
again to perform validations before shipping the files to the Cassandra
instances. The logic to validate the SSTables assumes
{{{}Murmur3Partitioner{}}}. This validation fails however when a bulk writer
job is running against a cluster using the {{RandomPartitioner}} with the
following stacktrace:
{code}
java.lang.RuntimeException: java.lang.IllegalStateException: Partitioner in
ValidationMetadata does not match TableMetaData:
org.apache.cassandra.dht.RandomPartitioner vs.
org.apache.cassandra.dht.Murmur3Partitioner
at
org.apache.cassandra.spark.utils.Throwing.lambda$function$2(Throwing.java:84)
at
java.base/java.util.stream.ReferencePipeline$3$1.accept(ReferencePipeline.java:197)
at
java.base/java.util.HashMap$KeySpliterator.forEachRemaining(HashMap.java:1707)
at
java.base/java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:509)
at
java.base/java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:499)
at
java.base/java.util.stream.ReduceOps$ReduceOp.evaluateSequential(ReduceOps.java:921)
at
java.base/java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234)
at
java.base/java.util.stream.ReferencePipeline.collect(ReferencePipeline.java:682)
at
org.apache.cassandra.spark.data.BasicSupplier.openAll(BasicSupplier.java:44)
at
org.apache.cassandra.bridge.CassandraBridgeImplementation.getCompactionScanner(CassandraBridgeImplementation.java:248)
at
org.apache.cassandra.spark.data.DataLayer.openCompactionScanner(DataLayer.java:262)
at
org.apache.cassandra.spark.bulkwriter.SSTableWriter.validateSSTables(SSTableWriter.java:129)
at
org.apache.cassandra.spark.bulkwriter.SSTableWriter.close(SSTableWriter.java:113)
at
org.apache.cassandra.spark.bulkwriter.RecordWriter.finalizeSSTable(RecordWriter.java:224)
at
org.apache.cassandra.spark.bulkwriter.RecordWriter.write(RecordWriter.java:122)
... 15 more
{code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
