Re: Infinite loop in org.apache.cassandra.hadoop.cql3.CqlBulkRecordWriter

2019-04-03 Thread Brett Marcott
Thanks for the recommendation Russell. Haven't looked into that code yet, but the docs didn't seem to indicate if it wrote sstables directly instead of going through normal write path. On Wed, Apr 3, 2019, 11:11 AM Russell Spitzer wrote: > I would recommend using the Spark Cassandra Connector i

Re: Infinite loop in org.apache.cassandra.hadoop.cql3.CqlBulkRecordWriter

2019-04-03 Thread Russell Spitzer
I would recommend using the Spark Cassandra Connector instead of the Hadoop based writers. The Hadoop code has not had a lot of love in a long time. See https://github.com/datastax/spark-cassandra-connector On Wed, Apr 3, 2019 at 12:21 PM Brett Marcott wrote: > Hi folks, > > I am noticing my sp

Infinite loop in org.apache.cassandra.hadoop.cql3.CqlBulkRecordWriter

2019-04-03 Thread Brett Marcott
Hi folks, I am noticing my spark jobs being stuck when using the org.apache.cassandra.hadoop.cql3.CqlBulkRecordWriter/CqlBulkOutputFormat. It seems that whenever there is a stream failure it may be expected behavior based on the code to infinite loop. Here are one executors logs: 19/04/03 15: