Re: How to tune Cassandra or Java Driver to get lower latency when there are a lot of writes?

2015-09-25 Thread Benyi Wang
Hi Ryan, As I said, saveToCassandra doesn't support "DELETE". This is why I modified the code of spark-cassandra-connector to allow me have DELETEs. What I change is how to bind a RDD row into a batch of CQL preparedStatements. On Fri, Sep 25, 2015 at 7:22 AM, Ryan Svihla

Re: How to tune Cassandra or Java Driver to get lower latency when there are a lot of writes?

2015-09-25 Thread Ryan Svihla
Why aren’t you using saveToCassandra (https://github.com/datastax/spark-cassandra-connector/blob/master/doc/5_saving.md )? They have a number of locality aware optimizations that will probably exceed your by

How to tune Cassandra or Java Driver to get lower latency when there are a lot of writes?

2015-09-24 Thread Benyi Wang
I have a cassandra cluster provides data to a web service. And there is a daily batch load writing data into the cluster. - Without the batch loading, the service’s Latency 99thPercentile is 3ms. But during the load, it jumps to 90ms. - I checked cassandra keyspace’s

Re: How to tune Cassandra or Java Driver to get lower latency when there are a lot of writes?

2015-09-24 Thread Gerard Maas
How are you loading the data? I mean, what insert method are you using? On Thu, Sep 24, 2015 at 9:58 PM, Benyi Wang wrote: > I have a cassandra cluster provides data to a web service. And there is a > daily batch load writing data into the cluster. > >- Without the

Re: How to tune Cassandra or Java Driver to get lower latency when there are a lot of writes?

2015-09-24 Thread Benyi Wang
I use Spark and spark-cassandra-connector with a customized Cassandra writer (spark-cassandra-connector doesn’t support DELETE). Basically the writer works as follows: - Bind a row in Spark RDD with either INSERT/Delete PreparedStatement - Create a BatchStatement for multiple rows -