Re: Sorl/DSE Spark

[Evelyn Smith]

Are you building a search engine -> Solr
Are you building an analytics function -> Spark

I feel they are used in significantly different use cases, what are you trying 
to build?

If it’s an analytics functionality that’s seperate from your operations 
functionality I’d build it in it’s own DC. Spark is notorious for causing 
latency spikes in Cassandra which is not great if you are are sensitive to 
that. 

Regards,
Evelyn.
> On 12 Apr 2018, at 6:55 am, kooljava2  wrote:
> 
> Hello,
> 
> We are exploring on configuring Sorl/Spark. Wanted to get input on this.
> 1) How do we decide which one to use?
> 2) Do we run this on a DC where there is less workload?
> 
> Any other suggestion or comments are appreciated.
> 
> Thank you.
> 

Re: Latest version and Features

[Michael Shuler]

On 04/11/2018 06:12 PM, Carlos Rolo wrote:
> 
> I blogged about this decision recently
> here: https://blog.pythian.com/what-cassandra-version-should-i-use-2018/

s/it the fact/is the fact/ typo, and possibly not 100% correct on the
statement in that sentence.

There are commits since the last 2.1 & 2.2 releases. Generally, we'll do
a last release with any fixes on the branch, before shuttering
development on older branches. 1.0 and 1.1 had a few commits after the
last releases, but 1.2 and 2.0 both had final releases with any bug
fixes we had in-tree. I expect we'll do the same with 2.1 and 2.2 to
wrap things up nicely.

-- 
Warm regards,
Michael

-
To unsubscribe, e-mail: user-unsubscr...@cassandra.apache.org
For additional commands, e-mail: user-h...@cassandra.apache.org

Re: Latest version and Features

[Jonathan Haddad]

I was going to say the same thing, but then I remembered 3.1 == 3.0.1.
There's nothing nothing that makes 3.11 a requirement, so that means
3.0.latest is the safest bet, with 3.11.2 being the one I'd personally go
with also.

On Wed, Apr 11, 2018 at 4:13 PM Carlos Rolo  wrote:

> If you are on 3.1.0 I would move forward to 3.11.2.
>
> I blogged about this decision recently here:
> https://blog.pythian.com/what-cassandra-version-should-i-use-2018/
>
> Regards,
>
> Carlos Juzarte Rolo
> Cassandra Consultant / Datastax Certified Architect / Cassandra MVP
>
> Pythian - Love your data
>
> rolo@pythian | Twitter: @cjrolo | Skype: cjr2k3 | Linkedin:
> *linkedin.com/in/carlosjuzarterolo
> *
> Mobile: +351 918 918 100 <+351%20918%20918%20100>
> www.pythian.com
>
> On Wed, Apr 11, 2018 at 4:27 PM, Nicolas Guyomar <
> nicolas.guyo...@gmail.com> wrote:
>
>> Everything is in the same document, you have a "New features" section
>> plus an "Upgrading" one
>>
>> On 11 April 2018 at 17:24, Abdul Patel  wrote:
>>
>>> Nicolas,
>>> I do see all new features but instructions for upgrade are mentioned in
>>> next section ..not sure if i missed it ..can you share that section?
>>>
>>>
>>> On Wednesday, April 11, 2018, Abdul Patel  wrote:
>>>
 Thanks .this is perfect

 On Wednesday, April 11, 2018, Nicolas Guyomar <
 nicolas.guyo...@gmail.com> wrote:

> Sorry, I should have give you this link instead :
> https://github.com/apache/cassandra/blob/trunk/NEWS.txt
>
> You'll find everything you need IMHO
>
> On 11 April 2018 at 17:05, Abdul Patel  wrote:
>
>> Thanks.
>>
>> Is the upgrade process staright forward do we have any documentation
>> to upgrade?
>>
>>
>> On Wednesday, April 11, 2018, Jonathan Haddad 
>> wrote:
>>
>>> Move to the latest 3.0, or if you're feeling a little more
>>> adventurous, 3.11.2.
>>>
>>> 4.0 discussion is happening now, nothing is decided.
>>>
>>> On Wed, Apr 11, 2018 at 7:35 AM Abdul Patel 
>>> wrote:
>>>
 Hi All,

 Our company is planning for upgrading cassandra to maitain the
 audit gudilines for patch cycle.
 We are currently on 3.1.0, whats the latest stable version and what
 are the new features?
 Will it be better to wait for 4.0? Any news on what will be new
 features in 4.0 ?

>>>
>
>>
>
> --
>
>
>
>

Re: Latest version and Features

[Carlos Rolo]

If you are on 3.1.0 I would move forward to 3.11.2.

I blogged about this decision recently here:
https://blog.pythian.com/what-cassandra-version-should-i-use-2018/

Regards,

Carlos Juzarte Rolo
Cassandra Consultant / Datastax Certified Architect / Cassandra MVP

Pythian - Love your data

rolo@pythian | Twitter: @cjrolo | Skype: cjr2k3 | Linkedin:
*linkedin.com/in/carlosjuzarterolo
*
Mobile: +351 918 918 100
www.pythian.com

On Wed, Apr 11, 2018 at 4:27 PM, Nicolas Guyomar 
wrote:

> Everything is in the same document, you have a "New features" section plus
> an "Upgrading" one
>
> On 11 April 2018 at 17:24, Abdul Patel  wrote:
>
>> Nicolas,
>> I do see all new features but instructions for upgrade are mentioned in
>> next section ..not sure if i missed it ..can you share that section?
>>
>>
>> On Wednesday, April 11, 2018, Abdul Patel  wrote:
>>
>>> Thanks .this is perfect
>>>
>>> On Wednesday, April 11, 2018, Nicolas Guyomar 
>>> wrote:
>>>
 Sorry, I should have give you this link instead :
 https://github.com/apache/cassandra/blob/trunk/NEWS.txt

 You'll find everything you need IMHO

 On 11 April 2018 at 17:05, Abdul Patel  wrote:

> Thanks.
>
> Is the upgrade process staright forward do we have any documentation
> to upgrade?
>
>
> On Wednesday, April 11, 2018, Jonathan Haddad 
> wrote:
>
>> Move to the latest 3.0, or if you're feeling a little more
>> adventurous, 3.11.2.
>>
>> 4.0 discussion is happening now, nothing is decided.
>>
>> On Wed, Apr 11, 2018 at 7:35 AM Abdul Patel 
>> wrote:
>>
>>> Hi All,
>>>
>>> Our company is planning for upgrading cassandra to maitain the audit
>>> gudilines for patch cycle.
>>> We are currently on 3.1.0, whats the latest stable version and what
>>> are the new features?
>>> Will it be better to wait for 4.0? Any news on what will be new
>>> features in 4.0 ?
>>>
>>

>

-- 


--

Sorl/DSE Spark

[kooljava2]

Hello,
We are exploring on configuring Sorl/Spark. Wanted to get input on this. 1) How 
do we decide which one to use?2) Do we run this on a DC where there is less 
workload?
Any other suggestion or comments are appreciated.
Thank you.

Re: JVM Tuning post

[Jonathan Haddad]

Re G1GC in Java 9, yes it's the default, but we explicitly specify the
collector when we start Cassandra.

Regarding load testing, some folks like cassandra-stress, but personally I
think second to production itself, there's nothing better than an
environment running the full applications stack with simulated load.  Yes,
it's a lot of setup, but imo it's necessary for any sort of performance /
acceptance testing.

On Wed, Apr 11, 2018 at 10:50 AM Pradeep Chhetri 
wrote:

> Thank you for writing this. The post is really very helpful.
>
> One question - My understanding is GC tuning depends a lot on the
> read/write workload and the data size. What will be the right way to
> simulate the production workload on a non-production environment in
> cassandra world.
>
> On Wed, Apr 11, 2018 at 8:54 PM, Russell Bateman 
> wrote:
>
>> Nice write-up. G1GC became the default garbage collection mechanism
>> beginning in Java 9, right?
>>
>>
>> On 04/11/2018 09:05 AM, Joao Serrachinha wrote:
>>
>> Many thanks to "The Last Pickle", also for TWCS advice's. Especially for
>> C* new features on version 3.11.1
>>
>> Regards,
>> João
>>
>> On 11/04/2018 16:00, Jon Haddad wrote:
>>
>> Hey folks,
>>
>> We (The Last Pickle) have helped a lot of teams with JVM tuning over
>> the years, finally managed to write some stuff down.  We’re hoping the
>> community finds it helpful.
>> http://thelastpickle.com/blog/2018/04/11/gc-tuning.html
>>
>> Jon
>>
>>
>>
>>
>>
>

Re: JVM Tuning post

[Pradeep Chhetri]

Thank you for writing this. The post is really very helpful.

One question - My understanding is GC tuning depends a lot on the
read/write workload and the data size. What will be the right way to
simulate the production workload on a non-production environment in
cassandra world.

On Wed, Apr 11, 2018 at 8:54 PM, Russell Bateman 
wrote:

> Nice write-up. G1GC became the default garbage collection mechanism
> beginning in Java 9, right?
>
>
> On 04/11/2018 09:05 AM, Joao Serrachinha wrote:
>
> Many thanks to "The Last Pickle", also for TWCS advice's. Especially for
> C* new features on version 3.11.1
>
> Regards,
> João
>
> On 11/04/2018 16:00, Jon Haddad wrote:
>
> Hey folks,
>
> We (The Last Pickle) have helped a lot of teams with JVM tuning over
> the years, finally managed to write some stuff down.  We’re hoping the
> community finds it helpful.
> http://thelastpickle.com/blog/2018/04/11/gc-tuning.html
>
> Jon
>
>
>
>
>

Re: Latest version and Features

[Nicolas Guyomar]

Everything is in the same document, you have a "New features" section plus
an "Upgrading" one

On 11 April 2018 at 17:24, Abdul Patel  wrote:

> Nicolas,
> I do see all new features but instructions for upgrade are mentioned in
> next section ..not sure if i missed it ..can you share that section?
>
>
> On Wednesday, April 11, 2018, Abdul Patel  wrote:
>
>> Thanks .this is perfect
>>
>> On Wednesday, April 11, 2018, Nicolas Guyomar 
>> wrote:
>>
>>> Sorry, I should have give you this link instead :
>>> https://github.com/apache/cassandra/blob/trunk/NEWS.txt
>>>
>>> You'll find everything you need IMHO
>>>
>>> On 11 April 2018 at 17:05, Abdul Patel  wrote:
>>>
 Thanks.

 Is the upgrade process staright forward do we have any documentation to
 upgrade?


 On Wednesday, April 11, 2018, Jonathan Haddad 
 wrote:

> Move to the latest 3.0, or if you're feeling a little more
> adventurous, 3.11.2.
>
> 4.0 discussion is happening now, nothing is decided.
>
> On Wed, Apr 11, 2018 at 7:35 AM Abdul Patel 
> wrote:
>
>> Hi All,
>>
>> Our company is planning for upgrading cassandra to maitain the audit
>> gudilines for patch cycle.
>> We are currently on 3.1.0, whats the latest stable version and what
>> are the new features?
>> Will it be better to wait for 4.0? Any news on what will be new
>> features in 4.0 ?
>>
>
>>>

Re: Latest version and Features

[Abdul Patel]

Nicolas,
I do see all new features but instructions for upgrade are mentioned in
next section ..not sure if i missed it ..can you share that section?

On Wednesday, April 11, 2018, Abdul Patel  wrote:

> Thanks .this is perfect
>
> On Wednesday, April 11, 2018, Nicolas Guyomar 
> wrote:
>
>> Sorry, I should have give you this link instead :
>> https://github.com/apache/cassandra/blob/trunk/NEWS.txt
>>
>> You'll find everything you need IMHO
>>
>> On 11 April 2018 at 17:05, Abdul Patel  wrote:
>>
>>> Thanks.
>>>
>>> Is the upgrade process staright forward do we have any documentation to
>>> upgrade?
>>>
>>>
>>> On Wednesday, April 11, 2018, Jonathan Haddad  wrote:
>>>
 Move to the latest 3.0, or if you're feeling a little more adventurous,
 3.11.2.

 4.0 discussion is happening now, nothing is decided.

 On Wed, Apr 11, 2018 at 7:35 AM Abdul Patel 
 wrote:

> Hi All,
>
> Our company is planning for upgrading cassandra to maitain the audit
> gudilines for patch cycle.
> We are currently on 3.1.0, whats the latest stable version and what
> are the new features?
> Will it be better to wait for 4.0? Any news on what will be new
> features in 4.0 ?
>

>>

Re: Latest version and Features

[Abdul Patel]

Thanks .this is perfect

On Wednesday, April 11, 2018, Nicolas Guyomar 
wrote:

> Sorry, I should have give you this link instead :
> https://github.com/apache/cassandra/blob/trunk/NEWS.txt
>
> You'll find everything you need IMHO
>
> On 11 April 2018 at 17:05, Abdul Patel  wrote:
>
>> Thanks.
>>
>> Is the upgrade process staright forward do we have any documentation to
>> upgrade?
>>
>>
>> On Wednesday, April 11, 2018, Jonathan Haddad  wrote:
>>
>>> Move to the latest 3.0, or if you're feeling a little more adventurous,
>>> 3.11.2.
>>>
>>> 4.0 discussion is happening now, nothing is decided.
>>>
>>> On Wed, Apr 11, 2018 at 7:35 AM Abdul Patel  wrote:
>>>
 Hi All,

 Our company is planning for upgrading cassandra to maitain the audit
 gudilines for patch cycle.
 We are currently on 3.1.0, whats the latest stable version and what are
 the new features?
 Will it be better to wait for 4.0? Any news on what will be new
 features in 4.0 ?

>>>
>

Re: Latest version and Features

[Nicolas Guyomar]

Sorry, I should have give you this link instead :
https://github.com/apache/cassandra/blob/trunk/NEWS.txt

You'll find everything you need IMHO

On 11 April 2018 at 17:05, Abdul Patel  wrote:

> Thanks.
>
> Is the upgrade process staright forward do we have any documentation to
> upgrade?
>
>
> On Wednesday, April 11, 2018, Jonathan Haddad  wrote:
>
>> Move to the latest 3.0, or if you're feeling a little more adventurous,
>> 3.11.2.
>>
>> 4.0 discussion is happening now, nothing is decided.
>>
>> On Wed, Apr 11, 2018 at 7:35 AM Abdul Patel  wrote:
>>
>>> Hi All,
>>>
>>> Our company is planning for upgrading cassandra to maitain the audit
>>> gudilines for patch cycle.
>>> We are currently on 3.1.0, whats the latest stable version and what are
>>> the new features?
>>> Will it be better to wait for 4.0? Any news on what will be new features
>>> in 4.0 ?
>>>
>>

Re: JVM Tuning post

[Russell Bateman]

Nice write-up. G1GC became the default garbage collection mechanism 
beginning in Java 9, right?



On 04/11/2018 09:05 AM, Joao Serrachinha wrote:

Many thanks to "The Last Pickle", also for TWCS advice's. Especially for
C* new features on version 3.11.1

Regards,
João

On 11/04/2018 16:00, Jon Haddad wrote:

Hey folks,

We (The Last Pickle) have helped a lot of teams with JVM tuning over
the years, finally managed to write some stuff down.  We’re hoping the
community finds it helpful.

http://thelastpickle.com/blog/2018/04/11/gc-tuning.html

Jon

Re: JVM Tuning post

[Joao Serrachinha]

Many thanks to "The Last Pickle", also for TWCS advice's. Especially for
C* new features on version 3.11.1

Regards,
João

On 11/04/2018 16:00, Jon Haddad wrote:
> Hey folks,
>
> We (The Last Pickle) have helped a lot of teams with JVM tuning over
> the years, finally managed to write some stuff down.  We’re hoping the
> community finds it helpful. 
>
> http://thelastpickle.com/blog/2018/04/11/gc-tuning.html
>
> Jon
>


-
To unsubscribe, e-mail: user-unsubscr...@cassandra.apache.org
For additional commands, e-mail: user-h...@cassandra.apache.org

Re: Latest version and Features

[Abdul Patel]

Thanks.

Is the upgrade process staright forward do we have any documentation to
upgrade?

On Wednesday, April 11, 2018, Jonathan Haddad  wrote:

> Move to the latest 3.0, or if you're feeling a little more adventurous,
> 3.11.2.
>
> 4.0 discussion is happening now, nothing is decided.
>
> On Wed, Apr 11, 2018 at 7:35 AM Abdul Patel  wrote:
>
>> Hi All,
>>
>> Our company is planning for upgrading cassandra to maitain the audit
>> gudilines for patch cycle.
>> We are currently on 3.1.0, whats the latest stable version and what are
>> the new features?
>> Will it be better to wait for 4.0? Any news on what will be new features
>> in 4.0 ?
>>
>

JVM Tuning post

[Jon Haddad]

Hey folks,

We (The Last Pickle) have helped a lot of teams with JVM tuning over the years, 
finally managed to write some stuff down.  We’re hoping the community finds it 
helpful. 

http://thelastpickle.com/blog/2018/04/11/gc-tuning.html 


Jon

Re: Latest version and Features

[Nicolas Guyomar]

Hi,

New features can be found here :
https://github.com/apache/cassandra/blob/cassandra-3.11/CHANGES.txt


On 11 April 2018 at 16:51, Jonathan Haddad  wrote:

> Move to the latest 3.0, or if you're feeling a little more adventurous,
> 3.11.2.
>
> 4.0 discussion is happening now, nothing is decided.
>
> On Wed, Apr 11, 2018 at 7:35 AM Abdul Patel  wrote:
>
>> Hi All,
>>
>> Our company is planning for upgrading cassandra to maitain the audit
>> gudilines for patch cycle.
>> We are currently on 3.1.0, whats the latest stable version and what are
>> the new features?
>> Will it be better to wait for 4.0? Any news on what will be new features
>> in 4.0 ?
>>
>

Re: Latest version and Features

[Jonathan Haddad]

Move to the latest 3.0, or if you're feeling a little more adventurous,
3.11.2.

4.0 discussion is happening now, nothing is decided.

On Wed, Apr 11, 2018 at 7:35 AM Abdul Patel  wrote:

> Hi All,
>
> Our company is planning for upgrading cassandra to maitain the audit
> gudilines for patch cycle.
> We are currently on 3.1.0, whats the latest stable version and what are
> the new features?
> Will it be better to wait for 4.0? Any news on what will be new features
> in 4.0 ?
>

Latest version and Features

[Abdul Patel]

Hi All,

Our company is planning for upgrading cassandra to maitain the audit
gudilines for patch cycle.
We are currently on 3.1.0, whats the latest stable version and what are the
new features?
Will it be better to wait for 4.0? Any news on what will be new features in
4.0 ?

Re: does c* 3.0 use one ring for all datacenters?

[Jinhua Luo]

Thanks for your reply. I also think separate rings are more reasonable.

So one ring for one dc is only for c* 1.x or 2.x without vnode?

Check these references:

https://docs.datastax.com/en/archived/cassandra/1.1/docs/initialize/token_generation.html
http://www.luketillman.com/one-token-ring-to-rule-them-all/
https://community.apigee.com/articles/13096/cassandra-token-distribution.html

Even the riak official said c* splits the ring across dc:
http://basho.com/posts/business/riak-vs-cassandra-an-updated-brief-comparison/

Why they said each dc has its own ring?


2018-04-11 19:55 GMT+08:00 Jacques-Henri Berthemet
:
> Hi,
>
> Each DC has the whole ring, each DC contains a copy of the same data. When 
> you add replication to a new DC, all data is copied to the new DC.
>
> Within a DC, each range of token is 'owned' by a (primary) node (and replicas 
> if you have RF > 1). If you add/remove a node in a DC, tokens will be 
> rearranged between all nodes within the DC only, the other DCs won't be 
> affected.
>
> --
> Jacques-Henri Berthemet
>
> -Original Message-
> From: Jinhua Luo [mailto:luajit...@gmail.com]
> Sent: Wednesday, April 11, 2018 12:35 PM
> To: user@cassandra.apache.org
> Subject: does c* 3.0 use one ring for all datacenters?
>
> Hi All,
>
> I know it seems a stupid question, but I am really confused about the 
> documents on the internet related to this topic, especially it seems that it 
> has different answers for c* with vnodes or not.
>
> Let's assume the token range is 1-100 for the whole cluster, how does it 
> distributed into the datacenters? Think that the number of datacenters is 
> dynamic in a cluster, if there is only one ring, then the token range would 
> change on each node when I add a new datacenter into the cluster? Then it 
> would involve data migration? It doesn't make sense.
>
> Looking forward to clarification for c* 3.0, thanks!
>
> -
> To unsubscribe, e-mail: user-unsubscr...@cassandra.apache.org
> For additional commands, e-mail: user-h...@cassandra.apache.org
>
>
> -
> To unsubscribe, e-mail: user-unsubscr...@cassandra.apache.org
> For additional commands, e-mail: user-h...@cassandra.apache.org

-
To unsubscribe, e-mail: user-unsubscr...@cassandra.apache.org
For additional commands, e-mail: user-h...@cassandra.apache.org

RE: does c* 3.0 use one ring for all datacenters?

[Jacques-Henri Berthemet]

Hi,

Each DC has the whole ring, each DC contains a copy of the same data. When you 
add replication to a new DC, all data is copied to the new DC.

Within a DC, each range of token is 'owned' by a (primary) node (and replicas 
if you have RF > 1). If you add/remove a node in a DC, tokens will be 
rearranged between all nodes within the DC only, the other DCs won't be 
affected.

--
Jacques-Henri Berthemet

-Original Message-
From: Jinhua Luo [mailto:luajit...@gmail.com] 
Sent: Wednesday, April 11, 2018 12:35 PM
To: user@cassandra.apache.org
Subject: does c* 3.0 use one ring for all datacenters?

Hi All,

I know it seems a stupid question, but I am really confused about the documents 
on the internet related to this topic, especially it seems that it has 
different answers for c* with vnodes or not.

Let's assume the token range is 1-100 for the whole cluster, how does it 
distributed into the datacenters? Think that the number of datacenters is 
dynamic in a cluster, if there is only one ring, then the token range would 
change on each node when I add a new datacenter into the cluster? Then it would 
involve data migration? It doesn't make sense.

Looking forward to clarification for c* 3.0, thanks!

-
To unsubscribe, e-mail: user-unsubscr...@cassandra.apache.org
For additional commands, e-mail: user-h...@cassandra.apache.org


-
To unsubscribe, e-mail: user-unsubscr...@cassandra.apache.org
For additional commands, e-mail: user-h...@cassandra.apache.org

Re: Why nodetool cleanup should be run sequentially after node joined a cluster

[Alain RODRIGUEZ]

I confirm what Christophe said.

I always ran them in parallel without any problem, really. Historically it
was using only one compactor and impact in my clusters have always been
acceptable.

Nonetheless, newer Cassandra versions allow multiple compactor to work in
parallel during cleanup and this can be really harmful - or really
efficient if resources are available and it is not impacting the read and
write operations. If all the nodes run cleanup in parallel, then limiting
the number of threads used per node is really important.

My colleague Anthony described this option here:
http://thelastpickle.com/blog/2017/08/14/limiting-nodetool-parallel-threads.html
.

C*heers,
---
Alain Rodriguez - @arodream - al...@thelastpickle.com
France / Spain

The Last Pickle - Apache Cassandra Consulting
http://www.thelastpickle.com


2018-04-11 6:04 GMT+01:00 Christophe Schmitz :

> Hi Mikhail,
>
>
> Nodetool cleanup can add a fair amount of extra load (mostly IO) on your
> Cassandra nodes. Therefore it is recommended to run it during lower cluster
> usage, and one node at a time, in order to limit the impact on your
> cluster. There are no technical limitations that would prevent you to run
> it at the same time. It's just a precaution measure.
>
> Cheers,
> Christophe
>
>
> On 11 April 2018 at 14:49, Mikhail Tsaplin  wrote:
>
>> Hi,
>> In https://docs.datastax.com/en/cassandra/3.0/cassandra/oper
>> ations/opsAddNodeToCluster.html
>> there is recommendation:
>> 6) After all new nodes are running, run nodetool cleanup
>> 
>>  on each of the previously existing nodes to remove the keys that no
>> longer belong to those nodes. Wait for cleanup to complete on one node
>> before running nodetool cleanup on the next node.
>>
>> I had added a new node to the cluster, and running nodetool cleanup
>> according to this recommendation - but it takes near 10 days to complete on
>> a single node. Is it safe to start it on all nodes?
>>
>
>
>
> --
>
> *Christophe Schmitz - **VP Consulting*
>
> AU: +61 4 03751980 / FR: +33 7 82022899
>
>    
>
>
> Read our latest technical blog posts here
> . This email has been sent on behalf
> of Instaclustr Pty. Limited (Australia) and Instaclustr Inc (USA). This
> email and any attachments may contain confidential and legally
> privileged information.  If you are not the intended recipient, do not copy
> or disclose its content, but please reply to this email immediately and
> highlight the error to the sender and then immediately delete the message.
>

Single Node Timeout Error and High Dropped Mutation after Upgradesstables

[hitesh dua]

 Hi ,


My Compression strategy in Production was *LZ4 Compression. *But I modified
it to Deflate

For compression change, we had to use *nodetool Upgradesstables *to
forcefully upgrade the compression strategy on all sstables

But once upgradesstabloes command completed on all the 5 nodes in the
cluster, My requests started to fail, both read and write

Replication Factor - 3
Read Consistency - 1
Write Consistency - 1
FYI - I am also using lightweight transaction
Cassandra Version 3.10

I am now facing Following Errors in my debug.log file and some of my
requests have started to fail :

Debug.log

ERROR [ReadRepairStage:82952] 2018-04-09 19:05:20,669
>> CassandraDaemon.java:229 - Exception in thread
>> Thread[ReadRepairStage:82952,5,main]
>
> org.apache.cassandra.exceptions.ReadTimeoutException: Operation timed out
>> - received only 0 responses.
>
> at 
> org.apache.cassandra.service.DataResolver$RepairMergeListener.close(DataResolver.java:171)
>> ~[apache-cassandra-3.10.jar:3.10]
>
> at org.apache.cassandra.db.partitions.UnfilteredPartitionIterat
>> ors$2.close(UnfilteredPartitionIterators.java:182)
>> ~[apache-cassandra-3.10.jar:3.10]
>
> at org.apache.cassandra.db.transform.BaseIterator.close(BaseIterator.java:82)
>> ~[apache-cassandra-3.10.jar:3.10]
>
> at 
> org.apache.cassandra.service.DataResolver.compareResponses(DataResolver.java:89)
>> ~[apache-cassandra-3.10.jar:3.10]
>
> at org.apache.cassandra.service.AsyncRepairCallback$1.runMayThr
>> ow(AsyncRepairCallback.java:50) ~[apache-cassandra-3.10.jar:3.10]
>
> at org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:28)
>> ~[apache-cassandra-3.10.jar:3.10]
>
> at 
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
>> ~[na:1.8.0_144]
>
> at 
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
>> ~[na:1.8.0_144]
>
> at org.apache.cassandra.concurrent.NamedThreadFactory.lambda$th
>> readLocalDeallocator$0(NamedThreadFactory.java:79)
>> ~[apache-cassandra-3.10.jar:3.10]
>
> at java.lang.Thread.run(Thread.java:748) ~[na:1.8.0_144]
>
> DEBUG [ReadRepairStage:82953] 2018-04-09 19:05:22,932
>> ReadCallback.java:242 - Digest mismatch:
>
> org.apache.cassandra.service.DigestMismatchException: Mismatch for key
>> DecoratedKey(-2666936192316364820, 5756f5b8e7b341afa22cef22c5d33260)
>> (d29a0e2a05f81315f0945dee5a210060 vs d41d8cd98f00b204e9800998ecf8427e)
>
> at 
> org.apache.cassandra.service.DigestResolver.compareResponses(DigestResolver.java:92)
>> ~[apache-cassandra-3.10.jar:3.10]
>
> at 
> org.apache.cassandra.service.ReadCallback$AsyncRepairRunner.run(ReadCallback.java:233)
>> ~[apache-cassandra-3.10.jar:3.10]
>
> at 
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
>> [na:1.8.0_144]
>
> at 
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
>> [na:1.8.0_144]
>
> at org.apache.cassandra.concurrent.NamedThreadFactory.lambda$th
>> readLocalDeallocator$0(NamedThreadFactory.java:79)
>> [apache-cassandra-3.10.jar:3.10]
>
> at java.lang.Thread.run(Thread.java:748) ~[na:1.8.0_144]
>
> INFO  [HintsDispatcher:767] 2018-04-09 19:05:24,874
>> HintsDispatchExecutor.java:283 - Finished hinted handoff of file
>> 68c7c130-6cf8-4864-bde8-1819f238045c-1523315072851-1.hints to endpoint
>> 68c7c130-6cf8-4864-bde8-1819f238045c, partially
>
> DEBUG [ReadRepairStage:82950] 2018-04-09 19:05:24,932
>> DataResolver.java:169 - Timeout while read-repairing after receiving all 1
>> data and digest responses
>
> ERROR [ReadRepairStage:82950] 2018-04-09 19:05:24,933
>> CassandraDaemon.java:229 - Exception in thread
>> Thread[ReadRepairStage:82950,5,main]
>
> org.apache.cassandra.exceptions.ReadTimeoutException: Operation timed out
>> - received only 0 responses.
>
> at 
> org.apache.cassandra.service.DataResolver$RepairMergeListener.close(DataResolver.java:171)
>> ~[apache-cassandra-3.10.jar:3.10]
>
> at org.apache.cassandra.db.partitions.UnfilteredPartitionIterat
>> ors$2.close(UnfilteredPartitionIterators.java:182)
>> ~[apache-cassandra-3.10.jar:3.10]
>
> at org.apache.cassandra.db.transform.BaseIterator.close(BaseIterator.java:82)
>> ~[apache-cassandra-3.10.jar:3.10]
>
> at 
> org.apache.cassandra.service.DataResolver.compareResponses(DataResolver.java:89)
>> ~[apache-cassandra-3.10.jar:3.10]
>
> at org.apache.cassandra.service.AsyncRepairCallback$1.runMayThr
>> ow(AsyncRepairCallback.java:50) ~[apache-cassandra-3.10.jar:3.10]
>
> at org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:28)
>> ~[apache-cassandra-3.10.jar:3.10]
>
> at 
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
>> ~[na:1.8.0_144]
>
> at 
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
>> ~[na:1.8.0_144]
>
> at org.apache.cassandra.concurrent.NamedThreadFactory.lambda$th
>> readLocalDeallocator$0(NamedThreadFactory.java:79)
>> ~[apache-cassandra-3.10.jar:3.10]
>
> at 

does c* 3.0 use one ring for all datacenters?

[Jinhua Luo]

Hi All,

I know it seems a stupid question, but I am really confused about the
documents on the internet related to this topic, especially it seems
that it has different answers for c* with vnodes or not.

Let's assume the token range is 1-100 for the whole cluster, how does
it distributed into the datacenters? Think that the number of
datacenters is dynamic in a cluster, if there is only one ring, then
the token range would change on each node when I add a new datacenter
into the cluster? Then it would involve data migration? It doesn't
make sense.

Looking forward to clarification for c* 3.0, thanks!

-
To unsubscribe, e-mail: user-unsubscr...@cassandra.apache.org
For additional commands, e-mail: user-h...@cassandra.apache.org

hinted handoff replay for a tombstoned record

[Karthick V]

Hi,
 Recently I Came across this line

Cassandra uses hinted handoff
> 
>  to replay the database mutations
>  the
> node missed while it was down. *Cassandra does not replay a mutation for
> a tombstoned record during its grace period*
>

Can any explain why we don't do so?

Ref :
https://docs.datastax.com/en/cassandra/3.0/cassandra/dml/dmlAboutDeletes.html