recovering from failed repair , cassandra 3.10

2017-05-31 Thread Micha
Hi, after failed repair on a three node cluster all nodes were down. It cannot start, since it finds a mismatch in a mc_txn_anticompactionafterrepair log file: "got ADD " "expected "ADD:..." The two log files are different: one has "ADD, ADD; REMOVE, REMOVE, COMMIT" the other is missing an

Convert single node C* to cluster (rebalancing problem)

2017-05-31 Thread Junaid Nasir
Cassandra ensure that adding or removing nodes are very easy and that load is balanced between nodes when a change is made. but it's not working in my case. I have a single node C* deployment (with 270 GB of data) and want to load balance the data on multiple nodes, I followed this guide

Re: recovering from failed repair , cassandra 3.10

2017-05-31 Thread Oleksandr Shulgin
On Wed, May 31, 2017 at 9:11 AM, Micha wrote: > Hi, > > after failed repair on a three node cluster all nodes were down. > To clarify, was it failed repair that brought the nodes down so that you had to start them back? Do you see any error messages or stack trace in

Re: Convert single node C* to cluster (rebalancing problem)

2017-05-31 Thread Junaid Nasir
after nodetool repair on new node following log appears in cassandra log INFO [StreamConnectionEstablisher:1] 2017-05-31 10:07:23,496 > StreamSession.java:266 - [Stream #f0eb9670-45e8-11e7-a17b-81e9a18c6eac] > Starting streaming to /10.128.1.2 > INFO [StreamConnectionEstablisher:1] 2017-05-31

Re: Convert single node C* to cluster (rebalancing problem)

2017-05-31 Thread Oleksandr Shulgin
On Wed, May 31, 2017 at 9:55 AM, Junaid Nasir wrote: > Cassandra ensure that adding or removing nodes are very easy and that load > is balanced between nodes when a change is made. but it's not working in my > case. > I have a single node C* deployment (with 270 GB of data) and

Re: Convert single node C* to cluster (rebalancing problem)

2017-05-31 Thread Junaid Nasir
Cassandra version is 3.10, and yes its not a production server. i have seen some warnings in logs saying token exist on both servers. other than that nothing. if you need any more settings/details please ask. thank you for your time On Wed, May 31, 2017 at 2:04 PM, Oleksandr Shulgin <

Re: recovering from failed repair , cassandra 3.10

2017-05-31 Thread Micha
The error which keeps it from starting is below. The files like "mc_txn_anticompactionafterrepair_19a46410-459f-11e7-91c7-4f4e8666b5c8.log" are on both disks of a node but are different . Of course, just renaming (deleting) the two files (or making them equal) makes cassandra start again. But I

Re: Netty SSL memory leak

2017-05-31 Thread Tom van der Woerdt
Hi John, That's the bug I filed the ticket for, yup. I recommend updating to a newer Cassandra version (3.0.11 or newer), which fixes this issue (and many others). Tom On Wed, May 31, 2017 at 12:39 AM, John Sanda wrote: > I have Cassandra 3.0.9 cluster that is hitting

Re: Stable version apache cassandra 3.X /3.0.X

2017-05-31 Thread Jonathan Haddad
I really wouldn't go by the tick tock blog post, considering tick tock is dead. I'm still not wild about putting any 3.0 or 3.x into production. 3.0 removed off heap memtables and there have been enough bugs in the storage engine that I'm still wary. My hope is to see 3.11.x get enough bug

Stable version apache cassandra 3.X /3.0.X

2017-05-31 Thread pabbireddy avinash
Hi, We are planning to deploy a cassandra production cluster on 3.X /3.0.X . Please let us know if there is any stable version in 3.X/3.0.X that we could deploy in production . Regards, Avinash.

Re: Stable version apache cassandra 3.X /3.0.X

2017-05-31 Thread Junaid Nasir
as mentioned here http://www.datastax.com/dev/blog/cassandra-2-2-3-0-and-beyond > Under normal conditions, we will NOT release 3.x.y stability releases for > x > 0. That is, we will have a traditional 3.0.y stability series, but the > odd-numbered bugfix-only releases will fill that role for the

Re: Convert single node C* to cluster (rebalancing problem)

2017-05-31 Thread Akhil Mehra
Hi Junaid, I noticed in the log files that data is being streamed from/to 10.128.1.2 but the address of your original node is 10.128.0.7. Are there any other Cassandra nodes on your local network the same cluster name. Regards, Akhil > 10.128.1.2 > On 31/05/2017, at 10:15 PM, Junaid Nasir

Re: Restarting nodes and reported load

2017-05-31 Thread Anthony Grasso
Hi Daniel, When you say that the nodes have to be restarted, are you just restarting the Cassandra service or are you restarting the machine? How are you reclaiming disk space at the moment? Does disk space free up after the restart? Regarding storage on nodes, keep in mind the more data stored

Re: Convert single node C* to cluster (rebalancing problem)

2017-05-31 Thread Junaid Nasir
sorry that was an old notetool status output (old cluster with a few configuration changes). new configuration is -- Address Load Tokens Owns (effective) Host ID Rack UN 10.128.1.2 270.75 GiB 256 52.6% e4427611-c247-42ee-9404-371e177f5f17

Re: Stable version apache cassandra 3.X /3.0.X

2017-05-31 Thread Carlos Rolo
On sync in Jon. Only go 3.0.x if you REALLY need something from there (ex: MV) even then, be carefull. 3.x wait for 3.11.x. 3.10 if you REALLY need something from there right now. Latest 2.2.x or 2.1.x if you are just doing baseline Cassandra and need the stability. Regards, Carlos Juzarte

RE: Stable version apache cassandra 3.X /3.0.X

2017-05-31 Thread Mark Furlong
I need to reduce my disk footprint, how is 3.0.14 for stability? Also, where do I find upgrade instructions and version requirements? Thanks Mark 801-705-7115 office From: Carlos Rolo [mailto:r...@pythian.com] Sent: Wednesday, May 31, 2017 11:17 AM To: Jonathan Haddad Cc:

Re: How to avoid flush if the data can fit into memtable

2017-05-31 Thread Kevin O'Connor
Great post Akhil! Thanks for explaining that. On Mon, May 29, 2017 at 5:43 PM, Akhil Mehra wrote: > Hi Preetika, > > After thinking about your scenario I believe your small SSTable size might > be due to data compression. By default, all tables enable SSTable >