Re: When Apache Cassandra 4.0 will release?

[Simon Fontana Oscarsson]

Hi,
To my knowledge there is no set date for 4.0, the community is
prioritizing QA over fast release which I think is great! Anytime
during Q4 is suggested. See the latest information from the dev mailing
list: https://lists.apache.org/thread.html/1a768d057d1af5a0f373c4c399a2
3e65cb04c61bbfff612634b9437c@%3Cdev.cassandra.apache.org%3E
Maybe we will get some more information on the summit in september.
There is a plugin available for audit logging on Cassandra 3.0 and 3.11
called ecAudit: https://github.com/Ericsson/ecaudit
You can find see what compatibility it has with the audit logging for
4.0 here: https://github.com/Ericsson/ecaudit/blob/release/c3.0.11/doc/
cassandra_compatibility.md#apache-cassandra-40 

-- 
SIMON FONTANA OSCARSSON
Software Developer

Ericsson
Ölandsgatan 1
37133 Karlskrona, Sweden
simon.fontana.oscars...@ericsson.com
www.ericsson.com
On fre, 2019-07-26 at 16:01 +, bhaskar pandey wrote:
> Hi,
> 
> When Apache Cassandra 4.0 will release? Any specific time line? I am
> eagerly waiting for audit log. 
> 
> Regards,
> 
> -
> To unsubscribe, e-mail: user-unsubscr...@cassandra.apache.org
> For additional commands, e-mail: user-h...@cassandra.apache.org
> 

smime.p7s
Description: S/MIME cryptographic signature

Re: Cassandra.Link Knowledge Base - v. 0.4

[Simon Fontana Oscarsson]

Great work Rahul and others! It's much appriciated.
 
-- 
SIMON FONTANA OSCARSSON
Software Developer

Ericsson
Ölandsgatan 1
37133 Karlskrona, Sweden
simon.fontana.oscars...@ericsson.com
www.ericsson.com
On lör, 2019-07-20 at 07:43 -0400, Rahul Singh wrote:
> Hey Cassandra community ,
> 
> Thanks for all the feedback in the past on my cassandra knowledge
> base project. Without the feedback cycle it’s not really for the
> community. 
> 
> V. 0.1 - Awesome Cassandra  readme.me 
> https://anant.github.io/awesome-cassandra 
> 
> Hundreds of Cassandra articles tools etc. organized in a table of
> contents. This currently still maintains the official Cassandra.Link
> redirection. 
> 
> V. 0.2 - 600+ Organized links in Wallabag exposed as a web interface 
> http://leaves.anant.us/leaves/#!/?tag=cassandra
> 
> V. 0.3 - those 600+ Indexed with some Natural language / entity
> extraction jazz to machine taxonomize
> 
> 
> V. 0.4 - 10-15 Blog feeds + organized links as a statically generates
> site. (Search not working but will be. )  
> 
> https://cassandra.netlify.com/
> 
> This last version was made possible by a few of our team members in
> Washington DC and in Delhi : Mohd Danish (Delhi) , Tanaka Mapondera
> (DC), Rishi Nair (DC/VA) and the numerous contributions of the
> Cassandra community in the form of tools, articles, and videos. 
> 
> We’d love to get folks to check it out and send critical feedback —
> directly to me or do a pull request on the awesome-cassandra repo
> (this is just for the readme with Cassandra knowledge organized in a
> Table or Contents. ) 
> 
> Best, 
> 
> rahul.xavier.si...@gmail.com http://cassandra.link

smime.p7s
Description: S/MIME cryptographic signature

Re: How to gracefully decommission a highly loaded node?

[Simon Fontana Oscarsson]

Hi,
If it already uses 100 % CPU I have a hard time seeing it being able to
do a decomission while serving requests. If you have a lot of free
space I would first try nodetool disableautocompaction. If you don't
see any progress in nodetool netstats you can also disablebinary,
disablethrift and disablehandoff to stop serving client requests. 

-- 
SIMON FONTANA OSCARSSON
Software Developer

Ericsson
Ölandsgatan 1
37133 Karlskrona, Sweden
simon.fontana.oscars...@ericsson.com
www.ericsson.com
On tis, 2018-12-04 at 14:21 +0330, onmstester onmstester wrote:
> One node suddenly uses 100% CPU, i suspect hardware problems and do
> not have time to trace that, so decided to just remove the node from
> the cluster, but although the node state changed to UL, but no sign
> of Leaving: the node is still compacting and flushing memtables,
> writing mutations and CPU is 100% for hours since.
> Is there any means to force a Cassandra Node to just decommission and
> stop doing normal things?
> Due to W.CL=ONE, i can not use removenode and shutdown the node
> 
> Best Regards
> Sent using Zoho Mail
> 
> 

smime.p7s
Description: S/MIME cryptographic signature

Re: System auth empty, how to populate it

[Simon Fontana Oscarsson]

Very strange. CassandraRoleManager will create cassandra user if it doesn't 
already exist (look in the setupDefaultRole method).
Perhaps this is an issue with consistency. After starting Cassandra, could you 
try do a nodetool repair on system_auth keyspace on all nodes.
I'm starting to run out of ideas :(

-- 
SIMON FONTANA OSCARSSON
Software Developer

Ericsson
Ölandsgatan 1
37133 Karlskrona, Sweden
simon.fontana.oscars...@ericsson.com
www.ericsson.com

On tis, 2018-07-17 at 16:18 +0200, Thomas Lété wrote:
> Yes I did that multiple time, always following the same procedure : stop 
> Cassandra, on all nodes, remove data, update config then restart nodes one by 
> one…
> 
> I really don’t understand when I could have done wrong...
> 
> > 
> > Le 17 juil. 2018 à 16:15, Simon Fontana Oscarsson 
> >  a écrit :
> > 
> > This is very strange behavior if Cassandra won't recreate the cassandra 
> > user when you delete the folder.
> > So just to make sure, you are stopping Cassandra on all nodes and deleting 
> > the data directory?
> > 

smime.p7s
Description: S/MIME cryptographic signature

Re: System auth empty, how to populate it

[Simon Fontana Oscarsson]

This is very strange behavior if Cassandra won't recreate the cassandra user 
when you delete the folder.
So just to make sure, you are stopping Cassandra on all nodes and deleting the 
data directory?

-- 
SIMON FONTANA OSCARSSON
Software Developer

Ericsson
Ölandsgatan 1
37133 Karlskrona, Sweden
simon.fontana.oscars...@ericsson.com
www.ericsson.com

On tis, 2018-07-17 at 16:01 +0200, Thomas Lété wrote:
> It’s empty...
> 
> > 
> > Le 17 juil. 2018 à 15:59, Horia Mocioi  a écrit :
> > 
> > Could you also send the output of "select * from system_auth.roles"?
> > (you will need to change authenticator to AllowAllAuthenticator and
> > authorizer to AllowAllAuthorizer) 
> > 
> > On tis, 2018-07-17 at 15:43 +0200, Thomas Lété wrote:
> > > 
> > > Ok I tried that, nothing better (I already tried dropping the entire
> > > system_auth folder that way, same result)
> > > 
> > > When I open the log, I found nothing about « Password » and when I
> > > search for « roles », I only find that :
> > > 
> > > DEBUG [main] 2018-07-17 15:37:39,420
> > > CompactionStrategyManager.java:380 - Recreating compaction strategy -
> > > disk boundaries are out of date for system_auth.roles.
> > > DEBUG [main] 2018-07-17 15:37:39,420 DiskBoundaryManager.java:53 -
> > > Refreshing disk boundary cache for system_auth.roles
> > > DEBUG [main] 2018-07-17 15:37:39,422 DiskBoundaryManager.java:56 -
> > > Updating boundaries from
> > > DiskBoundaries{directories=[DataDirectory{location=/home/cassandra/da
> > > ta}], positions=[max(9223372036854775807)], ringVersion=3,
> > > directoriesVersion=0} to
> > > DiskBoundaries{directories=[DataDirectory{location=/home/cassandra/da
> > > ta}], positions=[max(9223372036854775807)], ringVersion=16,
> > > directoriesVersion=0} for system_auth.roles
> > > 
> > > The configuration I use for Auth is the following :
> > > 
> > > authorizer: CassandraAuthorizer
> > > permissions_validity_in_ms: 2000
> > > permissions_update_interval_in_ms: 2000
> > > authenticator: PasswordAuthenticator
> > > credentials_validity_in_ms: 2000
> > > credentials_update_interval_in_ms: 2000
> > > 
> > > > 
> > > > Le 17 juil. 2018 à 15:26, Simon Fontana Oscarsson  > > > cars...@ericsson.com> a écrit :
> > > > 
> > > > Could you try the following steps?
> > > > 
> > > > Stop Cassandra.
> > > > Change authenticator in yaml to PasswordAuthenticator if not
> > > > already done.
> > > > Remove data directory with `rm -rf data/system_auth/roles-*`
> > > > Start Cassandra.
> > > > Login with `cqlsh -u cassandra -p cassandra`
> > > > 
> > > > Works for me.
> > > > 

smime.p7s
Description: S/MIME cryptographic signature

Re: System auth empty, how to populate it

[Simon Fontana Oscarsson]

Could you try the following steps?

Stop Cassandra.
Change authenticator in yaml to PasswordAuthenticator if not already done.
Remove data directory with `rm -rf data/system_auth/roles-*`
Start Cassandra.
Login with `cqlsh -u cassandra -p cassandra`

Works for me.

-- 
SIMON FONTANA OSCARSSON
Software Developer

Ericsson
Ölandsgatan 1
37133 Karlskrona, Sweden
simon.fontana.oscars...@ericsson.com
www.ericsson.com

On tis, 2018-07-17 at 14:37 +0200, Thomas Lété wrote:
> The latest : 3.11.2, the same as yours :(
> 
> > 
> > Le 17 juil. 2018 à 14:36, Horia Mocioi  a écrit :
> > 
> > What Cassandra version do you use?
> > 
> > On tis, 2018-07-17 at 14:23 +0200, Thomas Lété wrote:
> > > 
> > > Hi,
> > > 
> > > Thanks I tried that, made a node tool repair system_auth and I get a
> > > new error now :
> > > 
> > > Connection error: ('Unable to connect to any servers', {'127.0.0.1':
> > > AuthenticationFailed('Failed to authenticate to 127.0.0.1: Error from
> > > server: code=0100 [Bad credentials] message="Provided username
> > > cassandra and/or password are incorrect"',)})
> > > 
> > > Maybe it requires other records in the other tables ?
> > > 
> > > Thanks...
> > > 
> > > > 
> > > > Le 17 juil. 2018 à 12:00, Horia Mocioi 
> > > > a écrit :
> > > > 
> > > > Try executing in cqlsh:
> > > > 
> > > > insert into system_auth.roles (role , can_login , is_superuser ,
> > > > salted_hash ) VALUES ( 'cassandra', True, True,
> > > > '$2a$10$qQIh9pXDu0JNA7vQd7KRcO7VXryjbTu8XBQTC.tXcTpJHliH6S1aW');
> > > > 
> > > > The above hash encrypted value is for the default password
> > > > cassandra
> > > > 
> > > > After doing this you should enable PasswordAuthenticator and
> > > > restart
> > > > your node. Then try to authenticate in cqlsh.
> > > > 
> > > > I tested this on ccm with cassandra 3.11.2. You should try it also
> > > > on
> > > > your test systems and not trust a random guy on the web.
> > > > 
> > > > Horia
> > > > 
> > > > On tis, 2018-07-17 at 11:40 +0200, Thomas Lété wrote:
> > > > > 
> > > > > Hi Horia,
> > > > > 
> > > > > Thanks for your reply :-)
> > > > > As the keyspace was corrupt, Cassandra didn’t boot… So as we were
> > > > > in
> > > > > a hurry and not needed that keyspace, we dropped the files and
> > > > > put
> > > > > AllowAllAuthorizer.
> > > > > 
> > > > > Now even the roles table is empty.
> > > > > 
> > > > > When I enable PasswordAuthenticator, I’m not able to
> > > > > authenticate,
> > > > > cassandra user doesn’t exist…
> > > > > 
> > > > > Is there a way to insert the default user in the roles table ?
> > > > > 
> > > > > Yes the folder were recreated but without any data in the tables.
> > > > > I didn’t see any error in the logs.
> > > > > 
> > > > > 
> > > > > > 
> > > > > > 
> > > > > > Le 17 juil. 2018 à 11:26, Horia Mocioi  > > > > > om>
> > > > > > a écrit :
> > > > > > 
> > > > > > Hello,
> > > > > > 
> > > > > > Those tables are empty by default, except system_auth.roles
> > > > > > table
> > > > > > which
> > > > > > contains one entry(by default): the cassandra user/role.
> > > > > > CassandraRoleManager creates it on startup if it does not
> > > > > > exist.
> > > > > > 
> > > > > > Those tables will be populated once you start adding new
> > > > > > users/roles
> > > > > > and use authorization.
> > > > > > 
> > > > > > What do you mean by "I tried removing the system_auth folder
> > > > > > from
> > > > > > data
> > > > > > but the keyspace isn’t removed. "? Is the folder deleted? Did
> > > > > > you
> > > > > > restart and the folder was recreated or not? Did you get errors
> > > > 

Re: Cassandra recommended server uptime?

[Simon Fontana Oscarsson]

Not anything that I'm aware of. Cassandra can run at months/years without 
rebooting.
It is better to monitor your nodes and if you find anything abnormal a restart 
can help.

-- 
SIMON FONTANA OSCARSSON
Software Developer

Ericsson
Ölandsgatan 1
37133 Karlskrona, Sweden
simon.fontana.oscars...@ericsson.com
www.ericsson.com

On tis, 2018-07-17 at 12:09 +0300, Vsevolod Filaretov wrote:
> Good time of day everyone;
> 
> Does Cassandra have a "recommended uptime"? I.e., does regular Cassandra node 
> reboots help anything? Are periodical node reboots recommended for general 
> system stability?
> 
> Best regards, Vsevolod.

smime.p7s
Description: S/MIME cryptographic signature

Re: Write Time of a Row in Multi DC Cassandra Cluster

[Simon Fontana Oscarsson]

I thought you just wanted to test how big delay you have, that's why I 
suggested trace.

Your best option is to write with EACH_QUORUM as Alain said.
That way you will get a response when the write is successful on all dcs.
The downside is that the request will fail if one dc is down.
As usual it's up to you to decide if that's acceptable or not.
You could follow up with a LOCAL_QUORUM request if the EACH_QUORUM fails and 
rely on hints and repair to get consistency eventually.

-- 
SIMON FONTANA OSCARSSON
Software Developer

Ericsson
Ölandsgatan 1
37133 Karlskrona, Sweden
simon.fontana.oscars...@ericsson.com
www.ericsson.com

On tis, 2018-07-10 at 16:32 +, Saladi Naidu wrote:
> Simon,
> Trace would be significant burden on the cluster and it has to be on all the 
> time. I am trying to find a way to know when a row is written on demand 
> basis, is there a way to determine that?
>  
> Naidu Saladi 
> 
> 
> On Tuesday, July 10, 2018 2:24 AM, Simon Fontana Oscarsson 
>  wrote:
> 
> 
> Have you tried trace?
> -- 
> SIMON FONTANA OSCARSSON
> Software Developer
> 
> Ericsson
> Ölandsgatan 1
> 37133 Karlskrona, Sweden
> simon.fontana.oscars...@ericsson.com
> www.ericsson.com
> 
> On mån, 2018-07-09 at 19:30 +, Saladi Naidu wrote:
> > Cassandra is an eventual consistent DB, how to find when a row is actually 
> > written in multi DC environment? Here is the problem I am trying to solve 
> > 
> > - I have multi DC (3 DC's) Cassandra cluster/ring - One of the application 
> > wrote a row to DC1(using Local Quorum)  and within span of 50 ms, it tried 
> > to read same row from DC2 and could not find
> the
> > row. Our both DC's have sub milli second latency at network level, usually 
> > <2 ms. We promised 20 ms consistency. In this case Application could not 
> > find the row in DC2 in 50 ms
> > 
> > I tried to use "select WRITETIME(authorizations_json) from 
> > token_authorizations where " to find  when the Row is written in each 
> > DC, but both DC's returned same Timestamp. After further
> research
> > I found that Client V3 onwards Timestamp is supplied at Client level so 
> > WRITETIME does not help 
> > "https://docs.datastax.com/en/developer/java-driver/3.4/manual/query_timestamps/;
> > 
> > So how to determine when the row is actually written in each DC?
> > 
> >  
> > Naidu Saladi 
> 
> 

smime.p7s
Description: S/MIME cryptographic signature

Re: Write Time of a Row in Multi DC Cassandra Cluster

[Simon Fontana Oscarsson]

Have you tried trace?
-- 
SIMON FONTANA OSCARSSON
Software Developer

Ericsson
Ölandsgatan 1
37133 Karlskrona, Sweden
simon.fontana.oscars...@ericsson.com
www.ericsson.com

On mån, 2018-07-09 at 19:30 +, Saladi Naidu wrote:
> Cassandra is an eventual consistent DB, how to find when a row is actually 
> written in multi DC environment? Here is the problem I am trying to solve 
> 
> - I have multi DC (3 DC's) Cassandra cluster/ring - One of the application 
> wrote a row to DC1(using Local Quorum)  and within span of 50 ms, it tried to 
> read same row from DC2 and could not find the
> row. Our both DC's have sub milli second latency at network level, usually <2 
> ms. We promised 20 ms consistency. In this case Application could not find 
> the row in DC2 in 50 ms
> 
> I tried to use "select WRITETIME(authorizations_json) from 
> token_authorizations where " to find  when the Row is written in each DC, 
> but both DC's returned same Timestamp. After further research
> I found that Client V3 onwards Timestamp is supplied at Client level so 
> WRITETIME does not help 
> "https://docs.datastax.com/en/developer/java-driver/3.4/manual/query_timestamps/;
> 
> So how to determine when the row is actually written in each DC?
> 
>  
> Naidu Saladi 

smime.p7s
Description: S/MIME cryptographic signature

Re: Check Cluster Health

[Simon Fontana Oscarsson]

Running nodetool status is okay if you want the simplest solution.
But it generates a lot of output and creates a new JMX connection for every 
execution.
Cassandra uses JMX to expose metrics via mbeans.
Read this to get a first understanding: 
https://docs.datastax.com/en/cassandra/2.1/cassandra/operations/ops_monitoring_c.html
Use Jconsole to explore the different metrics. Use documentation as reference: 
https://cassandra.apache.org/doc/latest/operating/metrics.html

As for your solution I recommend one of the following:
* Create a simple JMX client and add your beans. You can do some simple logging 
with Logback or log4j. You can get some help by googling.
* Use a monitoring system such as Prometheus. This is the best solution but 
most time consuming.

-- 
SIMON FONTANA OSCARSSON
Software Developer

Ericsson
Ölandsgatan 1
37133 Karlskrona, Sweden
simon.fontana.oscars...@ericsson.com
www.ericsson.com

On fre, 2018-07-06 at 11:18 +0100, Thouraya TH wrote:
> Hi,
> Thank you so much for answers.
> 
> Please, can you explain more what's metric libraries ? and give me some 
> examples ?
> 
> Using nodetool status, to generate the history of my data center, i intend to 
> proceed as follows:
> 
> From a node A: 
> 
> For i  1 ..24 hours  (every 2 minutes do)
> 
> ./nodetool status >> file.txt
> 
> End For
> 
> 
> is it a good idea?
> 
> Thanks a lot.
> Kind regards. 
> 
> 2018-07-05 1:30 GMT+01:00 Anthony Grasso :
> > Hi,
> > 
> > Yes, you can use nodetool status to inspect the health/status of the 
> > cluster. Using nodetool status  will show the cluster 
> > health/status as well as the amount of data that each node has
> > for the specified .  Using nodetool status without the  
> > argument will only show the cluster health/status.
> > 
> > Unless there is a special reason for using nodetool to capture history, you 
> > may want to consider using metric libraries to capture and push information 
> > about each node to a metric server. It is
> > much easier to view the data captured on the metric server as there are 
> > tools already made for this. Using metrics libraries will save you time 
> > creating and maintaining a parser for the nodetool
> > output. It also makes monitoring the health of cluster very easy.
> > 
> > Regards,
> > Anthony
> > 
> > On Sun, 1 Jul 2018 at 20:19, Thouraya TH  wrote:
> > > Hi,
> > > Thank you so much for answer.
> > > Please, is it possible to use this command ?
> > > nodetool status mykeyspace
> > > 
> > > Datacenter: datacenter1
> > > ===
> > > Status=Up/Down
> > > |/ State=Normal/Leaving/Joining/Moving
> > > --  AddressLoad   Tokens  OwnsHost ID 
> > >   Rack
> > > UN  127.0.0.1  47.66 KB   1   33.3%   
> > > aaa1b7c1-6049-4a08-ad3e-3697a0e30e10  rack1
> > > UN  127.0.0.2  47.67 KB   1   33.3%   
> > > 1848c369-4306-4874-afdf-5c1e95b8732e  rack1
> > > UN
> > > Thank you so much.
> > > Kind regards. 
> > > 
> > > 2018-06-29 1:40 GMT+01:00 Rahul Singh :
> > > > 
> > > > 
> > > > When you run TPstats or Tablestats subcommands in nodetool you are 
> > > > actually accessing data inside Cassandra via JMX.
> > > > 
> > > > You can start there at first.
> > > > 
> > > > Rahul
> > > > On Jun 28, 2018, 10:55 AM -0500, Thouraya TH , 
> > > > wrote:
> > > > > Hi,
> > > > > 
> > > > > Please, how can check the health of my cluster / data center using 
> > > > > cassandra ?
> > > > > In fact i'd like to generate a hitory of the state of each node. an 
> > > > > history about the failure of my cluster ( 20% of failure in a day, 
> > > > > 40% of failure in a day etc...)
> > > > > 
> > > > > Thank you so much.
> > > > > Kind regards.
> > > 

smime.p7s
Description: S/MIME cryptographic signature

Re: nodetool status

[Simon Fontana Oscarsson]

Look here: https://cassandra.apache.org/doc/latest/development/ide.html

-- 
SIMON FONTANA OSCARSSON
Software Developer

Ericsson
Ölandsgatan 1
37133 Karlskrona, Sweden
simon.fontana.oscars...@ericsson.com
www.ericsson.com

On ons, 2018-07-04 at 10:10 +0100, Thouraya TH wrote:
> Thank you so much.
> I want to modify some lines in the source code of nodetool status.
> Is it possible ? If it's, how run recompile sources to have the new version 
> of cassandra ?
> 
> Kind regards. 
> 
> 2018-07-03 17:32 GMT+01:00 Joshua Galbraith :
> > https://github.com/apache/cassandra/blob/cassandra-3.11/src/java/org/apache/cassandra/tools/nodetool/Status.java
> > 
> > On Tue, Jul 3, 2018 at 7:57 AM, Thouraya TH  wrote:
> > > Hi all,
> > > Please, can you give me a link to the source code behind the command 
> > > "nodetool status" ?
> > > Thank you so much.
> > > KInd regards. 
> > > 
> > 
> > 
> > -- 
> > Joshua Galbraith | Lead Software Engineer | New Relic
> > 

smime.p7s
Description: S/MIME cryptographic signature

Re: JMX metrics for CL

[Simon Fontana Oscarsson]

It's not a change in the source code though since it's a plugin. You 
simply do the following:


 * Implement the QueryHandler interface with new metrics.
 * Add the compiled jar to the CLASSPATH in cassandra.in.sh
 * In cassandra-env.sh, set -Dcassandra.custom_query_handler_class to
   your custom class, e.g. JVM_OPTS="$JVM_OPTS
   -Dcassandra.custom_query_handler_class=example.com.MyQueryHandler.

BR,
Simon

On 2018-02-28 11:21, Kyrylo Lebediev wrote:


Thanks, Horia!

Wouldn't like to introduce any changes in source code.


Any alternatives how to trace CL's used from C* side?  If not 'since 
startup', then at least online metrics will be fine.



Regards,

Kyrill


*From:* Horia Mocioi 
*Sent:* Tuesday, February 27, 2018 7:38:23 PM
*To:* user@cassandra.apache.org
*Subject:* Re: JMX metrics for CL
Hello,

There are no such metrics that I am aware of.

One way you could do this is to have your own implementation of 
QueryHandler and your own metrics and filter the queries based on the 
CL and increment the according metric.


Then, in cassandra-env.sh you could specify to use your class using 
 -Dcassandra.custom_query_handler_class.


HTH,
Horia
On tis, 2018-02-27 at 08:26 +, Kyrylo Lebediev wrote:


Hello!


Is it possible to get counters  from C* side regarding CQL queries 
executed since startup for each CL?

For example:
CL ONE: NNN queries
CL QUORUM: MMM queries
etc

Regards,

Kyrill

Re: Best approach to prepare to shutdown a cassandra node

[Simon Fontana Oscarsson]

Yes, drain will always be run when Cassandra exits normally.

On 2017-10-20 00:57, Varun Gupta wrote:
Does, nodetool stopdaemon, implicitly drain too? or we should invoke 
drain and then stopdaemon?


On Mon, Oct 16, 2017 at 4:54 AM, Simon Fontana Oscarsson 
<simon.fontana.oscars...@ericsson.com 
<mailto:simon.fontana.oscars...@ericsson.com>> wrote:


Looking at the code in trunk, the stopdemon command invokes the
CassandraDaemon.stop() function which does a graceful shutdown by
stopping jmxServer and drains the node by the shutdown hook.

/Simon


On 2017-10-13 20:42, Javier Canillas wrote:

As far as I know, the nodetool stopdaemon is doing a "kill -9".

Or did it change?

2017-10-12 23:49 GMT-03:00 Anshu Vajpayee
<anshu.vajpa...@gmail.com <mailto:anshu.vajpa...@gmail.com>>:

Why are you killing when we have nodetool stopdaemon ?

On Fri, Oct 13, 2017 at 1:49 AM, Javier Canillas
<javier.canil...@gmail.com
<mailto:javier.canil...@gmail.com>> wrote:

That's what I thought.

Thanks!

2017-10-12 14:26 GMT-03:00 Hannu Kröger
<hkro...@gmail.com <mailto:hkro...@gmail.com>>:

Hi,

Drain should be enough.  It stops accepting writes
and after that cassandra can be safely shut down.

Hannu

On 12 October 2017 at 20:24:41, Javier Canillas
(javier.canil...@gmail.com
<mailto:javier.canil...@gmail.com>) wrote:


Hello everyone,

I have some time working with Cassandra, but every
time I need to shutdown a node (for any reason like
upgrading version or moving instance to another
host) I see several errors on the client
applications (yes, I'm using the official java driver).

By the way, I'm starting C* as a stand-alone process

<https://docs.datastax.com/en/cassandra/3.0/cassandra/initialize/referenceStartCprocess.html?hl=start>,
and C* version is 3.11.0.

The way I have implemented the shutdown process is
something like the following:

/# Drain all information from commitlog into sstables/
/bin/nodetool drain
/
/
/
/cassandra_pid=`ps -ef|grep
"java.*apache-cassandra"|grep -v "grep"|awk '{print
$2}'`
/
/if [ ! -z "$cassandra_pid" ] && [ "$cassandra_pid"
-ne "1" ]; then/
/  echo "Asking Cassandra to shutdown (nodetool
drain doesn't stop cassandra)"/
/  kill $cassandra_pid/
/
/
/  echo -n "+ Checking it is down. "/
/  counter=10/
/  while [ "$counter" -ne 0 -a ! kill -0
$cassandra_pid > /dev/null 2>&1 ]/
/  do/
/          echo -n ". "/
/((counter--))/
/sleep 1s/
/  done/
/  echo ""/
/  if ! kill -0 $cassandra_pid > /dev/null 2>&1; then/
/          echo "+ Its down."/
/  else/
/          echo "- Killing Cassandra."/
/          kill -9 $cassandra_pid/
/  fi/
/else/
/  echo "Care there was a problem finding Cassandra
PID"/
/fi/
/
/
Should I add at the beginning the following lines?

echo "shutdowing cassandra gracefully with: nodetool
disable gossip"
$CASSANDRA_HOME/$CASSANDRA_APP/bin/nodetool
disablegossip
echo "shutdowing cassandra gracefully with: nodetool
disable binary protocol"
$CASSANDRA_HOME/$CASSANDRA_APP/bin/nodetool
disablebinary
echo "shutdowing cassandra gracefully with: nodetool
thrift"
$CASSANDRA_HOME/$CASSANDRA_APP/bin/nodetool
disablethrift

The shutdown log is the following:

/WARN [RMI TCP Connection(10)-127.0.0.1] 2017-10-12
14:20:52,343 StorageService.java:321 - Stopping
gossip by operator request/
/INFO [RMI TCP Connection(10)-127.0.0.1] 2017-10-12
14:20:52,344 Gossiper.java:1532 - Announcing shutdown/
/INFO [RMI TCP Connection(10)-127.0.0.

Re: Best approach to prepare to shutdown a cassandra node

[Simon Fontana Oscarsson]

Looking at the code in trunk, the stopdemon command invokes the 
CassandraDaemon.stop() function which does a graceful shutdown by 
stopping jmxServer and drains the node by the shutdown hook.


/Simon

On 2017-10-13 20:42, Javier Canillas wrote:

As far as I know, the nodetool stopdaemon is doing a "kill -9".

Or did it change?

2017-10-12 23:49 GMT-03:00 Anshu Vajpayee >:


Why are you killing when we have nodetool stopdaemon ?

On Fri, Oct 13, 2017 at 1:49 AM, Javier Canillas
> wrote:

That's what I thought.

Thanks!

2017-10-12 14:26 GMT-03:00 Hannu Kröger >:

Hi,

Drain should be enough.  It stops accepting writes and
after that cassandra can be safely shut down.

Hannu

On 12 October 2017 at 20:24:41, Javier Canillas
(javier.canil...@gmail.com
) wrote:


Hello everyone,

I have some time working with Cassandra, but every time I
need to shutdown a node (for any reason like upgrading
version or moving instance to another host) I see several
errors on the client applications (yes, I'm using the
official java driver).

By the way, I'm starting C* as a stand-alone process

,
and C* version is 3.11.0.

The way I have implemented the shutdown process is
something like the following:

/# Drain all information from commitlog into sstables/
/bin/nodetool drain
/
/
/
/cassandra_pid=`ps -ef|grep "java.*apache-cassandra"|grep
-v "grep"|awk '{print $2}'`
/
/if [ ! -z "$cassandra_pid" ] && [ "$cassandra_pid" -ne
"1" ]; then/
/        echo "Asking Cassandra to shutdown (nodetool
drain doesn't stop cassandra)"/
/        kill $cassandra_pid/
/
/
/        echo -n "+ Checking it is down. "/
/counter=10/
/        while [ "$counter" -ne 0 -a ! kill -0
$cassandra_pid > /dev/null 2>&1 ]/
/        do/
/echo -n ". "/
/((counter--))/
/sleep 1s/
/        done/
/        echo ""/
/        if ! kill -0 $cassandra_pid > /dev/null 2>&1; then/
/echo "+ Its down."/
/        else/
/echo "- Killing Cassandra."/
/kill -9 $cassandra_pid/
/        fi/
/else/
/        echo "Care there was a problem finding Cassandra
PID"/
/fi/
/
/
Should I add at the beginning the following lines?

echo "shutdowing cassandra gracefully with: nodetool
disable gossip"
$CASSANDRA_HOME/$CASSANDRA_APP/bin/nodetool disablegossip
echo "shutdowing cassandra gracefully with: nodetool
disable binary protocol"
$CASSANDRA_HOME/$CASSANDRA_APP/bin/nodetool disablebinary
echo "shutdowing cassandra gracefully with: nodetool thrift"
$CASSANDRA_HOME/$CASSANDRA_APP/bin/nodetool disablethrift

The shutdown log is the following:

/WARN  [RMI TCP Connection(10)-127.0.0.1] 2017-10-12
14:20:52,343 StorageService.java:321 - Stopping gossip by
operator request/
/INFO  [RMI TCP Connection(10)-127.0.0.1] 2017-10-12
14:20:52,344 Gossiper.java:1532 - Announcing shutdown/
/INFO  [RMI TCP Connection(10)-127.0.0.1] 2017-10-12
14:20:52,355 StorageService.java:2268 - Node
/10.254.169.36  state jump to shutdown/
/INFO  [RMI TCP Connection(12)-127.0.0.1] 2017-10-12
14:20:56,141 Server.java:176 - Stop listening for CQL
clients/
/INFO  [RMI TCP Connection(16)-127.0.0.1] 2017-10-12
14:20:59,472 StorageService.java:1442 - DRAINING:
starting drain process/
/INFO  [RMI TCP Connection(16)-127.0.0.1] 2017-10-12
14:20:59,474 HintsService.java:220 - Paused hints dispatch/
/INFO  [RMI TCP Connection(16)-127.0.0.1] 2017-10-12
14:20:59,477 Gossiper.java:1532 - Announcing shutdown/
/INFO  [RMI TCP Connection(16)-127.0.0.1] 2017-10-12
14:20:59,480 StorageService.java:2268 - Node /127.0.0.1
 state jump to shutdown/
/INFO  [RMI TCP Connection(16)-127.0.0.1] 2017-10-12
14:21:01,483