Thanks Alex. We have 6 nodes in each DC with RF=3 with CL local qourum . and we stopped and started only one instance at a time . Tough nodetool status says all nodes UN and system.log says canssandra started and started listening . Jmx explrter shows instance stayed down longer how do we determine what caused the Cassandra unavialbe though log says its stared and listening ?
On Tue, Oct 29, 2019, 4:44 PM Oleksandr Shulgin < [email protected]> wrote: > On Tue, Oct 29, 2019 at 9:34 PM Rahul Reddy <[email protected]> > wrote: > >> >> We have our infrastructure on aws and we use ebs storage . And aws was >> retiring on of the node. Since our storage was persistent we did nodetool >> drain and stopped and start the instance . This caused 500 errors in the >> service. We have local_quorum and rf=3 why does stopping one instance cause >> application to have issues? >> > > Can you still look up what was the underlying error from Cassandra driver > in the application logs? Was it request timeout or not enough replicas? > > For example, if you only had 3 Cassandra nodes, restarting one of them > reduces your cluster capacity by 33% temporarily. > > Cheers, > -- > Alex > >
