Hey Guan, If it's the case that you actually replaced the nodes and already assigned new IP's to them this is probably common behaviour. Since Cassandra has a retention policy to keep dead nodes for 72 hours.
Best, Malte 2017-01-17 0:29 GMT+01:00 Sun, Guan <guan...@amazon.com>: > Hi all, > > We have a 6 node Cassandra cluster running on Amazon EC2. We replaced the > nodes one by one while Cassandra was running several days ago. After the > replacement, the nodetool status show that all the nodes were up and > running. However, the JMX metrics about down nodes kept showing there were > 6 nodes down for about 2.5 days before it became 0. > > Has anyone seen this before? Any idea why the JMX metrics takes so long to > reflect the actual status of the cluster? > > Thanks, > Guan > -- Malte Pickhan Software Engineer Zalando Payments SE & Co. KG *POSTAL ADDRESS* Zalando Payments SE & Co. KG 11501 Berlin *OFFICE* Zalando Payments SE & Co. KG Hörder Hafenstraße 1-3 44263 Dortmund Germany E-Mail: malte.pick...@zalando.de Web: *corporate.zalando.de <http://corporate.zalando.de/>* Jobs: jobs.zalando.de Zalando Payments SE & Co. KG Company registration: Amtsgericht Charlottenburg, HRA 48907 B VAT registration number: DE 289683421 General partner: Zalando SE, registered office: Berlin Company registration: Amtsgericht Charlottenburg, HRA 48907 B Management Board: Robert Gentz, David Schneider, Rubin Ritter Chairperson of Supervisory: Lothar Lanz
