I have a 4 node ES cluster. I am adding new docs at a rate of about 3k/sec. After an hour my cluster status turns to yellow. I see that node 2 is no longer part of the cluster when I curl the "_cluster/state" endpoint. Hitting the "_cluster/health" endpoint reveals that I have 872 unassigned shards.
I login to node 2 and find ES running. I want to restart ES on node 2 to see if it will rejoin my cluster. I issue the command: "curl -XPOST 'http://localhost:9200/_cluster/nodes/_local/_shutdown' " and get back curl: (56) Failure when receiving data from the peer. I try this several more times and ES will still not shut down on node 2. I believe at this point if I kill -9 the process the cluster will move from "yellow" to "red". How do I gracefully recover from this situation? Is it possible? Thanks, Brad -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/3e8c18e8-6f7c-48b7-a7a4-ed0c2a072871%40googlegroups.com. For more options, visit https://groups.google.com/groups/opt_out.
