Hello, I recently ran in to the following scenario:
Solr, version 7.5, in a docker container, running as cloud, with an external zookeeper ensemble of 3 zookeepers. Instructions were followed to make a root first, this was set correctly, as could be seen by the solr logs outputting the connect info. root command is: "bin/solr zk mkroot /solr -z <addrress>" For a yet undetermined reason, the zookeeper ensemble had some kind of split-brain occur. At a later point, Solr was restarted and then suddenly all its directories were gone. By which I mean: the directories containing the configuration and the data. The stopwords, the schema, the solr config, the "shard1_replica_n2" directories, those directories. Those were gone without a trace. As far as I can tell, solr started, asked zookeeper for its config, zookeeper returned an empty config and consequently "made it so". I am by no means very knowledgeable about solr internals. Can anyone chime in as to what happened here and how to prevent it? Is more info needed? Ideally, if something like this were to happen, I'd like for either solr to not delete folders or if that's not possible, add some kind of pre-startup check that stops solr from going any further if things go wrong. Regards, Koen
