I did an optimize on this index and it looks like it caused a shard to become corrupted. Or maybe the optimize just brought the shard corruption to light?
On the node that reported the corrupted shard I tried shutting it down, moving the shard out and then restarting. Unfortunately the next node that got that shard then started with the same corruption issues. The errors: Mar 24 01:40:17 localhost elasticsearch: [bma.0][WARN ][indices.cluster ] [Meteorite II] [1-2013][0] failed to start shard Mar 24 01:40:17 localhost org.elasticsearch.index.gateway.IndexShardGatewayRecoveryException: [1-2013][0] failed to fetch index version after copying it over Mar 24 01:40:17 localhost elasticsearch: [bma.0][WARN ][cluster.action.shard ] [Meteorite II] [1-2013][0] sending failed shard for [1-2013][0], node[ZzXsIZCsTyWD2emFuU0idg], [P], s[INITIALIZING], indexUUID [_na_], reason [Failed to start shard, message [IndexShardGatewayRecoveryException[[1-2013][0] failed to fetch index version after copying it over]; nested: CorruptIndexException[[1-2013][0] Corrupted index [corrupted_OahNymObSTyBzCCPu1FuJA] caused by: CorruptIndexException[docs out of order (1493829 <= 1493874 ) (docOut: org.apache.lucene.store.RateLimitedIndexOutput@2901a3e1)]]; ]] I tried using CheckIndex, but had this issue: java.lang.IllegalArgumentException: A SPI class of type org.apache.lucene.codecs.PostingsFormat with name 'es090' does not exist. You need to add the corresponding JAR file supporting this SPI to your classpath.The current classpath supports the following names: [Pulsing41, SimpleText, Memory, BloomFilter, Direct, FSTPulsing41, FSTOrdPulsing41, FST41, FSTOrd41, Lucene40, Lucene41] When running with: java -cp /usr/share/elasticsearch/lib/lucene-codecs-4.9.1.jar:/usr/share/elasticsearch/lib/lucene-core-4.9.1.jar -ea:org.apache.lucene... org.apache.lucene.index.CheckIndex I'm not a java programmer so after I tried other classpath combinations I was out of ideas. Any tips? Looking at _cat/shards the replica is currently marked "unassigned" while the primary is "initializing". Thanks! -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/31fa3d97-02fa-4d1c-b507-d413051f2ea3%40googlegroups.com. For more options, visit https://groups.google.com/d/optout.
