Sounds like the inevitable "add more nodes" situation. How much RAM on each node, how big is your data set?
Regards, Mark Walkom Infrastructure Engineer Campaign Monitor email: [email protected] web: www.campaignmonitor.com On 13 May 2014 19:59, Paul <[email protected]> wrote: > We are seeing a slow down in shard initialization speed as the number of > shards/indices grows in our cluster. > > With 0-100's of indices/shards existing in the cluster a new bulk creation > of indices up the 100's at a time is fine, we see them pass through the > states and get a green cluster in a reasonable amount of time. > > As the total cluster size grows to 1000+ indices (3000+ shards) we begin > to notice that the first rounds of initialization take longer to process, > it seems to speed up after the first few batches, but this slow down leads > to "failed to process cluster event (create-index [index_1112], cause > [auto(bulk api)]) within 30s" type messages in the Master logs - the > indices are eventually created. > > > Has anyone else experienced this? (did you find the cause / way to fix?) > > Is this somewhat expected behaviour? - are we approaching something > incorrectly? (there are 3 data nodes involved, with 3 shards per index) > > -- > You received this message because you are subscribed to the Google Groups > "elasticsearch" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to [email protected]. > To view this discussion on the web visit > https://groups.google.com/d/msgid/elasticsearch/f34157df-b34e-4d69-a8bd-d8cffb2e5667%40googlegroups.com<https://groups.google.com/d/msgid/elasticsearch/f34157df-b34e-4d69-a8bd-d8cffb2e5667%40googlegroups.com?utm_medium=email&utm_source=footer> > . > For more options, visit https://groups.google.com/d/optout. > -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/CAEM624bazPsFZH5BtX8L1HCKEiP9_jz9_YLKOPhG%2BcES6DhQSQ%40mail.gmail.com. For more options, visit https://groups.google.com/d/optout.
