Re: EsRejectedExecutionException when searching date based indices.

2014-02-26 Thread Alex Clark
That is correct, I was mixing the terms nodes and shards (sorry about that). I'm running the test on a single node (machine). I've chosen 20 shards so we could eventually go to a 20 server cluster without re-indexing. It's unlikely we'll ever need to go that high but we never know and given

Re: EsRejectedExecutionException when searching date based indices.

2014-02-26 Thread joergpra...@gmail.com
I think you have a misconception about shard over-allocation and re-indexing, so you should read https://groups.google.com/d/msg/elasticsearch/49q-_AgQCp8/MRol0t9asEcJ where kimchy explains how over-allocation of shards work. If you have time-series indexes, you need not 20 shards per day, just

EsRejectedExecutionException when searching date based indices.

2014-02-25 Thread Alex Clark
Hello all, I’m getting failed nodes when running searches and I’m hoping someone can point me in the right direction. I have indices created per day to store messages. The pattern is pretty straight forward: the index for January 1 is messages_20140101, for January 2 is messages_20140102

Re: EsRejectedExecutionException when searching date based indices.

2014-02-25 Thread David Pilato
You are mixing nodes and shards, right? How many elasticsearch nodes do you have to manage your 7300 shards? Why did you set 20 shards per index? You can increase the queue size in elasticsearch.yml but I'm not sure it's the right thing to do here. My 2 cents -- David ;-) Twitter : @dadoonet /