Thanks Alex & Jorg. > can you explain what kind of problems you have? The problem is getting EsRejectedExecutionException during bulk indexing. This error started appearing after our move to 0.90.9. We wanted to save on memory so have set bloom filter settings as false initially. In fact we migrated to 0.90.9 for this change only.
> And why you think that not loading bloom filters will solve it? Thinking was with bloom filter false the indexing time will increase and that may fill up the queue fast so tried with true as well. But the error is coming independent of this setting. yes checking with http://localhost:9200/_nodes/thread_pool/stats?pretty=true for knowing the bulk rejections. > the soundest method is to check your bulk indexing code for capping the number of concurrent bulk requests. Or you may increase the number of nodes/shards. > Right now it seems your concurrency setting is too high for the capacity of your cluster (just a wild guess, without knowing your numbers) out setup: 4 shards per index. max 150 indices or so & have single node setup. In our case 1 bulk call has 2000 entries to be indexed & 4 concurrent bulk calls. Will also reduce bulk entries size and max 3 concurrent bulk calls and check on this again to see if that helps. Thanks. Btw since current indexing rate was working fine with 0.90.1 this got introduced with 0.90.9 for sure against our bulk indexing rate. Thanks, Ram On Monday, February 3, 2014 4:35:49 PM UTC+5:30, Jörg Prante wrote: > > The new setting of bounded bulk queue is a precaution not to overwhelm the > internal ES buffers. > > Unbounded queue size is potentially dangerous since your node operations > may get stalled by too many bulk requests. > > You could increase bulk queue_size from 50 to 100 if that works better. > But, the soundest method is to check your bulk indexing code for capping > the number of concurrent bulk requests. Or you may increase the number of > nodes/shards. Right now it seems your concurrency setting is too high for > the capacity of your cluster (just a wild guess, without knowing your > numbers). > > Jörg > > On Fri, Jan 31, 2014 at 6:48 AM, Ramchandra Phadake < > [email protected] <javascript:>> wrote: > >> Hi, >> >> Recently we migrated from 0.90.1 to 0.90.9. We did change >> index.codec.bloom.load=false initially but even with true we obsetve same >> behaviour. >> Nothing else has changed from indexing code. We are using bulk API. Any >> pointers why we can get EsRejectedExecutionException? >> This doesn't frequently in indexing process. >> >> We did try by setting with few thousand size but this exception comes but >> rarely. >> After setting bulk queue to -1 it appears to be working. -1 means >> unbounded size so is it safe to set to -1. >> >> Thanks, >> Ram >> >> -- >> You received this message because you are subscribed to the Google Groups >> "elasticsearch" group. >> To unsubscribe from this group and stop receiving emails from it, send an >> email to [email protected] <javascript:>. >> To view this discussion on the web visit >> https://groups.google.com/d/msgid/elasticsearch/89c6e465-c23f-4234-9667-cd09be4a7178%40googlegroups.com >> . >> For more options, visit https://groups.google.com/groups/opt_out. >> > > -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/17fabc2a-7cda-4e15-8ec3-eb6662ca5ca8%40googlegroups.com. For more options, visit https://groups.google.com/groups/opt_out.
