Yes, merges can hurt, but you can throttle them: http://search-lucene.com/?q=throttle+merge&fc_project=ElasticSearch You can easily correlate search latency with merges, flushes, and refreshes with something like SPM for Elasticsearch. This could help you figure out how much you need to throttle merges.
Otis -- Monitoring * Alerting * Anomaly Detection * Centralized Log Management Solr & Elasticsearch Support * http://sematext.com/ On Wednesday, April 15, 2015 at 1:10:04 AM UTC-4, Debayan Banerjee wrote: > > I do observe latency spikes, and massive cluster outages during bulk > indexing. I have moved to giving only 1 thread each to bulk and index > pools. It is now a 4 core machine so I guess I have 2 cores free for search. > > What are the other thread pools that can misbehave? Refresh, merge? > > On Tue, Apr 14, 2015 at 7:37 PM, Adrien Grand <[email protected] > <javascript:>> wrote: > >> Unfortunately it would not be enough since there are also merging threads >> that take CPI and I/O. I would recommend to stick with the defaults and >> only try to change settings if you observe latency spikes at search time >> which are due to indexing. >> >> On Tue, Apr 14, 2015 at 3:57 AM, Debayan Banerjee <[email protected] >> <javascript:>> wrote: >> >>> I have a setup where there is occasional heavy indexing but it is >>> critical to continue serving read requests. Would it then be a good idea to >>> assign only n-1 cores to index and bulk threadpools (n-1 threadpool size) >>> while leave one core free for just search? >>> >>> -- >>> >>> Debayan Banerjee >>> >>> >>> -- >>> You received this message because you are subscribed to the Google >>> Groups "elasticsearch" group. >>> To unsubscribe from this group and stop receiving emails from it, send >>> an email to [email protected] <javascript:>. >>> To view this discussion on the web visit >>> https://groups.google.com/d/msgid/elasticsearch/CAOvawCs%3D89nX06iBUYE4AFw%3D2dTX%2Byjg7qjeoxSHFZNJdGzspQ%40mail.gmail.com >>> >>> <https://groups.google.com/d/msgid/elasticsearch/CAOvawCs%3D89nX06iBUYE4AFw%3D2dTX%2Byjg7qjeoxSHFZNJdGzspQ%40mail.gmail.com?utm_medium=email&utm_source=footer> >>> . >>> For more options, visit https://groups.google.com/d/optout. >>> >> >> >> >> -- >> Adrien >> >> -- >> You received this message because you are subscribed to the Google Groups >> "elasticsearch" group. >> To unsubscribe from this group and stop receiving emails from it, send an >> email to [email protected] <javascript:>. >> To view this discussion on the web visit >> https://groups.google.com/d/msgid/elasticsearch/CAO5%3DkAhJ_TA%3D2aWAeXPJH-XEX9UH945R2%3DyoqLwwTCmQ8-HAug%40mail.gmail.com >> >> <https://groups.google.com/d/msgid/elasticsearch/CAO5%3DkAhJ_TA%3D2aWAeXPJH-XEX9UH945R2%3DyoqLwwTCmQ8-HAug%40mail.gmail.com?utm_medium=email&utm_source=footer> >> . >> For more options, visit https://groups.google.com/d/optout. >> > > > > -- > > Debayan Banerjee > devOps engineer > +91 8800846550 > > -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/4570c610-b3c7-4641-8b8b-81fc1b098b4c%40googlegroups.com. For more options, visit https://groups.google.com/d/optout.
