shift wrote:
I am seeing a high number of rejections for the bulk thread pool
on a 32 core system. Should I leave the thread pool size fixed
to the # of cores and the default queue size at 50? Are these
rejections re-processed?
From my clients sending bulk documents (logstash), do I need to
limit the number of connections to 32? I currently have 200
output threads to each elasticsearch node.
The rejections are telling you that ES's bulk thread pool is busy
and it can't enqueue any more to wait for an open thread. They
aren't retried. The exception your client gets is the final word
for that request.
Lower your logstash threads to 16 or 32, monitor rejections, and
gradually raise. You could also increase the queue size, but keep
in mind that's only useful to handle spikes. You probably don't
want to keep thousands around waiting since they take resources.
Drew
"bulk" : {
"threads" : 32,
* "queue" : 50,*
"active" : 32,
* "rejected" : 12592108,*
"largest" : 32,
"completed" : 584407554
}
Thanks! Any feedback is appreciated.
--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to [email protected].
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/m2lhvnpwx8.fsf%40mid.raines.me.
For more options, visit https://groups.google.com/d/optout.
