Hi Jimmy, almost 1 minute for index optimization is definitely too long for the hardware used for the ES cluster you've mentioned before. You should investigate in that direction.
Cheers, Jochen On Thursday, 10 November 2016 09:47:39 UTC+1, Jimmy Chen wrote: > > Sorry I forgot to mention I increased the VM total memory to 8G and heap > to 4G. We are storing 20000 messages I believe per index, and the storage > is on SSD, so I cannot imagine the node being the bottleneck. Would you say > it's better to lower the amount of messages per index? > > On Wednesday, November 9, 2016 at 11:41:01 PM UTC-8, Jochen Schalanda > wrote: >> >> Hi Jimmy, >> >> On Wednesday, 9 November 2016 19:41:50 UTC+1, Jimmy Chen wrote: >>> >>> I bumped the memory to 4G for Xms and Xms. >>> >> >> Using 4 GiB of heap memory on a system with only 4 GiB of main memory is >> not recommended. Try using 2 or 3 GiB only so that the operating system can >> still use some memory for buffers and caches. >> >> >> It seems like the index optimization task may be related to the problem? >> >> >> This is a clear sign that your Elasticsearch nodes are either too slow or >> that your indices are too big. Index optimization (or "force merging") >> should be in the range of a few seconds, not almost 1 minute. >> >> Cheers, >> Jochen >> > -- You received this message because you are subscribed to the Google Groups "Graylog Users" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To view this discussion on the web visit https://groups.google.com/d/msgid/graylog2/a77ee872-2432-4726-a8f8-ca7b6b203632%40googlegroups.com. For more options, visit https://groups.google.com/d/optout.
