Is there anything specific on ES end I could start with? On Thursday, November 10, 2016 at 1:20:27 AM UTC-8, Jochen Schalanda wrote: > > Hi Jimmy, > > almost 1 minute for index optimization is definitely too long for the > hardware used for the ES cluster you've mentioned before. You should > investigate in that direction. > > Cheers, > Jochen > > On Thursday, 10 November 2016 09:47:39 UTC+1, Jimmy Chen wrote: >> >> Sorry I forgot to mention I increased the VM total memory to 8G and heap >> to 4G. We are storing 20000 messages I believe per index, and the storage >> is on SSD, so I cannot imagine the node being the bottleneck. Would you say >> it's better to lower the amount of messages per index? >> >> On Wednesday, November 9, 2016 at 11:41:01 PM UTC-8, Jochen Schalanda >> wrote: >>> >>> Hi Jimmy, >>> >>> On Wednesday, 9 November 2016 19:41:50 UTC+1, Jimmy Chen wrote: >>>> >>>> I bumped the memory to 4G for Xms and Xms. >>>> >>> >>> Using 4 GiB of heap memory on a system with only 4 GiB of main memory is >>> not recommended. Try using 2 or 3 GiB only so that the operating system can >>> still use some memory for buffers and caches. >>> >>> >>> It seems like the index optimization task may be related to the problem? >>> >>> >>> This is a clear sign that your Elasticsearch nodes are either too slow >>> or that your indices are too big. Index optimization (or "force merging") >>> should be in the range of a few seconds, not almost 1 minute. >>> >>> Cheers, >>> Jochen >>> >>
-- You received this message because you are subscribed to the Google Groups "Graylog Users" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To view this discussion on the web visit https://groups.google.com/d/msgid/graylog2/5e68cfd9-35e8-4a0f-9db9-095efb0b3083%40googlegroups.com. For more options, visit https://groups.google.com/d/optout.
