Sorry, typo. It's currently 20000000 messages per index and average size of approximately 12GB.
On Thursday, November 10, 2016 at 12:47:39 AM UTC-8, Jimmy Chen wrote: > > Sorry I forgot to mention I increased the VM total memory to 8G and heap > to 4G. We are storing 20000 messages I believe per index, and the storage > is on SSD, so I cannot imagine the node being the bottleneck. Would you say > it's better to lower the amount of messages per index? > > On Wednesday, November 9, 2016 at 11:41:01 PM UTC-8, Jochen Schalanda > wrote: >> >> Hi Jimmy, >> >> On Wednesday, 9 November 2016 19:41:50 UTC+1, Jimmy Chen wrote: >>> >>> I bumped the memory to 4G for Xms and Xms. >>> >> >> Using 4 GiB of heap memory on a system with only 4 GiB of main memory is >> not recommended. Try using 2 or 3 GiB only so that the operating system can >> still use some memory for buffers and caches. >> >> >> It seems like the index optimization task may be related to the problem? >> >> >> This is a clear sign that your Elasticsearch nodes are either too slow or >> that your indices are too big. Index optimization (or "force merging") >> should be in the range of a few seconds, not almost 1 minute. >> >> Cheers, >> Jochen >> > -- You received this message because you are subscribed to the Google Groups "Graylog Users" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To view this discussion on the web visit https://groups.google.com/d/msgid/graylog2/f4e05f6e-d948-40cf-a0ba-d69e582abc99%40googlegroups.com. For more options, visit https://groups.google.com/d/optout.
