On Tuesday, April 14, 2015 at 10:23:32 AM UTC-7, [email protected] wrote: > > Dear, I've installed the current versions of Graylog and Elasticsearch: > > graylog-server 1.0.1-1 / graylog-web 1.0.1-1 / graylog2-stream-dashboard > 0.90.0-1 /elasticsearch 1.5.1 > > My server is Debian Wheezy, with 2 processors and 20 GB RAM (now I have 15 > GB free). >
Graylog needs more processors than that for your load. My rough guesstimate is that you need one core per 500 messages per second, plus a couple more cores for overhead, web, etc. You can validate this by looking at idle% on iostat -c 1 , my guess is that you'll see less than 10% idle on a consistent basis. > Everything works OK, but because of the high volume of received logs, I > get these two error messages: > > Journal utilization is too high 9 minutes ago > > Journal utilization is too high and may go over the limit soon. Please > verify that your Elasticsearch cluster is healthy and fast enough. You may > also want to review your Graylog journal settings and set a higher limit. > (Node: *b7b62947-250e-473b-b8df-7083d6df9886*, journal utilization: > 101.0%) > > Uncommited messages deleted from journal 9 minutes ago > > Some messages were deleted from the Graylog journal before they could be > written to Elasticsearch. Please verify that your Elasticsearch cluster is > healthy and fast enough. You may also want to review your Graylog journal > settings and set a higher limit. (Node: > *b7b62947-250e-473b-b8df-7083d6df9886*) > > Also the JVM in the Node tab is using 750MB of 972MB heap space, and there > are 1 million messages in the journal. > > Please, how can I tune the system in order to avoid these messages and > expand the heap space??? I'm using the default settings for elasticsearch > and graylog. > You fail to mention what operating system you are running under. Without that, we can't help you. If you're running on AWS Linux or Centos, you need to modify /etc/sysconfig/graylog-server to add the following to your GRAYLOG_SERVER_JAVA_OPTS : "-Xmx2000m" . That'll raise your heap size to 2Gb. In general adding more memory than 2gb really won't help Graylog, it'll just make garbage collection take longer, though "-XX:+UseG1GC" may help maintain responsiveness with more memory (that changes to the G1GC garbage collector, which sacrifices performance in favor of responsiveness). Graylog is more CPU-intensive than memory-intensive (Elasticsearch is exactly the opposite, it needs lots of memory and doesn't use much CPU). What is your Elasticsearch cluster configuration? Same machine as the Graylog server? In that case, you are *definitely* consuming all your CPU. Please note that if you've used the commercial solution whose name starts with an "s", Graylog uses *considerably* more resources than that solution. So yes, your configuration would have been adequate for that solution, but Graylog needs considerably more CPU to operate reliably and will certainly fall over on a regular basis on a machine that only has two cores if you're attempting to throw 1,000 message per second at it. -- You received this message because you are subscribed to the Google Groups "Graylog Users" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To view this discussion on the web visit https://groups.google.com/d/msgid/graylog2/722db3d8-5713-4510-8506-10d5f0d5ffbb%40googlegroups.com. For more options, visit https://groups.google.com/d/optout.
