Re: [graylog2] Increase OUTPUT speed

[Roberto Carna]

Dear Marcus, special thanks for your comments.

Please just a brief question: I have this line in
/etc/default/graylog-server (considering I defined the HEAP SIZE =
16GB for Elasticsearch):

GRAYLOG_SERVER_JAVA_OPTS="-Xms16g -Xmx18g -XX:NewRatio=1
-XX:PermSize=256m -XX:MaxPermSize=512m -server -XX:+ResizeTLAB
-XX:+UseConcMarkSweepGC -XX:+CMSConcurrentMTEnabled
-XX:+CMSClassUnloadingEnabled -XX:+UseParNewGC
-XX:-OmitStackTraceInFastThrow"

What's the meaning of -XX:PermSize=256m -XX:MaxPermSize=512m ??? Do I
have to increase these options?

Thanks again, regards.

2016-07-27 5:28 GMT-03:00 Marcus Franke :
> Hi Roberto,
>
> If you have performance issues with the processing, then you have two
> choices:
>
> Have a look into your server.conf settings.
> There are several options to possibly speed up the output performance
> output_batch_size defaults to 25? If you have thousands of messages in the
> incoming then the batches are too small
> outputbuffer_processors .. if you have plenty of idle cpu power raise the
> value
>
> If you still have problems with the output buffer filling up you should have
> a look at tuning your es cluster for more indexing speed.
>
> Then there is the obvious option, do you have enough RAM in your server?
> Have you raised Xms and Xmx settings for the jvm graylog-server is running
> in? On RHEL based distributions this can be found in
> /etc/sysconfig/graylog-server.
>
> For example, I have set them to 24G for my setup, be aware of the java
> compressed pointers and thus you should not raise above 31G.
>
>
> regards,
> Marcus
>
> --
> You received this message because you are subscribed to a topic in the
> Google Groups "Graylog Users" group.
> To unsubscribe from this topic, visit
> https://groups.google.com/d/topic/graylog2/Yz3jmpfqnwQ/unsubscribe.
> To unsubscribe from this group and all its topics, send an email to
> graylog2+unsubscr...@googlegroups.com.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/graylog2/CAFRuYVcjSWLi1mKs%3DD9CJj5C41n%2BNXqaL2C2F_dfAbV_v7P%3DKw%40mail.gmail.com.
>
> For more options, visit https://groups.google.com/d/optout.

-- 
You received this message because you are subscribed to the Google Groups 
"Graylog Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to graylog2+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/graylog2/CAG2Qp6t0vhgrYoxK81C8snYUzuS9qyEOa_Bg4bTa4gqfo1cTzQ%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

Re: [graylog2] Increase OUTPUT speed

[Marcus Franke]

Hi Roberto,

If you have performance issues with the processing, then you have two
choices:

Have a look into your server.conf settings.
There are several options to possibly speed up the output performance
output_batch_size defaults to 25? If you have thousands of messages in the
incoming then the batches are too small
outputbuffer_processors .. if you have plenty of idle cpu power raise the
value

If you still have problems with the output buffer filling up you should
have a look at tuning your es cluster for more indexing speed.

Then there is the obvious option, do you have enough RAM in your server?
Have you raised Xms and Xmx settings for the jvm graylog-server is running
in? On RHEL based distributions this can be found in
/etc/sysconfig/graylog-server.

For example, I have set them to 24G for my setup, be aware of the java
compressed pointers and thus you should not raise above 31G.


regards,
Marcus

-- 
You received this message because you are subscribed to the Google Groups 
"Graylog Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to graylog2+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/graylog2/CAFRuYVcjSWLi1mKs%3DD9CJj5C41n%2BNXqaL2C2F_dfAbV_v7P%3DKw%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

[graylog2] Increase OUTPUT speed

[robertocarna36]

Dear, my OUTPUT is too slow so the journal of my Graylog is increasing time 
after time.

How can I speed up the OUTPUT in order to make it faster than the INPUT 
always??

Thanks a lot,

Roberto

-- 
You received this message because you are subscribed to the Google Groups 
"Graylog Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to graylog2+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/graylog2/a500adc6-ed17-467b-82f1-272e15346a49%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.