Hi there, I just made my 2nd attempt to upgrade from GL 1.3.4/ES 1.7.1 to GL2.1.1/ES 2.3.5 and expierenced similar problems like I had when upgrading to GL2.0 (and decided to rollback).
Issues are mainly in throughput rate - the processing buffer is always at 100% and the disk journal just keeps on growing since more messages come in than can be processed. The top right counter often remains at 0 outgoing messages and peaks sporadically at 2k m/s. In 1.3.4 it remains fairly constantly at 2.5k m/s and the processing buffer fills up at burst times to like 50% maximum. Incoming messages peak at 8k at times. My setup is fairly straight forward: 2 VMs - 1 VM with Graylog (6 CPU, 12GB RAM), 1 VM with ES-DB (4 CPU, 34GB RAM). We give ES 18GB RAM (well, Java that is). Storage wise I keep rotating indexes by size every 40GB and keep 7 indexes (roughly 5 days of data) Configs: Graylog 1.3.4 (working conf): http://pastebin.com/TjUbNJps 2.1.1: http://pastebin.com/8WYgGYWB ES 1.7.1: http://pastebin.com/bNh3FY1k 2.3.5: http://pastebin.com/eWm5x4zH Now, last time I tried upgrading I got answers in IRC that there´s a known perfomance bottleneck with GL not using HTTP as a means of putting data into ES and would likely be the issue I expierenced. Lastly, this is the log form GL 2.1.1 which had some errors which "might" be related just to the front end (according to this: https://github.com/Graylog2/graylog2-server/issues/2345), also, I noted a different TCP-Connection pattern after upgrading: http://imgur.com/n7l8C1b GL logs: http://pastebin.com/X5K3ZucP Any help will be appreciated since I´d really like to upgrade to GL 2.x. -- You received this message because you are subscribed to the Google Groups "Graylog Users" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To view this discussion on the web visit https://groups.google.com/d/msgid/graylog2/65a972cb-a79f-4e4a-a043-ee90e2734678%40googlegroups.com. For more options, visit https://groups.google.com/d/optout.
