Hi Joshua, I am going to have to make a few assumptions here. The fact that your ES cluster is red will directly result in your journal filling up. Graylog will journal all the messages until ES appears green before it sends the messages.
I'd focus my time on looking through the ES logs for indications of what is causing Elasticsearch to be red. Until that goes green, nothing you change in mongo or Graylog will resolve the problem. Let us know if the ES log reports anything. Good luck! > On 9/12/2016, at 08:01, Joshua Waclawski <[email protected]> wrote: > > Hello Werner, > > Thanks for the reply and I apologize for the long delay, my project was > shelved briefly to handle some new work. With that out of the way I can get > back into this project... > > Well to start out, my graylog server crashes almost weekly due to the > application running out of some sort of resource, be it memory or journal > space; I'd like to figure out what configurations to change to stop this. > I'm running this on a dual-core server with 4 gb of memory and plenty of disk > space. > > Here are the errors I'm actually looking at on my server now: > Elasticsearch Cluster Unhealthy(RED) > Uncommited Messages Deleted from Journal > Journal Utilization is Too High > This is running on a default installation using the AWS-EC2 image. I have 11 > servers reporting to this server with a 1 minute average of 10 msg/s, so the > load on this server is incredibly low for what Graylog should be able to > handle. On top of that we have several new vulnerabilities on the system > that need addressing. > > My biggest problem right now is trying to find all of the configuration files > that I need to review and change. The documentation available does not match > how everything is setup in the EC2 image available on Amazon Web Services. > The mongod.cnf file required to configure mongoDB doesn't exist on this > server and the location stated in the documentation for the JVM settings file > is incorrect, as it's not in the /etc/default folder. I even tried doing a > search for the graylog-server file for JVM configuration and it showed me > three, all of which are in the /opt/graylog directory, so I have no clue > which one to edit. > > All I really need right now is a mapping of where the configuration files are > on the EC2 image and I should be able to go from there. > >> On Monday, November 21, 2016 at 5:43:52 PM UTC-5, Werner van der Merwe wrote: >> Hi Joshua, >> >> Hardware requirements: >> It is obviously very difficult to give you exact numbers. The requirements >> for 300 syslog messages vs 300 multi-line logs where one extracts 50 key >> value pairs per entry will have different requirements. That said, 300 >> messages is trivial and you can get away with very low resources, at a guess >> I'd say 1 or 2 cores and 4-8Gb ram will be more than adequate, again, >> depending on what you are sending it. >> >> Configs are actually fairly straight-forward. Most GrayLog config has moved >> to Mongo, so you only _really_ need to worry about server.conf file in >> /etc/graylog/server. >> Keep in mind that Mongo and Elasticsearch is used by Graylog, but not >> included, so their configs are managed separately. >> >> If you have more specific questions around the config, that might help as >> well. >> >> Apart from that, if you are using puppet, the manifest they provide is >> really good and does everything for you! >> >> You mention you've hit a roadblock - are you struggling to get the system >> running, is it running slow? Might help to let us know a bit more detail on >> what is keeping you from moving forward? >> >> Cheers >> >>> On Friday, November 18, 2016 at 5:50:17 AM UTC+13, Joshua Waclawski wrote: >>> As the title states, I'm pretty new to Graylog and Elasticsearch. I've >>> read the documentation thoroughly and I've watched a few educational videos >>> describing how elasticsearch works from the ground up; everything is very, >>> very cool and I'm excited to start using it! Using the AMI provided on the >>> Github, I've setup an EC2 instance and have started work on learning how to >>> configure and use this tool, but I've hit a bit of a road block and need >>> some answers... >>> Hardware requirements - what exactly are they? I'm attempting to deploy >>> graylog to an environment that receives no more than maybe 200-300 messages >>> per second, if that. I can't imagine that managing a few thousand logs per >>> minute requires 12gb of RAM to do, but I'm new to Elasticsearch so I'm >>> asking for clarification. Every white paper, forum post, blog post, or >>> guide that I've read so far assumes 5000+ messages per second. >>> Configurations - This is what annoys me the most. The configuration files >>> are very, very scattered (at least they seem so) and the official >>> documentation does a very poor job of explaining what's required to >>> configure for basic functionality on a single server. Using the AWS AMI, >>> what configuration files need editing to inflict changes upon the system? >>> I'm seeing .conf, .yml, and .cfg files all over my operating system with >>> seemingly redundant settings that I can't find explanations for. Again, >>> this could be my ignorance of the architecture, but the file hierarchy is >>> explained no where. >>> That's really it for now.. if I can hammer out these answers with some >>> level of certainty then I'll have what I need to move forward in >>> configuring and testing this software. As thing stand now, I have no idea >>> if I'm editing the proper config file half the time as all I receive are >>> errors in logs and alerts in the webUI. > > -- > You received this message because you are subscribed to a topic in the Google > Groups "Graylog Users" group. > To unsubscribe from this topic, visit > https://groups.google.com/d/topic/graylog2/KlZkOlCdffE/unsubscribe. > To unsubscribe from this group and all its topics, send an email to > [email protected]. > To view this discussion on the web visit > https://groups.google.com/d/msgid/graylog2/904086fc-f92f-415c-b78b-4ca327b04894%40googlegroups.com. > For more options, visit https://groups.google.com/d/optout. -- You received this message because you are subscribed to the Google Groups "Graylog Users" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To view this discussion on the web visit https://groups.google.com/d/msgid/graylog2/C4545B87-6BD3-458A-94B8-6AFFFA6A7CA5%40gmail.com. For more options, visit https://groups.google.com/d/optout.
