[graylog2] Increase OUTPUT speed
Dear, my OUTPUT is too slow so the journal of my Graylog is increasing time after time. How can I speed up the OUTPUT in order to make it faster than the INPUT always?? Thanks a lot, Roberto -- You received this message because you are subscribed to the Google Groups "Graylog Users" group. To unsubscribe from this group and stop receiving emails from it, send an email to graylog2+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/graylog2/a500adc6-ed17-467b-82f1-272e15346a49%40googlegroups.com. For more options, visit https://groups.google.com/d/optout.
[graylog2] Settings for Journal when utilization is too high
Dear, I'm using Graylog 1.3 with CPU x 10, RAM x 40GB and HD x 1.5 TB. The input is about 4500 logs/second. Today I have received this warning: Journal utilization is too hig Journal utilization is too high and may go over the limit soon. Please verify that your Elasticsearch cluster is healthy and fast enough. You may also want to review your Graylog journal settings and set a higher limit. (Node: *ea2b7f43-cce0-4288-b344-a4e748e3c372*, journal utilization: 96.0%) and now the journal has 12 millons of logs (in disk). I've increased the heap size to 16 GB: /etc/default/elasticsearch: ES_HEAP_SIZE=16g and I've done this: /etc/default/graylog-server: GRAYLOG_SERVER_JAVA_OPTS="-Xms16g -Xmx20g -XX:NewRatio=1 -XX:PermSize=256m -XX:MaxPermSize=512m -server -XX:+ResizeTLAB -XX:+UseConcMarkSweepGC -XX:+CMSConcurrentMTEnabled -XX:+CMSClassUnloadingEnabled -XX:+UseParNewGC -XX:-OmitStackTraceInFastThrow" How can I solve this journal problem please??? Thanks a lot, Roberto -- You received this message because you are subscribed to the Google Groups "Graylog Users" group. To unsubscribe from this group and stop receiving emails from it, send an email to graylog2+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/graylog2/33510f0b-cde7-4c5b-9636-867a03440fd0%40googlegroups.com. For more options, visit https://groups.google.com/d/optout.
[graylog2] When to scale resources for Graylog???
People, I have a Graylog 1.3 server in just one Linux box (Debian 8), so I mean I have one Elasticsearch node. Nowadays I'm receiveing about 4000/6000 logs/second. I had to increase the memory heap size of JVM, and used CPU x 10 and RAM x 40GB and after that everything seems OK, because I have near 200/800 unprocessed messages as maximum everytime. When do you recommend to scale to more Elasticsearch nodes or to have diferent MongoDB's or somethinh like that??? Is there a logs/seg threshold meaning I have to scale to a distributed architecture??? Thanks a lot!!! Roberto -- You received this message because you are subscribed to the Google Groups "Graylog Users" group. To unsubscribe from this group and stop receiving emails from it, send an email to graylog2+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/graylog2/46b85a17-54fb-4f99-8493-fdfa5add8c77%40googlegroups.com. For more options, visit https://groups.google.com/d/optout.
[graylog2] Re: Several indices from 1 and 2 hours ago
Dear Jochen, I'm using this Graylog version on a Debian 8 server: graylog-server 1.3.3-1 all Graylog server graylog-web 1.3.3-1 all Graylog web My indices configuration in /etc/graylog/server/server.conf is: rotation_strategy = time elasticsearch_max_time_per_index = 3d elasticsearch_max_number_of_indices = 10 retention_strategy = delete Please can you tell I'm OK ??? Do you say every time I reboot my server or restart the graylog-server service I could have problems with the indices??? Thanks a lot!! El lunes, 25 de julio de 2016, 11:32:31 (UTC-3), Jochen Schalanda escribió: > > Hi Roberto, > > which exact version of Graylog are you using? > > There were some versions of Graylog which would rotate the indices on > startup if the time-based rotation strategy was being used, even if the > shouldn't be rotated according to their age. > > Would it be feasible for you to upgrade to Graylog 2.x? > > Cheers, > Jochen > > On Monday, 25 July 2016 16:22:31 UTC+2, Roberto Carna wrote: >> >> Dear, I've cloned a Graylog 1.3 virtual machine with its corresponding >> indices, to a new one. This new one Graylog virtual machine started >> with the same indices, and after that I've deleted some of them. >> >> But today I was analyzing the Graylog options, and I realized that the >> indices don't respond in accordance to my current configuration: >> "rotates the indices every 3 days and keeps a maximum number of 10 >> indices", as follow: >> >> Graylog2_90: Contains messages up to a few seconds ago (1.8GiB / >> 4,198,541 messages) >> >> Graylog2_89: Contains messages from an hour ago up to in 3 hours >> (2.3GiB / 6,943,219 messages) >> >> Graylog2_88: Contains messages from an hour ago up to in 2 hours >> (307.7MiB / 887,500 messages) >> >> Graylog2_87: Contains messages from an hour ago up to in 2 hours >> (823.1MiB / 2,434,500 messages) >> >> ... >> >> Graylog2_81: Contains messages from 5 days ago up to 4 days ago >> (27.8GiB / 84,685,427 messages) >> >> What can I do in order to have my indices matching the current >> configuration I defined? >> >> Thanks a lot, regards. >> >> Roberto >> > -- You received this message because you are subscribed to the Google Groups "Graylog Users" group. To unsubscribe from this group and stop receiving emails from it, send an email to graylog2+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/graylog2/eb43727c-8bfa-42ea-b6d6-94e682c49b3b%40googlegroups.com. For more options, visit https://groups.google.com/d/optout.
[graylog2] Backup of indices in Graylog 1.3
Hi people, I have Graylog 1.3 as my syslog server. I have setup the following strategy: 10 indices 3 days for indice delete and not close total: 30 days of data I want to backup the indices to a Networker EMC server, but all the indices I have in the Graylog web interface are not closed. Can I backup a non-closed indice (or index) ??? Or when I have to restore it after a long time I will can't do that because the indice was not closed ??? Thanks a lot, regards. Roberto -- You received this message because you are subscribed to the Google Groups "Graylog Users" group. To unsubscribe from this group and stop receiving emails from it, send an email to graylog2+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/graylog2/cd30e7a4-3881-4e38-9011-c1c07079a710%40googlegroups.com. For more options, visit https://groups.google.com/d/optout.
[graylog2] Graylog /var parition always increases
Dear, I have GRaylog 1.3 and I setup a /var partition of 1.5 TB. I define 10 indexes of 3 days each, and every index is deleted after that. In despite of this strategy, the /var partition of Graylog server always increases and when it reaches 95% aprox, the Graylog stop logging. What can I do in order to maintain the /var partition size in a value that never reaches the 90%??? Because in the firts days of my Graylog server, I had 3 indexes of 3 days each, after that 20 indexes of 3 days, now 10 indexes of 3 days...and always my /var partition gets nearly 100%. Thanks a lot, Roberto -- You received this message because you are subscribed to the Google Groups "Graylog Users" group. To unsubscribe from this group and stop receiving emails from it, send an email to graylog2+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/graylog2/dbdecac3-2271-4b9b-8ae2-8e2fcb8aeceb%40googlegroups.com. For more options, visit https://groups.google.com/d/optout.
[graylog2] Re: Elasticsearch cluster unavailable: Graylog 1.3.3 and Elasticsearch 2.2.0
Sorry, I've read that Graylog 1.3 is not compatible with Elasticsearch 2.x. So I've installed Elasticsearch 1.7.5 and everything is OK. Regards, El jueves, 18 de febrero de 2016, 11:50:14 (UTC-3), roberto...@gmail.com escribió: > > Dear, I've installed a syslog server with the last versions of > Elasticsearch 2.2.0 and Graylog 1.3.3, both installed via APT-GET packages > as I've done later with older versions, following these guides: > > > https://www.elastic.co/guide/en/elasticsearch/reference/current/setup-repositories.html > > http://docs.graylog.org/en/latest/pages/installation/operating_system_packages.html > > (for Debian) > > The elasticsearch, graylog-server and graylog-web services are up, I setup > the cluster.name = graylog2 in both elasticsearch.yml and server.conf, > and I have only one Node. > > Also in server.conf I defined: > > elasticsearch_discovery_zen_ping_multicast_enabled = false > elasticsearch_discovery_zen_ping_unicast_hosts = > es-node-1.example.org:9300,es-node-2.example.org:9300 > > In /var/log/elasticsearch I have no logs from elasticsearch, just from > graylog. > > When I enter the Graylog web interface, I see the message: > > *"Elasticsearch cluster unavailable" and "**Cluster information currently > unavailable"* > > What can be the problem? It's a fresh implementation, with older version of > elasticsearch and graylog I followedd the same steps and the syslog worked OK. > > Thanks a lot!!! > > Roberto > > > > > -- You received this message because you are subscribed to the Google Groups "Graylog Users" group. To unsubscribe from this group and stop receiving emails from it, send an email to graylog2+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/graylog2/77021d45-56fb-4802-81a8-e26b5dcb608e%40googlegroups.com. For more options, visit https://groups.google.com/d/optout.
[graylog2] I have no outgoing messages from Graylog
Dear, I have Graylog 1.2 but right now I have a lot of incoming messages but no outgoing messages at all, so my journal space is increasing a lot: *Processing 1500 incoming and 0 outgoing msg/s. 1,877,835 unprocessed messages* I can see just this error or warning: *Elasticsearch cluster is red.* Shards: 92 active, 0 initializing, 0 relocating, 4 unassigned What can be the problem? How can I get outgoing messages again ? Thanks a lot, Roberto -- You received this message because you are subscribed to the Google Groups "Graylog Users" group. To unsubscribe from this group and stop receiving emails from it, send an email to graylog2+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/graylog2/87b06813-965a-49de-914b-3391eb7d4e5c%40googlegroups.com. For more options, visit https://groups.google.com/d/optout.
[graylog2] Journal filling in a short time
Dear, Ia have Graylog 1.2 with just one Elasticsearch node. I receive lots of logs from different devices. After a pair of hours, I often notice that incoming messages are higher than outgoing messages, and so the journal is fullfilled and the message processing mechanism stops, and I have to delete messages from journal manually. This is a sample verbose message from the Nodes of Graylog: Processing *1,126* incoming and *500* outgoing msg/s. *130,739 unprocessed messages* are currently in the journal, in 1 segments. *857 messages* have been appended to, and *857 messages* have been read from the journal in the last second. Is there any way to process more messages and have higher outgoing messages? Or any other way to avoid the fullfilling of the journal ? Thanks a lot, Roberto -- You received this message because you are subscribed to the Google Groups "Graylog Users" group. To unsubscribe from this group and stop receiving emails from it, send an email to graylog2+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/graylog2/5626cf24-5d87-43dc-82c1-c13bbac5fb50%40googlegroups.com. For more options, visit https://groups.google.com/d/optout.
[graylog2] Graylog 1.2 rotation strategy
Dear, I have a Graylog 1.2 server which receives lot of messages per seconds. I need to have a rotation strategy in order to mantain 6 months of logs, and after that time the indexes will be deleted. I think I have to add this lines to the /etc/graylog/server/server.conf file: rotation_strategy = time elasticsearch_max_time_per_index = 1d elasticsearch_max_number_of_indices = 180 # 6 months retention_strategy = delete Is this configuration OK??? And also, is it a good idea to increase the elasticsearch_max_time_per_index from 1d to 10d, and in this case elasticsearch_max_number_of_indices = 18??? Thanking in advance. Roberto. -- You received this message because you are subscribed to the Google Groups "Graylog Users" group. To unsubscribe from this group and stop receiving emails from it, send an email to graylog2+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/graylog2/8ab1d821-c69e-4cfe-93b6-4f73b244403a%40googlegroups.com. For more options, visit https://groups.google.com/d/optout.
[graylog2] Graylog.-server service doesn't start after remove /var/lib/graylog-server/journal/* files
Dear, I have Graylog 1.1 and today I have to remove all the files under /var/lib/graylog-server/journal/. I remove all the files without stopping any service (elasticsearch, graylog-web and graylog-server). After that, I reboot the server but the graylog-server doesn't start at all, and I can see this error log message.can you help please ??? Thanks a lot. 2015-09-28T12:00:42.414-03:00 INFO [CmdLineTool] Loaded plugins: [Anonymous Usage Statistics 1.0.5 [org.graylog.plugins.usagestatistics.UsageStatsPlugin]] 2015-09-28T12:00:42.469-03:00 INFO [MongoDbConfiguration] You're using deprecated configuration options for MongoDB. Please use mongodb_uri. 2015-09-28T12:00:42.504-03:00 INFO [MongoDbConfiguration] Suggested value for mongodb_uri = mongodb://graylog2:GrayPnet@127.0.0.1:27017/graylog2 2015-09-28T12:00:42.533-03:00 INFO [CmdLineTool] Running with JVM arguments: -Xms1g -Xmx1g -XX:NewRatio=1 -XX:PermSize=128m -XX:MaxPermSize=256m -XX:+ResizeTLAB -XX:+UseConcMarkSweepGC -XX:+CMSConcurrentMTEnabled -XX:+CMSClassUnloadingEnabled -XX:+UseParNewGC -XX:-OmitStackTraceInFastThrow -Dlog4j.configuration=file:///etc/graylog/server/log4j.xml -Djava.library.path=/usr/share/graylog-server/lib/sigar 2015-09-28T12:00:45.834-03:00 INFO [InputBufferImpl] Message journal is enabled. 2015-09-28T12:00:46.083-03:00 INFO [LogManager] Found clean shutdown file. Skipping recovery for all logs in data directory '/var/lib/graylog-server/journal' 2015-09-28T12:00:46.084-03:00 INFO [LogManager] Loading log 'messagejournal-0' 2015-09-28T12:00:46.113-03:00 INFO [Log] Completed load of log messagejournal-0 with log end offset 0 2015-09-28T12:00:46.125-03:00 INFO [KafkaJournal] Initialized Kafka based journal at /var/lib/graylog-server/journal 2015-09-28T12:00:46.138-03:00 INFO [InputBufferImpl] Initialized InputBufferImpl with ring size <65536> and wait strategy , running 2 parallel message handlers. 2015-09-28T12:00:46.318-03:00 INFO [NodeId] Node ID: b7b62947-250e-473b-b8df-7083d6df9886 2015-09-28T12:00:46.486-03:00 INFO [node] [graylog2-server] version[1.5.2], pid[3720], build[62ff986/2015-04-27T09:21:06Z] 2015-09-28T12:00:46.487-03:00 INFO [node] [graylog2-server] initializing ... 2015-09-28T12:00:46.496-03:00 INFO [plugins] [graylog2-server] loaded [graylog2-monitor], sites [] 2015-09-28T12:00:48.786-03:00 INFO [node] [graylog2-server] initialized 2015-09-28T12:00:48.796-03:00 INFO [ProcessBuffer] Initialized ProcessBuffer with ring size <65536> and wait strategy . 2015-09-28T12:00:50.543-03:00 INFO [RulesEngineProvider] No static rules file loaded. 2015-09-28T12:00:50.741-03:00 INFO [OutputBuffer] Initialized OutputBuffer with ring size <65536> and wait strategy . 2015-09-28T12:00:51.221-03:00 INFO [Version] HV01: Hibernate Validator 5.1.3.Final 2015-09-28T12:00:51.630-03:00 INFO [ServerBootstrap] Graylog server 1.1.1 (893e8e7) starting up. (JRE: Oracle Corporation 1.7.0_79 on Linux 3.2.0-4-amd64) 2015-09-28T12:00:51.645-03:00 INFO [PeriodicalsService] Starting 21 periodicals ... 2015-09-28T12:00:51.729-03:00 INFO [Periodicals] Starting [org.graylog2.periodical.ThroughputCounterManagerThread] periodical in [0s], polling every [1s]. 2015-09-28T12:00:51.711-03:00 INFO [node] [graylog2-server] starting ... 2015-09-28T12:00:51.780-03:00 INFO [Periodicals] Starting [org.graylog2.periodical.ThroughputCalculator] periodical in [0s], polling every [1s]. 2015-09-28T12:00:51.787-03:00 INFO [Periodicals] Starting [org.graylog2.periodical.AlertScannerThread] periodical in [10s], polling every [60s]. 2015-09-28T12:00:51.793-03:00 INFO [Periodicals] Starting [org.graylog2.periodical.BatchedElasticSearchOutputFlushThread] periodical in [0s], polling every [1s]. 2015-09-28T12:00:51.796-03:00 INFO [Periodicals] Starting [org.graylog2.periodical.ClusterHealthCheckThread] periodical in [0s], polling every [20s]. 2015-09-28T12:00:51.806-03:00 INFO [Periodicals] Starting [org.graylog2.periodical.ContentPackLoaderPeriodical] periodical, running forever. 2015-09-28T12:00:51.817-03:00 INFO [Periodicals] Starting [org.graylog2.periodical.DeadLetterThread] periodical, running forever. 2015-09-28T12:00:51.818-03:00 INFO [Periodicals] Starting [org.graylog2.periodical.GarbageCollectionWarningThread] periodical, running forever. 2015-09-28T12:00:51.825-03:00 INFO [Periodicals] Starting [org.graylog2.periodical.IndexerClusterCheckerThread] periodical in [0s], polling every [30s]. 2015-09-28T12:00:51.845-03:00 INFO [Periodicals] Starting [org.graylog2.periodical.IndexRetentionThread] periodical in [0s], polling every [300s]. 2015-09-28T12:00:51.846-03:00 INFO [IndexRetentionThread] Elasticsearch cluster not available, skipping index retention checks. 2015-09-28T12:00:51.846-03:00 INFO [Periodicals] Starting [org.graylog2.periodical.IndexRotationThread] periodical in [0s], polling every [10s]. 2015-09-28T12:00:51.848-03:00 INFO [Periodicals] Starting
[graylog2] Re: I receive a lot of logs but Graylog only shows a few
Dear, I've read the link about ASA's remote logging but it's the same I've done. The problem is that lot of ASA logs come to my Graylog server, I see them with tcpdump, but just a little part of them are displayed on the web interfaceIs it possible that all the logs arent't displayed but any reason I don't know??? Thanks a lot!!! El viernes, 24 de abril de 2015, 3:16:24 (UTC-3), Fisz escribió: Hi, There are many types of sending logs from ASA. For ex. you can send different logs on ASA ASDM, and different on syslog server. This topic might interest you: http://www.cisco.com/c/en/us/support/docs/security/pix-500-series-security-appliances/63884-config-asa-00.html On Thursday, April 23, 2015 at 3:22:32 PM UTC+2, roberto...@gmail.com wrote: Dear, I have Graylog 1.0.1 installed in a Debian Wheezy box. Everything works OK, except the Cisco ASA incoming logs. When I'm in Graylog terminal, I execute tcpdump pointing to Cisco ASA IP, and I can see a lot of incoming logsbut when I'm in the Graylog web interface, and choose the Cisco ASA source, there are a few logs. What can be the problem with this situation??? Thanks a lot, Roberto. -- You received this message because you are subscribed to the Google Groups graylog2 group. To unsubscribe from this group and stop receiving emails from it, send an email to graylog2+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
[graylog2] Re: I receive a lot of logs but Graylog only shows a few
From tcpdump I get lines like these, and I can see ICMP unreachable messages but from Graylog to Cisco ASA I think they're not relevant: 10:22:44.814404 IP Cisco-ASA.syslog GRAYLOG.syslog: SYSLOG local4.warning, length: 166 10:22:44.814445 IP GRAYLOG Cisco-ASA: ICMP GRAYLOG udp port syslog unreachable, length 202 10:22:49.823279 IP Cisco-ASA.syslog GRAYLOG.syslog: SYSLOG local4.warning, length: 166 10:22:49.823313 IP GRAYLOG Cisco-ASA: ICMP GRAYLOG udp port syslog unreachable, length 202 10:22:54.823912 IP Cisco-ASA.syslog GRAYLOG.syslog: SYSLOG local4.warning, length: 166 10:22:54.823953 IP GRAYLOG Cisco-ASA: ICMP GRAYLOG udp port syslog unreachable, length 202 10:22:59.823951 IP Cisco-ASA.syslog GRAYLOG.syslog: SYSLOG local4.warning, length: 166 10:22:59.823981 IP GRAYLOG Cisco-ASA: ICMP GRAYLOG udp port syslog unreachable, length 202 10:23:04.831671 IP Cisco-ASA.syslog GRAYLOG.syslog: SYSLOG local4.warning, length: 166 10:23:04.831710 IP GRAYLOG Cisco-ASA: ICMP GRAYLOG udp port syslog unreachable, length 202 10:23:09.832059 IP Cisco-ASA.syslog GRAYLOG.syslog: SYSLOG local4.warning, length: 166 10:23:09.832085 IP GRAYLOG Cisco-ASA: ICMP GRAYLOG udp port syslog unreachable, length 202 El viernes, 24 de abril de 2015, 10:16:25 (UTC-3), roberto...@gmail.com escribió: Not nowmy Indices section is green without failures...any idea please??? Thanks again. El viernes, 24 de abril de 2015, 10:01:45 (UTC-3), Fisz escribió: Do you have some indexer failures in graylog? On Friday, April 24, 2015 at 2:14:28 PM UTC+2, roberto...@gmail.com wrote: Dear, I've read the link about ASA's remote logging but it's the same I've done. The problem is that lot of ASA logs come to my Graylog server, I see them with tcpdump, but just a little part of them are displayed on the web interfaceIs it possible that all the logs arent't displayed but any reason I don't know??? Thanks a lot!!! El viernes, 24 de abril de 2015, 3:16:24 (UTC-3), Fisz escribió: Hi, There are many types of sending logs from ASA. For ex. you can send different logs on ASA ASDM, and different on syslog server. This topic might interest you: http://www.cisco.com/c/en/us/support/docs/security/pix-500-series-security-appliances/63884-config-asa-00.html On Thursday, April 23, 2015 at 3:22:32 PM UTC+2, roberto...@gmail.com wrote: Dear, I have Graylog 1.0.1 installed in a Debian Wheezy box. Everything works OK, except the Cisco ASA incoming logs. When I'm in Graylog terminal, I execute tcpdump pointing to Cisco ASA IP, and I can see a lot of incoming logsbut when I'm in the Graylog web interface, and choose the Cisco ASA source, there are a few logs. What can be the problem with this situation??? Thanks a lot, Roberto. -- You received this message because you are subscribed to the Google Groups graylog2 group. To unsubscribe from this group and stop receiving emails from it, send an email to graylog2+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
[graylog2] I receive a lot of logs but Graylog only shows a few
Dear, I have Graylog 1.0.1 installed in a Debian Wheezy box. Everything works OK, except the Cisco ASA incoming logs. When I'm in Graylog terminal, I execute tcpdump pointing to Cisco ASA IP, and I can see a lot of incoming logsbut when I'm in the Graylog web interface, and choose the Cisco ASA source, there are a few logs. What can be the problem with this situation??? Thanks a lot, Roberto. -- You received this message because you are subscribed to the Google Groups graylog2 group. To unsubscribe from this group and stop receiving emails from it, send an email to graylog2+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
[graylog2] Re: graylog-server doesn't start automatically
In the /etc/init.d/graylog-server file I add the line: /bin/sleep 20 and the graylog-server service starts perfectly. Maybe graylog-server has to wait more time for any condition I don't know??? Regards, Roberto El jueves, 16 de abril de 2015, 10:46:06 (UTC-3), roberto...@gmail.com escribió: Dear, I've installed Graylog 1.0.1. Elasticsearch and graylog-web start automatically but graylog-server doesn't. I edit /etc/rc.local with: /etc/init.d/graylog-server start but after reboot the graylog-server is stopped. The only way to start the service is executing manually from terminal: # service graylog-server start How can I do in order to start graylog-server automatically on boot??? Thanks a lot, Roberto -- You received this message because you are subscribed to the Google Groups graylog2 group. To unsubscribe from this group and stop receiving emails from it, send an email to graylog2+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
[graylog2] graylog-server doesn't start automatically
Dear, I've installed Graylog 1.0.1. Elasticsearch and graylog-web start automatically but graylog-server doesn't. I edit /etc/rc.local with: /etc/init.d/graylog-server start but after reboot the graylog-server is stopped. The only way to start the service is executing manually from terminal: # service graylog-server start How can I do in order to start graylog-server automatically on boot??? Thanks a lot, Roberto -- You received this message because you are subscribed to the Google Groups graylog2 group. To unsubscribe from this group and stop receiving emails from it, send an email to graylog2+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
[graylog2] Journal utilization is too high and uncommited messages
Dear, I've installed the current versions of Graylog and Elasticsearch: graylog-server 1.0.1-1 / graylog-web 1.0.1-1 / graylog2-stream-dashboard 0.90.0-1 /elasticsearch 1.5.1 My server is Debian Wheezy, with 2 processors and 20 GB RAM (now I have 15 GB free). Everything works OK, but because of the high volume of received logs, I get these two error messages: Journal utilization is too high 9 minutes ago Journal utilization is too high and may go over the limit soon. Please verify that your Elasticsearch cluster is healthy and fast enough. You may also want to review your Graylog journal settings and set a higher limit. (Node: *b7b62947-250e-473b-b8df-7083d6df9886*, journal utilization: 101.0%) Uncommited messages deleted from journal 9 minutes ago Some messages were deleted from the Graylog journal before they could be written to Elasticsearch. Please verify that your Elasticsearch cluster is healthy and fast enough. You may also want to review your Graylog journal settings and set a higher limit. (Node: *b7b62947-250e-473b-b8df-7083d6df9886*) Also the JVM in the Node tab is using 750MB of 972MB heap space, and there are 1 million messages in the journal. Please, how can I tune the system in order to avoid these messages and expand the heap space??? I'm using the default settings for elasticsearch and graylog. Special thanks, Roberto -- You received this message because you are subscribed to the Google Groups graylog2 group. To unsubscribe from this group and stop receiving emails from it, send an email to graylog2+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
[graylog2] Two problems: processing streams disabled and login failure
Dear, I have Graylog as my syslog server with these packages: graylog2-server 0.20.6-1 graylog2-stream-dashboard 0.90.0-1 graylog2-web 0.20.6-1 In /etc/init.d/elasticsearch, I also add: *ES_MIN_MEM=2g* *ES_MAX_MEM=6g* *But I have two problems:* *1) I login Graylog web interface*, everything works OK using Firefox browser, I logout or stand login and the next day when I want to login in Graylog web interface, I get the error Sorry, those credentials are invalid. I check that graylog2-server and graylog2-web are running, and so I have to reboot the server to login succesfully.this situation occurs day after dayWhat could be the problem??? 2) Also day after day, I get the error Processing of stream has been disabled due to excesive processing time, and all the streas are paused until I resume them...can you help me pleae??? Thanks a lot, Roberto -- You received this message because you are subscribed to the Google Groups graylog2 group. To unsubscribe from this group and stop receiving emails from it, send an email to graylog2+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
Re: [graylog2] Logs from Cisco ASA with bad source field
Bernd, I've created a Raw INPUT as you said but after that all the sources from Windows servers are bad. So maybe I can correct de Cisco servers logs but I buy a new problem with my Windows servers. Is there any universal solution ? Maybe like Alejandro says, installing just a syslog-ng for cisco servers and forward the logs after that to graylog?? Thanks again, Roberto El lunes, 2 de marzo de 2015, 7:58:30 (UTC-3), Bernd Ahlers escribió: Roberto, you replace the Syslog input with a Raw input. The extractors are applied to the Raw input to parse the logs then. In your setup, remove the Syslog input and start a Raw input on the same port. Then add the extractors as described in the blog post I sent you earlier. Regards, Bernd On 27 February 2015 at 20:17, roberto...@gmail.com javascript: wrote: Dear Bernd, thanks for your helpful respondebut now I have a new question. I have a Graylog2 server with just one INPUT Syslog UDP listening on port UDP/10514, and the tutorial said I have to create another INPUT Raw suppose listening on port UDP/. How can I connect the raw input with the syslog input ??? I got lost... Thanks in advance, Roberto El viernes, 27 de febrero de 2015, 13:57:08 (UTC-3), Bernd Ahlers escribió: Roberto, the Cisco ASA does not send valid Syslog, unfortunately. You have to create a Raw input and create extractors. There is a blog post about this here: http://spottedhyena.co.uk/2015/01/graylog2-cisco-asa-cisco-catalyst/ Hope that helps! Regards, Bernd On 27 February 2015 at 15:57, roberto...@gmail.com wrote: Dear, I have a Graylog2 version 0.20.6 as our syslog server of our company. I defined an INPUT Syslog UDP running on port UDP/10514, and after that we point several Windows and Linux servers to the Graylog2 with no problems. But in the case of the Cisco ASA firewalls, we have a problem because the source sometimes matches something like: :%ASA-session-6-302013: In the Cisco ASA's I setup: logging enable logging emblem logging trap informational logging history debugging logging asdm debugging logging device-id hostname logging host inside_Frontend 10.1.1.1 format emblem I want to have the original hostname in the source field, so what can I do??? Regards, Roberto -- You received this message because you are subscribed to the Google Groups graylog2 group. To unsubscribe from this group and stop receiving emails from it, send an email to graylog2+u...@googlegroups.com. For more options, visit https://groups.google.com/d/optout. -- Developer Tel.: +49 (0)40 609 452 077 Fax.: +49 (0)40 609 452 078 TORCH GmbH - A Graylog company Steckelhörn 11 20457 Hamburg Germany Commercial Reg. (Registergericht): Amtsgericht Hamburg, HRB 125175 Geschäftsführer: Lennart Koopmann (CEO) -- You received this message because you are subscribed to the Google Groups graylog2 group. To unsubscribe from this group and stop receiving emails from it, send an email to graylog2+u...@googlegroups.com javascript:. For more options, visit https://groups.google.com/d/optout. -- Developer Tel.: +49 (0)40 609 452 077 Fax.: +49 (0)40 609 452 078 TORCH GmbH - A Graylog company Steckelhörn 11 20457 Hamburg Germany Commercial Reg. (Registergericht): Amtsgericht Hamburg, HRB 125175 Geschäftsführer: Lennart Koopmann (CEO) -- You received this message because you are subscribed to the Google Groups graylog2 group. To unsubscribe from this group and stop receiving emails from it, send an email to graylog2+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
Re: [graylog2] Logs from Cisco ASA with bad source field
Bernd, thanks a lot for your help... Now I understand what you tell me, but just a comment: When I created the new Syslog UDP INPUT, I chek the rDNS resolution option. Because a don't have configured an internal DNS for reverse resolution in my Graylog server, the source fields now are just IP's and not hostnamesthis is better than having thrash in the source field. I think this solution is good, but I'll try what you suggest. Thanks a lot, Roberto El lunes, 2 de marzo de 2015, 13:02:16 (UTC-3), Bernd Ahlers escribió: Roberto, ah, okay. Sorry, I didn't know that you have other machines reporting via Syslog. Then you should create the Syslog input again. Make sure that the Syslog and Raw input are not listening on the same port! So you either have to change the port on your Cisco ASA or on your windows machines. Regarding syslog-ng: You can install syslog-ng and forward the Cisco ASA messages via that one. But then you have to pre-process the messages in syslog-ng. Otherwise the same messages would arrive in Graylog. Regards, Bernd On 2 March 2015 at 16:47, roberto...@gmail.com javascript: wrote: Bernd, I've created a Raw INPUT as you said but after that all the sources from Windows servers are bad. So maybe I can correct de Cisco servers logs but I buy a new problem with my Windows servers. Is there any universal solution ? Maybe like Alejandro says, installing just a syslog-ng for cisco servers and forward the logs after that to graylog?? Thanks again, Roberto El lunes, 2 de marzo de 2015, 7:58:30 (UTC-3), Bernd Ahlers escribió: Roberto, you replace the Syslog input with a Raw input. The extractors are applied to the Raw input to parse the logs then. In your setup, remove the Syslog input and start a Raw input on the same port. Then add the extractors as described in the blog post I sent you earlier. Regards, Bernd On 27 February 2015 at 20:17, roberto...@gmail.com wrote: Dear Bernd, thanks for your helpful respondebut now I have a new question. I have a Graylog2 server with just one INPUT Syslog UDP listening on port UDP/10514, and the tutorial said I have to create another INPUT Raw suppose listening on port UDP/. How can I connect the raw input with the syslog input ??? I got lost... Thanks in advance, Roberto El viernes, 27 de febrero de 2015, 13:57:08 (UTC-3), Bernd Ahlers escribió: Roberto, the Cisco ASA does not send valid Syslog, unfortunately. You have to create a Raw input and create extractors. There is a blog post about this here: http://spottedhyena.co.uk/2015/01/graylog2-cisco-asa-cisco-catalyst/ Hope that helps! Regards, Bernd On 27 February 2015 at 15:57, roberto...@gmail.com wrote: Dear, I have a Graylog2 version 0.20.6 as our syslog server of our company. I defined an INPUT Syslog UDP running on port UDP/10514, and after that we point several Windows and Linux servers to the Graylog2 with no problems. But in the case of the Cisco ASA firewalls, we have a problem because the source sometimes matches something like: :%ASA-session-6-302013: In the Cisco ASA's I setup: logging enable logging emblem logging trap informational logging history debugging logging asdm debugging logging device-id hostname logging host inside_Frontend 10.1.1.1 format emblem I want to have the original hostname in the source field, so what can I do??? Regards, Roberto -- You received this message because you are subscribed to the Google Groups graylog2 group. To unsubscribe from this group and stop receiving emails from it, send an email to graylog2+u...@googlegroups.com. For more options, visit https://groups.google.com/d/optout. -- Developer Tel.: +49 (0)40 609 452 077 Fax.: +49 (0)40 609 452 078 TORCH GmbH - A Graylog company Steckelhörn 11 20457 Hamburg Germany Commercial Reg. (Registergericht): Amtsgericht Hamburg, HRB 125175 Geschäftsführer: Lennart Koopmann (CEO) -- You received this message because you are subscribed to the Google Groups graylog2 group. To unsubscribe from this group and stop receiving emails from it, send an email to graylog2+u...@googlegroups.com. For more options, visit https://groups.google.com/d/optout. -- Developer Tel.: +49 (0)40 609 452 077 Fax.: +49 (0)40 609 452 078 TORCH GmbH - A Graylog company Steckelhörn 11 20457 Hamburg Germany Commercial Reg. (Registergericht): Amtsgericht Hamburg, HRB 125175 Geschäftsführer: Lennart Koopmann (CEO) --
[graylog2] Logs from Cisco ASA with bad source field
Dear, I have a Graylog2 version 0.20.6 as our syslog server of our company. I defined an INPUT Syslog UDP running on port UDP/10514, and after that we point several Windows and Linux servers to the Graylog2 with no problems. But in the case of the Cisco ASA firewalls, we have a problem because the source sometimes matches something like: :%ASA-session-6-302013: In the Cisco ASA's I setup: logging enable logging emblem logging trap informational logging history debugging logging asdm debugging logging device-id hostname logging host inside_Frontend 10.1.1.1 format emblem I want to have the original hostname in the source field, so what can I do??? Regards, Roberto -- You received this message because you are subscribed to the Google Groups graylog2 group. To unsubscribe from this group and stop receiving emails from it, send an email to graylog2+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
[graylog2] Source field with bad format
Dear, I have Graylog 0.20.6. I receive logs from Linux and Windows servers very well, but my problem is with Cisco ASA logs, in the source field I receive something like this and not IP or hostname: Source: %ASA-6-100881 Source: %link-up-1 etc. What can I do in order to convert these sources in the corresponding IP's or hostnames ??? Thanks a lot, Roberto -- You received this message because you are subscribed to the Google Groups graylog2 group. To unsubscribe from this group and stop receiving emails from it, send an email to graylog2+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
