Hi Joi, This happens randomly after the graylog-collector has started (usually within a couple of hours of running). The logs being monitored do not have any rotation on them (they grow forever) so I don't think it is a rotation issue.
Any other ideas to try? I am stuck and I really need the graylog-collector to run on our AIX box. On Friday, April 8, 2016 at 9:35:23 AM UTC-7, Joi Owen wrote: > > What time does this normally happen? Is it after logrotate has run? Does > collector know it needs to re-open the logs after they're rotated? It may > be trying to read from the old inodes and doesn't realize it needs to > switch. > > It is possible to configure logrotate to deal with this on its own without > changing the inode. > > > On Fri, Apr 8, 2016 at 11:20 AM, Mirza Dedic <[email protected] > <javascript:>> wrote: > >> Hello, >> >> We are using graylog-collector v0.4.2 on AIX 7.1 and we can start the >> logger and it works for awhile (couple of hours) and then it suddenly stops >> sending logs with no error messages in the logs.. >> >> I also changed the COLLECTOR_JAVA_DEFAULTS to -Xms512m -Xmx768m but no >> luck. >> >> On the graylog-server the collector is visible while it is working, when >> it stops sending logs the collector is no longer visible on the server side >> but the graylog-collector process is still running in AIX with no errors in >> logs. I have to kill the process (kill -9 since kill -15 does not stop it) >> and then restart the process. >> >> Any idea how to troubleshoot the cause of this? >> >> *The collector.conf consists of this..* >> server-url = "http://172.16.x.x:12900"; >> enable-registration = true >> collector-id = "file:config/collector-id" >> >> inputs { >> 1 { >> type = "file" >> path = "/usr1/dbs/1.lg" >> outputs = "gelf-tcp" >> } >> 2 { >> type = "file" >> path = "/usr1/dbs/2.lg" >> outputs = "gelf-tcp" >> } >> 3 { >> type = "file" >> path = "/usr1/dbs/3.lg" >> outputs = "gelf-tcp" >> } >> 4 { >> type = "file" >> path = "/usr1/dbs/4.lg" >> outputs = "gelf-tcp" >> } >> 5 { >> type = "file" >> path = "/usr1/dbs/5.lg" >> outputs = "gelf-tcp" >> } >> 6 { >> type = "file" >> path = "/usr1/dbs/6.lg" >> outputs = "gelf-tcp" >> } >> 7 { >> type = "file" >> path = "/usr1/dbs/7.lg" >> outputs = "gelf-tcp" >> } >> 8 { >> type = "file" >> path = "/usr1/dbs/8.lg" >> outputs = "gelf-tcp" >> } >> } >> >> outputs { >> gelf-tcp { >> type = "gelf" >> host = "172.16.x.x" >> port = 12201 >> client-tls = false >> client-queue-size = 5120 >> client-connect-timeout = 10000 >> client-reconnect-delay = 2000 >> client-tcp-no-delay = true >> client-send-buffer-size = 32768 >> } >> } >> >> *In the log all we see is...* >> [main] cli.commands.Run - Starting Collector v0.4.2 (commit 2609a38) >> [main] cli.commands.Run - Running on AIX AIX 7.1 (ppc64) >> [main] collector.utils.CollectorId - Collector ID: >> 19c8fecb-0f04-4503-9943-15cfbac98458 >> [main] outputs.gelf.GelfOutput - Starting GELF transport: >> org.graylog2.gelfclient.GelfConfiguration@3c8510bd >> [main] cli.commands.Run - Service RUNNING: BufferProcessor [RUNNING] >> [main] cli.commands.Run - Service RUNNING: FileObserver [RUNNING] >> [main] cli.commands.Run - Service RUNNING: MetricService [RUNNING] >> [main] cli.commands.Run - Service RUNNING: MemoryReporterService [RUNNING] >> [main] cli.commands.Run - Service RUNNING: HeartbeatService [RUNNING] >> [main] cli.commands.Run - Service RUNNING: FileInput{id='1', >> path-set='SinglePathSet{path=/usr1/dbs/1.lg}', reader-buffer-size='102400', >> message-fields='MessageFields{}', charset='UTF-8', outputs='gelf-tcp', >> content-splitter='NEWLINE', reader-interval='100'} >> [main] cli.commands.Run - Service RUNNING: FileInput{id='2', >> path-set='SinglePathSet{path=/usr1/dbs/2.lg}', reader-buffer-size='102400', >> message-fields='MessageFields{}', charset='UTF-8', outputs='gelf-tcp', >> content-splitter='NEWLINE', reader-interval='100'} >> [main] cli.commands.Run - Service RUNNING: FileInput{id='3', >> path-set='SinglePathSet{path=/usr1/dbs/3.lg}', reader-buffer-size='102400', >> message-fields='MessageFields{}', charset='UTF-8', outputs='gelf-tcp', >> content-splitter='NEWLINE', reader-interval='100'} >> [main] cli.commands.Run - Service RUNNING: FileInput{id='4', >> path-set='SinglePathSet{path=/usr1/dbs/4.lg}', reader-buffer-size='102400', >> message-fields='MessageFields{}', charset='UTF-8', outputs='gelf-tcp', >> content-splitter='NEWLINE', reader-interval='100'} >> [main] cli.commands.Run - Service RUNNING: FileInput{id='5', >> path-set='SinglePathSet{path=/usr1/dbs/5.lg}', reader-buffer-size='102400', >> message-fields='MessageFields{}', charset='UTF-8', outputs='gelf-tcp', >> content-splitter='NEWLINE', reader-interval='100'} >> [main] cli.commands.Run - Service RUNNING: FileInput{id='6', >> path-set='SinglePathSet{path=/usr1/dbs/6.lg}', reader-buffer-size='102400', >> message-fields='MessageFields{}', charset='UTF-8', outputs='gelf-tcp', >> content-splitter='NEWLINE', reader-interval='100'} >> [main] cli.commands.Run - Service RUNNING: GelfOutput{port='12201', >> id='gelf-tcp', client-send-buffer-size='32768', host='172.16.x.x', >> inputs='', client-reconnect-delay='2000', client-connect-timeout='10000', >> client-tcp-no-delay='true', client-queue-size='5120'} >> [main] cli.commands.Run - Service RUNNING: FileInput{id='7', >> path-set='SinglePathSet{path=/usr1/dbs/7.lg}', reader-buffer-size='102400', >> message-fields='MessageFields{}', charset='UTF-8', outputs='gelf-tcp', >> content-splitter='NEWLINE', reader-interval='100'} >> [main] cli.commands.Run - Service RUNNING: FileInput{id='8', >> path-set='SinglePathSet{path=/usr1/dbs/8.lg}', reader-buffer-size='102400', >> message-fields='MessageFields{}', charset='UTF-8', outputs='gelf-tcp', >> content-splitter='NEWLINE', reader-interval='100'}', reader-interval='100'} >> >> >> -- >> You received this message because you are subscribed to the Google Groups >> "Graylog Users" group. >> To unsubscribe from this group and stop receiving emails from it, send an >> email to [email protected] <javascript:>. >> To view this discussion on the web visit >> https://groups.google.com/d/msgid/graylog2/3806ada3-9eb1-4126-a415-d7c0bf997ae1%40googlegroups.com >> >> <https://groups.google.com/d/msgid/graylog2/3806ada3-9eb1-4126-a415-d7c0bf997ae1%40googlegroups.com?utm_medium=email&utm_source=footer> >> . >> For more options, visit https://groups.google.com/d/optout. >> > > > > -- > > No matter what we think of Linux versus FreeBSD, etc., the one thing I > really like about Linux is that it has Microsoft worried. Anything > that kicks a monopoly in the pants has got to be good for something. > - Chris Johnson > > -- You received this message because you are subscribed to the Google Groups "Graylog Users" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To view this discussion on the web visit https://groups.google.com/d/msgid/graylog2/ad5bd1a9-bb37-40fa-9f46-2aa42cc874d7%40googlegroups.com. For more options, visit https://groups.google.com/d/optout.
