Hello, I am testing Graylog and enjoying the result so far, but I have a little problem with the Graylog Collector.
First, my test environment. A single server with: CentOS 7.1 graylog-server 1.2.0-5 gralog-web 1.20-5 ElasticSearch 1.7.2-1 mongodb-org 3.0.6-1 All installed via repositories, as the Graylog documentation. I'm getting the log of my mail server via UDP syslog. OK. In Graylog-server I created a GELF TCP input without TLS. The most basic possible. <https://lh3.googleusercontent.com/-CU0Mh4LS1Jw/VgKIbpIvTmI/AAAAAAAAXsU/RwDGZT_kQ-g/s1600/gelf.jpg> On my web server, I installed graylog-collector to collect the Apache files, so far without success. The logs are in */var/log/httpd*: -rw-r--r--. 1 root root 1672608 Set 23 07:24 access_log > -rw-r--r--. 1 root root 4562861 Ago 30 03:24 access_log-20150830 > -rw-r--r--. 1 root root 4022486 Set 6 03:28 access_log-20150906 > -rw-r--r--. 1 root root 3737037 Set 14 03:17 access_log-20150914 > -rw-r--r--. 1 root root 3614028 Set 20 03:49 access_log-20150920 > -rw-r--r--. 1 root root 556885 Set 23 07:25 cin_access_log > -rw-r--r--. 1 root root 2920062 Ago 30 03:18 cin_access_log-20150830 > -rw-r--r--. 1 root root 1712851 Set 6 03:11 cin_access_log-20150906 > -rw-r--r--. 1 root root 2491648 Set 14 03:16 cin_access_log-20150914 > -rw-r--r--. 1 root root 1008866 Set 20 03:46 cin_access_log-20150920 > -rw-r--r--. 1 root root 389809 Set 23 07:25 cin_error_log > -rw-r--r--. 1 root root 1785021 Ago 30 03:14 cin_error_log-20150830 > -rw-r--r--. 1 root root 953798 Set 6 03:11 cin_error_log-20150906 > -rw-r--r--. 1 root root 1573745 Set 14 03:16 cin_error_log-20150914 > -rw-r--r--. 1 root root 701103 Set 20 03:38 cin_error_log-20150920 > -rw-r--r--. 1 root root 35403621 Set 23 07:27 cnen_access_log > -rw-r--r--. 1 root root 89242524 Ago 30 03:23 cnen_access_log-20150830 > -rw-r--r--. 1 root root 80264588 Set 6 03:27 cnen_access_log-20150906 > -rw-r--r--. 1 root root 75353899 Set 14 03:17 cnen_access_log-20150914 > -rw-r--r--. 1 root root 71218462 Set 20 03:48 cnen_access_log-20150920 > -rw-r--r--. 1 root root 4348477 Set 23 07:26 cnen_error_log > -rw-r--r--. 1 root root 10175396 Ago 30 03:23 cnen_error_log-20150830 > -rw-r--r--. 1 root root 9243166 Set 6 03:27 cnen_error_log-20150906 > -rw-r--r--. 1 root root 9227132 Set 14 03:16 cnen_error_log-20150914 > -rw-r--r--. 1 root root 8543551 Set 20 03:48 cnen_error_log-20150920 > -rw-r--r--. 1 root root 501 Set 20 03:49 error_log > -rw-r--r--. 1 root root 618 Ago 30 03:24 error_log-20150830 > -rw-r--r--. 1 root root 781 Set 6 03:28 error_log-20150906 > -rw-r--r--. 1 root root 1496 Set 14 03:17 error_log-20150914 > -rw-r--r--. 1 root root 1628 Set 20 03:49 error_log-20150920 First, I tested this setting to capture the logs that start with "cnen": // Graylog Collector example configuration. // URL to REST API of Graylog server this collector registers at server-url = "http://xxx.xxx.xxx.xxx:12900"; // Enable registration with the Graylog server. (enabled by default) enable-registration = true collector-id = "file:/etc/graylog/collector/collector-id" inputs { apache-cnen { type = "file" path-glob-root = "/var/log/httpd" path-glob-pattern = "cnen_{access,error}_log*" } } outputs { gelf-1 { type = "gelf" host = "xxx.xxx.xxx.xxx" port = 12201 } // Prints all messages to STDOUT. Useful for debugging. Do not enable in production usage! console { type = "stdout" inputs = "apache-cnen" } } Then, I have this result in Graylog-collector log: 2015-09-23T06:46:31.907-0300 INFO [main] cli.commands.Run - Starting > Collector v0.4.1 (commit 36a0856) > 2015-09-23T06:46:31.912-0300 INFO [main] cli.commands.Run - Running on > CentOS Linux release 7.1.1503 (Core) Linux 3.10.0-229.7.2.el7.x86_64 > (amd64) > 2015-09-23T06:46:33.951-0300 INFO [main] collector.utils.CollectorId - > Collector ID: 16d46398-a0fa-49cd-b502-4a1917b247f7 > 2015-09-23T06:46:34.028-0300 INFO [main] collector.file.FileReaderService > - Configured files for input "apache-access" do not exist yet. They will be > followed once they are created. > 2015-09-23T06:46:34.033-0300 INFO [main] outputs.gelf.GelfOutput - > Starting GELF transport: org.graylog2.gelfclient.GelfConfiguration@6d23017e > 2015-09-23T06:46:35.516-0300 INFO [main] collector.file.FileReaderService > - Configured files for input "apache-error" do not exist yet. They will be > followed once they are created. > 2015-09-23T06:46:35.551-0300 INFO [main] cli.commands.Run - Service > RUNNING: MemoryReporterService [RUNNING] > 2015-09-23T06:46:35.551-0300 INFO [main] cli.commands.Run - Service > RUNNING: BufferProcessor [RUNNING] > 2015-09-23T06:46:35.552-0300 INFO [main] cli.commands.Run - Service > RUNNING: FileObserver [RUNNING] > 2015-09-23T06:46:35.552-0300 INFO [main] cli.commands.Run - Service > RUNNING: MetricService [RUNNING] > 2015-09-23T06:46:35.553-0300 INFO [main] cli.commands.Run - Service > RUNNING: StdoutOutput{inputs='apache-access', id='console'} > 2015-09-23T06:46:35.554-0300 INFO [main] cli.commands.Run - Service > RUNNING: HeartbeatService [RUNNING] > 2015-09-23T06:46:35.557-0300 INFO [main] cli.commands.Run - Service > RUNNING: FileInput{outputs='', content-splitter='NEWLINE', charset='UTF-8', > message-fields='MessageFields{}', reader-buffer-size='102400', > reader-interval='100', id='apache-access', > path-set='SinglePathSet{path=/var/log/httpd/cnen_access_log}'} > 2015-09-23T06:46:35.558-0300 INFO [main] cli.commands.Run - Service > RUNNING: GelfOutput{client-send-buffer-size='-1', port='12201', inputs='', > host='200.156.7.166', client-reconnect-delay='1000', > client-tcp-no-delay='true', id='gelf-1', client-queue-size='512', > client-connect-timeout='5000'} > 2015-09-23T06:46:35.559-0300 INFO [main] cli.commands.Run - Service > RUNNING: FileInput{outputs='', content-splitter='NEWLINE', charset='UTF-8', > message-fields='MessageFields{}', reader-buffer-size='102400', > reader-interval='100', id='apache-error', > path-set='SinglePathSet{path=/var/log/httpd/cnen_error_log}'} > 2015-09-23T07:25:18.432-0300 INFO [Thread-1] cli.commands.Run - > Stopping... > 2015-09-23T07:25:23.561-0300 INFO [main] cli.commands.Run - Starting > Collector v0.4.1 (commit 36a0856) > 2015-09-23T07:25:23.565-0300 INFO [main] cli.commands.Run - Running on > CentOS Linux release 7.1.1503 (Core) Linux 3.10.0-229.7.2.el7.x86_64 > (amd64) > 2015-09-23T07:25:25.386-0300 INFO [main] collector.utils.CollectorId - > Collector ID: 16d46398-a0fa-49cd-b502-4a1917b247f7 > 2015-09-23T07:25:25.449-0300 WARN [main] collector.file.GlobPathSet - > Unable to change into directory /var/log/httpd/cnen_access_log-20150920 - > Check permissions > 2015-09-23T07:25:25.449-0300 WARN [main] collector.file.GlobPathSet - > Unable to change into directory /var/log/httpd/error_log-20150920 - Check > permissions > 2015-09-23T07:25:25.450-0300 WARN [main] collector.file.GlobPathSet - > Unable to change into directory /var/log/httpd/access_log-20150830 - Check > permissions > 2015-09-23T07:25:25.450-0300 WARN [main] collector.file.GlobPathSet - > Unable to change into directory /var/log/httpd/cin_access_log-20150830 - > Check permissions > 2015-09-23T07:25:25.450-0300 WARN [main] collector.file.GlobPathSet - > Unable to change into directory /var/log/httpd/cin_error_log-20150830 - > Check permissions > 2015-09-23T07:25:25.450-0300 WARN [main] collector.file.GlobPathSet - > Unable to change into directory /var/log/httpd/cnen_access_log-20150830 - > Check permissions > 2015-09-23T07:25:25.450-0300 WARN [main] collector.file.GlobPathSet - > Unable to change into directory /var/log/httpd/error_log-20150830 - Check > permissions > 2015-09-23T07:25:25.451-0300 WARN [main] collector.file.GlobPathSet - > Unable to change into directory /var/log/httpd/access_log-20150906 - Check > permissions > 2015-09-23T07:25:25.451-0300 WARN [main] collector.file.GlobPathSet - > Unable to change into directory /var/log/httpd/cin_access_log-20150906 - > Check permissions > 2015-09-23T07:25:25.451-0300 WARN [main] collector.file.GlobPathSet - > Unable to change into directory /var/log/httpd/cin_error_log-20150906 - > Check permissions > 2015-09-23T07:25:25.451-0300 WARN [main] collector.file.GlobPathSet - > Unable to change into directory /var/log/httpd/error_log-20150906 - Check > permissions > 2015-09-23T07:25:25.452-0300 WARN [main] collector.file.GlobPathSet - > Unable to change into directory /var/log/httpd/access_log-20150914 - Check > permissions > 2015-09-23T07:25:25.452-0300 WARN [main] collector.file.GlobPathSet - > Unable to change into directory /var/log/httpd/cin_access_log-20150914 - > Check permissions > 2015-09-23T07:25:25.452-0300 WARN [main] collector.file.GlobPathSet - > Unable to change into directory /var/log/httpd/cin_error_log-20150914 - > Check permissions > 2015-09-23T07:25:25.452-0300 WARN [main] collector.file.GlobPathSet - > Unable to change into directory /var/log/httpd/cnen_access_log-20150914 - > Check permissions > 2015-09-23T07:25:25.452-0300 WARN [main] collector.file.GlobPathSet - > Unable to change into directory /var/log/httpd/cnen_error_log-20150914 - > Check permissions > 2015-09-23T07:25:25.453-0300 WARN [main] collector.file.GlobPathSet - > Unable to change into directory /var/log/httpd/error_log-20150914 - Check > permissions > 2015-09-23T07:25:25.453-0300 WARN [main] collector.file.GlobPathSet - > Unable to change into directory /var/log/httpd/access_log-20150920 - Check > permissions > 2015-09-23T07:25:25.453-0300 WARN [main] collector.file.GlobPathSet - > Unable to change into directory /var/log/httpd/cin_access_log-20150920 - > Check permissions > 2015-09-23T07:25:25.453-0300 WARN [main] collector.file.GlobPathSet - > Unable to change into directory /var/log/httpd/cin_error_log-20150920 - > Check permissions > 2015-09-23T07:25:25.454-0300 WARN [main] collector.file.GlobPathSet - > Unable to change into directory /var/log/httpd/cnen_error_log-20150920 - > Check permissions > 2015-09-23T07:25:25.454-0300 WARN [main] collector.file.GlobPathSet - > Unable to change into directory /var/log/httpd/error_log - Check permissions > 2015-09-23T07:25:25.454-0300 WARN [main] collector.file.GlobPathSet - > Unable to change into directory /var/log/httpd/cnen_error_log-20150830 - > Check permissions > 2015-09-23T07:25:25.454-0300 WARN [main] collector.file.GlobPathSet - > Unable to change into directory /var/log/httpd/cnen_access_log-20150906 - > Check permissions > 2015-09-23T07:25:25.473-0300 WARN [main] collector.file.GlobPathSet - > Unable to change into directory /var/log/httpd/cnen_error_log-20150906 - > Check permissions > 2015-09-23T07:25:25.473-0300 WARN [main] collector.file.GlobPathSet - > Unable to change into directory /var/log/httpd/access_log - Check > permissions > 2015-09-23T07:25:25.474-0300 WARN [main] collector.file.GlobPathSet - > Unable to change into directory /var/log/httpd/cin_access_log - Check > permissions > 2015-09-23T07:25:25.474-0300 WARN [main] collector.file.GlobPathSet - > Unable to change into directory /var/log/httpd/cin_error_log - Check > permissions > 2015-09-23T07:25:25.474-0300 WARN [main] collector.file.GlobPathSet - > Unable to change into directory /var/log/httpd/cnen_access_log - Check > permissions > 2015-09-23T07:25:25.474-0300 WARN [main] collector.file.GlobPathSet - > Unable to change into directory /var/log/httpd/cnen_error_log - Check > permissions > *2015-09-23T07:25:25.475-0300 INFO [main] > collector.file.FileReaderService - Configured files for input "apache-cnen" > do not exist yet. They will be followed once they are created.* > 2015-09-23T07:25:25.511-0300 INFO [main] outputs.gelf.GelfOutput - > Starting GELF transport: org.graylog2.gelfclient.GelfConfiguration@17a1e4ca > 2015-09-23T07:25:25.931-0300 INFO [main] cli.commands.Run - Service > RUNNING: BufferProcessor [RUNNING] > 2015-09-23T07:25:25.931-0300 INFO [main] cli.commands.Run - Service > RUNNING: FileObserver [RUNNING] > 2015-09-23T07:25:25.931-0300 INFO [main] cli.commands.Run - Service > RUNNING: MemoryReporterService [RUNNING] > 2015-09-23T07:25:25.931-0300 INFO [main] cli.commands.Run - Service > RUNNING: MetricService [RUNNING] > 2015-09-23T07:25:25.933-0300 INFO [main] cli.commands.Run - Service > RUNNING: StdoutOutput{inputs='apache-access', id='console'} > 2015-09-23T07:25:25.933-0300 INFO [main] cli.commands.Run - Service > RUNNING: HeartbeatService [RUNNING] > 2015-09-23T07:25:25.936-0300 INFO [main] cli.commands.Run - Service > RUNNING: FileInput{outputs='', content-splitter='NEWLINE', charset='UTF-8', > message-fields='MessageFields{}', reader-buffer-size='102400', > reader-interval='100', id='apache-cnen', > path-set='GlobPathSet{rootPath=/var/log/httpd, > pattern=cnen_{access,error}_log*}'} > 2015-09-23T07:25:25.951-0300 INFO [main] cli.commands.Run - Service > RUNNING: GelfOutput{client-send-buffer-size='-1', port='12201', inputs='', > host='200.156.7.166', client-reconnect-delay='1000', > client-tcp-no-delay='true', id='gelf-1', client-queue-size='512', > client-connect-timeout='5000'} What does the bold line? I tried to change to a single file: inputs { apache-cnen { type = "file" path = "/var/log/httpd/cnen_access_log" } } And the result: 2015-09-23T07:56:59.450-0300 INFO [main] cli.commands.Run - Starting > Collector v0.4.1 (commit 36a0856) > 2015-09-23T07:56:59.454-0300 INFO [main] cli.commands.Run - Running on > CentOS Linux release 7.1.1503 (Core) Linux 3.10.0-229.7.2.el7.x86_64 > (amd64) > 2015-09-23T07:57:01.415-0300 INFO [main] collector.utils.CollectorId - > Collector ID: 16d46398-a0fa-49cd-b502-4a1917b247f7 > 2015-09-23T07:57:01.450-0300 INFO [main] outputs.gelf.GelfOutput - > Starting GELF transport: org.graylog2.gelfclient.GelfConfiguration@63dd899 > *2015-09-23T07:57:01.995-0300 INFO [main] > collector.file.FileReaderService - Configured files for input "apache-cnen" > do not exist yet. They will be followed once they are created.* > 2015-09-23T07:57:02.011-0300 INFO [main] cli.commands.Run - Service > RUNNING: BufferProcessor [RUNNING] > 2015-09-23T07:57:02.011-0300 INFO [main] cli.commands.Run - Service > RUNNING: FileObserver [RUNNING] > 2015-09-23T07:57:02.011-0300 INFO [main] cli.commands.Run - Service > RUNNING: MemoryReporterService [RUNNING] > 2015-09-23T07:57:02.011-0300 INFO [main] cli.commands.Run - Service > RUNNING: MetricService [RUNNING] > 2015-09-23T07:57:02.011-0300 INFO [main] cli.commands.Run - Service > RUNNING: HeartbeatService [RUNNING] > 2015-09-23T07:57:02.014-0300 INFO [main] cli.commands.Run - Service > RUNNING: GelfOutput{client-send-buffer-size='-1', port='12201', inputs='', > host='200.156.7.166', client-reconnect-delay='1000', > client-tcp-no-delay='true', id='gelf-1', client-queue-size='512', > client-connect-timeout='5000'} > 2015-09-23T07:57:02.017-0300 INFO [main] cli.commands.Run - Service > RUNNING: FileInput{outputs='', content-splitter='NEWLINE', charset='UTF-8', > message-fields='MessageFields{}', reader-buffer-size='102400', > reader-interval='100', id='apache-cnen', > path-set='SinglePathSet{path=/var/log/httpd/cnen_access_log}'} > 2015-09-23T07:57:02.017-0300 INFO [main] cli.commands.Run - Service > RUNNING: StdoutOutput{inputs='apache-access', id='console'} Again, the same bold line... Sorry folks, but where I am missing? :( Some kind of special permission for graylog-collector user can read the Apache logs is necessary? Thanks, Emerson -- You received this message because you are subscribed to the Google Groups "Graylog Users" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To view this discussion on the web visit https://groups.google.com/d/msgid/graylog2/30c4ec94-763e-4653-8690-9378f380ace4%40googlegroups.com. For more options, visit https://groups.google.com/d/optout.
