Hi Emerson,
please verify that the Graylog Collector has sufficient permissions to read
files in the /var/log/httpd/ directory. Usually there's a dedicated group
(e. g. log or adm, or the Apache httpd user apache or httpd) which is
allowed to read those files. You can add the Graylog Collector user to one
of those groups with the following command (replace $GROUP with the actual
name of the group):
gpasswd -a graylog-collector $GROUP
Of course you can also run the Graylog Collector with root privileges, but
I wouldn't advise to do so for security reasons.
Additionally, the Graylog Collector currently doesn't support reading and
processing old log files, so the path-glob-pattern should probably be
"cnen_{access,error}_log" (without the wildcard at the end).
Cheers,
Jochen
On Wednesday, 23 September 2015 13:14:48 UTC+2, Emerson Coimbra wrote:
>
> Hello,
>
> I am testing Graylog and enjoying the result so far, but I have a little
> problem with the Graylog Collector.
>
> First, my test environment. A single server with:
> CentOS 7.1
> graylog-server 1.2.0-5
> gralog-web 1.20-5
> ElasticSearch 1.7.2-1
> mongodb-org 3.0.6-1
>
> All installed via repositories, as the Graylog documentation.
>
> I'm getting the log of my mail server via UDP syslog. OK.
>
>
> In Graylog-server I created a GELF TCP input without TLS. The most basic
> possible.
>
>
> <https://lh3.googleusercontent.com/-CU0Mh4LS1Jw/VgKIbpIvTmI/AAAAAAAAXsU/RwDGZT_kQ-g/s1600/gelf.jpg>
>
>
> On my web server, I installed graylog-collector to collect the Apache
> files, so far without success. The logs are in */var/log/httpd*:
>
> -rw-r--r--. 1 root root 1672608 Set 23 07:24 access_log
>> -rw-r--r--. 1 root root 4562861 Ago 30 03:24 access_log-20150830
>> -rw-r--r--. 1 root root 4022486 Set 6 03:28 access_log-20150906
>> -rw-r--r--. 1 root root 3737037 Set 14 03:17 access_log-20150914
>> -rw-r--r--. 1 root root 3614028 Set 20 03:49 access_log-20150920
>> -rw-r--r--. 1 root root 556885 Set 23 07:25 cin_access_log
>> -rw-r--r--. 1 root root 2920062 Ago 30 03:18 cin_access_log-20150830
>> -rw-r--r--. 1 root root 1712851 Set 6 03:11 cin_access_log-20150906
>> -rw-r--r--. 1 root root 2491648 Set 14 03:16 cin_access_log-20150914
>> -rw-r--r--. 1 root root 1008866 Set 20 03:46 cin_access_log-20150920
>> -rw-r--r--. 1 root root 389809 Set 23 07:25 cin_error_log
>> -rw-r--r--. 1 root root 1785021 Ago 30 03:14 cin_error_log-20150830
>> -rw-r--r--. 1 root root 953798 Set 6 03:11 cin_error_log-20150906
>> -rw-r--r--. 1 root root 1573745 Set 14 03:16 cin_error_log-20150914
>> -rw-r--r--. 1 root root 701103 Set 20 03:38 cin_error_log-20150920
>> -rw-r--r--. 1 root root 35403621 Set 23 07:27 cnen_access_log
>> -rw-r--r--. 1 root root 89242524 Ago 30 03:23 cnen_access_log-20150830
>> -rw-r--r--. 1 root root 80264588 Set 6 03:27 cnen_access_log-20150906
>> -rw-r--r--. 1 root root 75353899 Set 14 03:17 cnen_access_log-20150914
>> -rw-r--r--. 1 root root 71218462 Set 20 03:48 cnen_access_log-20150920
>> -rw-r--r--. 1 root root 4348477 Set 23 07:26 cnen_error_log
>> -rw-r--r--. 1 root root 10175396 Ago 30 03:23 cnen_error_log-20150830
>> -rw-r--r--. 1 root root 9243166 Set 6 03:27 cnen_error_log-20150906
>> -rw-r--r--. 1 root root 9227132 Set 14 03:16 cnen_error_log-20150914
>> -rw-r--r--. 1 root root 8543551 Set 20 03:48 cnen_error_log-20150920
>> -rw-r--r--. 1 root root 501 Set 20 03:49 error_log
>> -rw-r--r--. 1 root root 618 Ago 30 03:24 error_log-20150830
>> -rw-r--r--. 1 root root 781 Set 6 03:28 error_log-20150906
>> -rw-r--r--. 1 root root 1496 Set 14 03:17 error_log-20150914
>> -rw-r--r--. 1 root root 1628 Set 20 03:49 error_log-20150920
>
>
>
>
> First, I tested this setting to capture the logs that start with "cnen":
>
> // Graylog Collector example configuration.
>
>
> // URL to REST API of Graylog server this collector registers at
> server-url = "http://xxx.xxx.xxx.xxx:12900";
>
> // Enable registration with the Graylog server. (enabled by default)
> enable-registration = true
>
> collector-id = "file:/etc/graylog/collector/collector-id"
>
> inputs {
> apache-cnen {
> type = "file"
> path-glob-root = "/var/log/httpd"
> path-glob-pattern = "cnen_{access,error}_log*"
> }
> }
>
> outputs {
> gelf-1 {
> type = "gelf"
> host = "xxx.xxx.xxx.xxx"
> port = 12201
> }
>
> // Prints all messages to STDOUT. Useful for debugging. Do not enable in
> production usage!
> console {
> type = "stdout"
> inputs = "apache-cnen"
> }
> }
>
>
> Then, I have this result in Graylog-collector log:
>
> 2015-09-23T06:46:31.907-0300 INFO [main] cli.commands.Run - Starting
>> Collector v0.4.1 (commit 36a0856)
>> 2015-09-23T06:46:31.912-0300 INFO [main] cli.commands.Run - Running on
>> CentOS Linux release 7.1.1503 (Core) Linux 3.10.0-229.7.2.el7.x86_64
>> (amd64)
>> 2015-09-23T06:46:33.951-0300 INFO [main] collector.utils.CollectorId -
>> Collector ID: 16d46398-a0fa-49cd-b502-4a1917b247f7
>> 2015-09-23T06:46:34.028-0300 INFO [main]
>> collector.file.FileReaderService - Configured files for input
>> "apache-access" do not exist yet. They will be followed once they are
>> created.
>> 2015-09-23T06:46:34.033-0300 INFO [main] outputs.gelf.GelfOutput -
>> Starting GELF transport: org.graylog2.gelfclient.GelfConfiguration@6d23017e
>> 2015-09-23T06:46:35.516-0300 INFO [main]
>> collector.file.FileReaderService - Configured files for input
>> "apache-error" do not exist yet. They will be followed once they are
>> created.
>> 2015-09-23T06:46:35.551-0300 INFO [main] cli.commands.Run - Service
>> RUNNING: MemoryReporterService [RUNNING]
>> 2015-09-23T06:46:35.551-0300 INFO [main] cli.commands.Run - Service
>> RUNNING: BufferProcessor [RUNNING]
>> 2015-09-23T06:46:35.552-0300 INFO [main] cli.commands.Run - Service
>> RUNNING: FileObserver [RUNNING]
>> 2015-09-23T06:46:35.552-0300 INFO [main] cli.commands.Run - Service
>> RUNNING: MetricService [RUNNING]
>> 2015-09-23T06:46:35.553-0300 INFO [main] cli.commands.Run - Service
>> RUNNING: StdoutOutput{inputs='apache-access', id='console'}
>> 2015-09-23T06:46:35.554-0300 INFO [main] cli.commands.Run - Service
>> RUNNING: HeartbeatService [RUNNING]
>> 2015-09-23T06:46:35.557-0300 INFO [main] cli.commands.Run - Service
>> RUNNING: FileInput{outputs='', content-splitter='NEWLINE', charset='UTF-8',
>> message-fields='MessageFields{}', reader-buffer-size='102400',
>> reader-interval='100', id='apache-access',
>> path-set='SinglePathSet{path=/var/log/httpd/cnen_access_log}'}
>> 2015-09-23T06:46:35.558-0300 INFO [main] cli.commands.Run - Service
>> RUNNING: GelfOutput{client-send-buffer-size='-1', port='12201', inputs='',
>> host='200.156.7.166', client-reconnect-delay='1000',
>> client-tcp-no-delay='true', id='gelf-1', client-queue-size='512',
>> client-connect-timeout='5000'}
>> 2015-09-23T06:46:35.559-0300 INFO [main] cli.commands.Run - Service
>> RUNNING: FileInput{outputs='', content-splitter='NEWLINE', charset='UTF-8',
>> message-fields='MessageFields{}', reader-buffer-size='102400',
>> reader-interval='100', id='apache-error',
>> path-set='SinglePathSet{path=/var/log/httpd/cnen_error_log}'}
>> 2015-09-23T07:25:18.432-0300 INFO [Thread-1] cli.commands.Run -
>> Stopping...
>> 2015-09-23T07:25:23.561-0300 INFO [main] cli.commands.Run - Starting
>> Collector v0.4.1 (commit 36a0856)
>> 2015-09-23T07:25:23.565-0300 INFO [main] cli.commands.Run - Running on
>> CentOS Linux release 7.1.1503 (Core) Linux 3.10.0-229.7.2.el7.x86_64
>> (amd64)
>> 2015-09-23T07:25:25.386-0300 INFO [main] collector.utils.CollectorId -
>> Collector ID: 16d46398-a0fa-49cd-b502-4a1917b247f7
>> 2015-09-23T07:25:25.449-0300 WARN [main] collector.file.GlobPathSet -
>> Unable to change into directory /var/log/httpd/cnen_access_log-20150920 -
>> Check permissions
>> 2015-09-23T07:25:25.449-0300 WARN [main] collector.file.GlobPathSet -
>> Unable to change into directory /var/log/httpd/error_log-20150920 - Check
>> permissions
>> 2015-09-23T07:25:25.450-0300 WARN [main] collector.file.GlobPathSet -
>> Unable to change into directory /var/log/httpd/access_log-20150830 - Check
>> permissions
>> 2015-09-23T07:25:25.450-0300 WARN [main] collector.file.GlobPathSet -
>> Unable to change into directory /var/log/httpd/cin_access_log-20150830 -
>> Check permissions
>> 2015-09-23T07:25:25.450-0300 WARN [main] collector.file.GlobPathSet -
>> Unable to change into directory /var/log/httpd/cin_error_log-20150830 -
>> Check permissions
>> 2015-09-23T07:25:25.450-0300 WARN [main] collector.file.GlobPathSet -
>> Unable to change into directory /var/log/httpd/cnen_access_log-20150830 -
>> Check permissions
>> 2015-09-23T07:25:25.450-0300 WARN [main] collector.file.GlobPathSet -
>> Unable to change into directory /var/log/httpd/error_log-20150830 - Check
>> permissions
>> 2015-09-23T07:25:25.451-0300 WARN [main] collector.file.GlobPathSet -
>> Unable to change into directory /var/log/httpd/access_log-20150906 - Check
>> permissions
>> 2015-09-23T07:25:25.451-0300 WARN [main] collector.file.GlobPathSet -
>> Unable to change into directory /var/log/httpd/cin_access_log-20150906 -
>> Check permissions
>> 2015-09-23T07:25:25.451-0300 WARN [main] collector.file.GlobPathSet -
>> Unable to change into directory /var/log/httpd/cin_error_log-20150906 -
>> Check permissions
>> 2015-09-23T07:25:25.451-0300 WARN [main] collector.file.GlobPathSet -
>> Unable to change into directory /var/log/httpd/error_log-20150906 - Check
>> permissions
>> 2015-09-23T07:25:25.452-0300 WARN [main] collector.file.GlobPathSet -
>> Unable to change into directory /var/log/httpd/access_log-20150914 - Check
>> permissions
>> 2015-09-23T07:25:25.452-0300 WARN [main] collector.file.GlobPathSet -
>> Unable to change into directory /var/log/httpd/cin_access_log-20150914 -
>> Check permissions
>> 2015-09-23T07:25:25.452-0300 WARN [main] collector.file.GlobPathSet -
>> Unable to change into directory /var/log/httpd/cin_error_log-20150914 -
>> Check permissions
>> 2015-09-23T07:25:25.452-0300 WARN [main] collector.file.GlobPathSet -
>> Unable to change into directory /var/log/httpd/cnen_access_log-20150914 -
>> Check permissions
>> 2015-09-23T07:25:25.452-0300 WARN [main] collector.file.GlobPathSet -
>> Unable to change into directory /var/log/httpd/cnen_error_log-20150914 -
>> Check permissions
>> 2015-09-23T07:25:25.453-0300 WARN [main] collector.file.GlobPathSet -
>> Unable to change into directory /var/log/httpd/error_log-20150914 - Check
>> permissions
>> 2015-09-23T07:25:25.453-0300 WARN [main] collector.file.GlobPathSet -
>> Unable to change into directory /var/log/httpd/access_log-20150920 - Check
>> permissions
>> 2015-09-23T07:25:25.453-0300 WARN [main] collector.file.GlobPathSet -
>> Unable to change into directory /var/log/httpd/cin_access_log-20150920 -
>> Check permissions
>> 2015-09-23T07:25:25.453-0300 WARN [main] collector.file.GlobPathSet -
>> Unable to change into directory /var/log/httpd/cin_error_log-20150920 -
>> Check permissions
>> 2015-09-23T07:25:25.454-0300 WARN [main] collector.file.GlobPathSet -
>> Unable to change into directory /var/log/httpd/cnen_error_log-20150920 -
>> Check permissions
>> 2015-09-23T07:25:25.454-0300 WARN [main] collector.file.GlobPathSet -
>> Unable to change into directory /var/log/httpd/error_log - Check permissions
>> 2015-09-23T07:25:25.454-0300 WARN [main] collector.file.GlobPathSet -
>> Unable to change into directory /var/log/httpd/cnen_error_log-20150830 -
>> Check permissions
>> 2015-09-23T07:25:25.454-0300 WARN [main] collector.file.GlobPathSet -
>> Unable to change into directory /var/log/httpd/cnen_access_log-20150906 -
>> Check permissions
>> 2015-09-23T07:25:25.473-0300 WARN [main] collector.file.GlobPathSet -
>> Unable to change into directory /var/log/httpd/cnen_error_log-20150906 -
>> Check permissions
>> 2015-09-23T07:25:25.473-0300 WARN [main] collector.file.GlobPathSet -
>> Unable to change into directory /var/log/httpd/access_log - Check
>> permissions
>> 2015-09-23T07:25:25.474-0300 WARN [main] collector.file.GlobPathSet -
>> Unable to change into directory /var/log/httpd/cin_access_log - Check
>> permissions
>> 2015-09-23T07:25:25.474-0300 WARN [main] collector.file.GlobPathSet -
>> Unable to change into directory /var/log/httpd/cin_error_log - Check
>> permissions
>> 2015-09-23T07:25:25.474-0300 WARN [main] collector.file.GlobPathSet -
>> Unable to change into directory /var/log/httpd/cnen_access_log - Check
>> permissions
>> 2015-09-23T07:25:25.474-0300 WARN [main] collector.file.GlobPathSet -
>> Unable to change into directory /var/log/httpd/cnen_error_log - Check
>> permissions
>> *2015-09-23T07:25:25.475-0300 INFO [main]
>> collector.file.FileReaderService - Configured files for input "apache-cnen"
>> do not exist yet. They will be followed once they are created.*
>> 2015-09-23T07:25:25.511-0300 INFO [main] outputs.gelf.GelfOutput -
>> Starting GELF transport: org.graylog2.gelfclient.GelfConfiguration@17a1e4ca
>> 2015-09-23T07:25:25.931-0300 INFO [main] cli.commands.Run - Service
>> RUNNING: BufferProcessor [RUNNING]
>> 2015-09-23T07:25:25.931-0300 INFO [main] cli.commands.Run - Service
>> RUNNING: FileObserver [RUNNING]
>> 2015-09-23T07:25:25.931-0300 INFO [main] cli.commands.Run - Service
>> RUNNING: MemoryReporterService [RUNNING]
>> 2015-09-23T07:25:25.931-0300 INFO [main] cli.commands.Run - Service
>> RUNNING: MetricService [RUNNING]
>> 2015-09-23T07:25:25.933-0300 INFO [main] cli.commands.Run - Service
>> RUNNING: StdoutOutput{inputs='apache-access', id='console'}
>> 2015-09-23T07:25:25.933-0300 INFO [main] cli.commands.Run - Service
>> RUNNING: HeartbeatService [RUNNING]
>> 2015-09-23T07:25:25.936-0300 INFO [main] cli.commands.Run - Service
>> RUNNING: FileInput{outputs='', content-splitter='NEWLINE', charset='UTF-8',
>> message-fields='MessageFields{}', reader-buffer-size='102400',
>> reader-interval='100', id='apache-cnen',
>> path-set='GlobPathSet{rootPath=/var/log/httpd,
>> pattern=cnen_{access,error}_log*}'}
>> 2015-09-23T07:25:25.951-0300 INFO [main] cli.commands.Run - Service
>> RUNNING: GelfOutput{client-send-buffer-size='-1', port='12201', inputs='',
>> host='200.156.7.166', client-reconnect-delay='1000',
>> client-tcp-no-delay='true', id='gelf-1', client-queue-size='512',
>> client-connect-timeout='5000'}
>
>
> What does the bold line?
>
> I tried to change to a single file:
>
> inputs {
> apache-cnen {
> type = "file"
> path = "/var/log/httpd/cnen_access_log"
> }
> }
>
>
> And the result:
>
> 2015-09-23T07:56:59.450-0300 INFO [main] cli.commands.Run - Starting
>> Collector v0.4.1 (commit 36a0856)
>> 2015-09-23T07:56:59.454-0300 INFO [main] cli.commands.Run - Running on
>> CentOS Linux release 7.1.1503 (Core) Linux 3.10.0-229.7.2.el7.x86_64
>> (amd64)
>> 2015-09-23T07:57:01.415-0300 INFO [main] collector.utils.CollectorId -
>> Collector ID: 16d46398-a0fa-49cd-b502-4a1917b247f7
>> 2015-09-23T07:57:01.450-0300 INFO [main] outputs.gelf.GelfOutput -
>> Starting GELF transport: org.graylog2.gelfclient.GelfConfiguration@63dd899
>> *2015-09-23T07:57:01.995-0300 INFO [main]
>> collector.file.FileReaderService - Configured files for input "apache-cnen"
>> do not exist yet. They will be followed once they are created.*
>> 2015-09-23T07:57:02.011-0300 INFO [main] cli.commands.Run - Service
>> RUNNING: BufferProcessor [RUNNING]
>> 2015-09-23T07:57:02.011-0300 INFO [main] cli.commands.Run - Service
>> RUNNING: FileObserver [RUNNING]
>> 2015-09-23T07:57:02.011-0300 INFO [main] cli.commands.Run - Service
>> RUNNING: MemoryReporterService [RUNNING]
>> 2015-09-23T07:57:02.011-0300 INFO [main] cli.commands.Run - Service
>> RUNNING: MetricService [RUNNING]
>> 2015-09-23T07:57:02.011-0300 INFO [main] cli.commands.Run - Service
>> RUNNING: HeartbeatService [RUNNING]
>> 2015-09-23T07:57:02.014-0300 INFO [main] cli.commands.Run - Service
>> RUNNING: GelfOutput{client-send-buffer-size='-1', port='12201', inputs='',
>> host='200.156.7.166', client-reconnect-delay='1000',
>> client-tcp-no-delay='true', id='gelf-1', client-queue-size='512',
>> client-connect-timeout='5000'}
>> 2015-09-23T07:57:02.017-0300 INFO [main] cli.commands.Run - Service
>> RUNNING: FileInput{outputs='', content-splitter='NEWLINE', charset='UTF-8',
>> message-fields='MessageFields{}', reader-buffer-size='102400',
>> reader-interval='100', id='apache-cnen',
>> path-set='SinglePathSet{path=/var/log/httpd/cnen_access_log}'}
>> 2015-09-23T07:57:02.017-0300 INFO [main] cli.commands.Run - Service
>> RUNNING: StdoutOutput{inputs='apache-access', id='console'}
>
>
> Again, the same bold line... Sorry folks, but where I am missing? :(
>
> Some kind of special permission for graylog-collector user can read the
> Apache logs is necessary?
>
>
>
> Thanks,
> Emerson
>
>
>
--
You received this message because you are subscribed to the Google Groups
"Graylog Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to [email protected].
To view this discussion on the web visit
https://groups.google.com/d/msgid/graylog2/2bf88600-c504-47be-aea1-9b5bb25b2bbe%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
