Hey there,
Ive got a big issue with malformed date format, so the nginx extractor is
rejecting incoming messages and the dashboards are empty.
My setup:
Debian 7.9
graylog-server 1.3.0-3
graylog-web 1.3.0-3
java 1.8.0.66
In the nginx site of my webserver I configured this log_format:
log_format graylog2_format '$remote_addr - $remote_user [$time_local]
"$request" $status $body_bytes_sent "$http_referer" "$http_user_agent"
"$http_x_forwarded_for"
<msec=$msec|connection=$connection|connection_requests=$connection_requests|millis=$request_time>
"$host"';
Example:
XX.XX.XXX.XX - - [10/Dec/2015:16:41:02 +0000] "GET
/?xxxxxxx&xxx=xx&ref=xxxxxxx&xxxxx=xx&xxx_xxxx=xx HTTP/1.1" 302 1236 "-"
"Mozilla/5.0
(Windows NT 6.3; WOW64; rv:42.0) Gecko/20100101 Firefox/42.0"
The configuration in the graylog webinterface of the extractor of nginx
access_log for Request Timestamp (Regular Expression):
Regular Expression: nginx:.+?\[(.+?)\]
Field matches this regular expression: ^\S+\s+nginx:
Add converter: numeric
(x) Convert to date type
Format String: dd/MMM/YYYY:HH:mm:ss Z
But there is no incoming message, because graylog-server throws an
IllegalArgumentException:
2015-12-10T16:46:19.321Z ERROR [Extractor] Could not apply converter [date]
of extractor [ea55a025-d293-4a54-8b66-284afc77e6fd].
java.lang.IllegalArgumentException: Invalid format: "10/Dec/2015:16:46:19
+0000" is malformed at "Dec/2015:16:46:19 +0000"
at org.joda.time.format.DateTimeFormatter.parseDateTime(
DateTimeFormatter.java:899)
at org.joda.time.DateTime.parse(DateTime.java:160)
at org.graylog2.inputs.converters.DateConverter.convert(
DateConverter.java:59)
at org.graylog2.plugin.inputs.Extractor.runConverters(Extractor.java
:247)
at org.graylog2.plugin.inputs.Extractor.runExtractor(Extractor.java:
232)
at org.graylog2.filters.ExtractorFilter.filter(ExtractorFilter.java:
62)
at org.graylog2.buffers.processors.ServerProcessBufferProcessor.
handleMessage(ServerProcessBufferProcessor.java:97)
at org.graylog2.shared.buffers.processors.ProcessBufferProcessor.
dispatchMessage(ProcessBufferProcessor.java:82)
at org.graylog2.shared.buffers.processors.ProcessBufferProcessor.
onEvent(ProcessBufferProcessor.java:61)
at org.graylog2.shared.buffers.processors.ProcessBufferProcessor.
onEvent(ProcessBufferProcessor.java:35)
at com.lmax.disruptor.WorkProcessor.run(WorkProcessor.java:138)
at com.codahale.metrics.
InstrumentedExecutorService$InstrumentedRunnable.run(
InstrumentedExecutorService.java:176)
at java.util.concurrent.ThreadPoolExecutor.runWorker(
ThreadPoolExecutor.java:1142)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(
ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)
Im quite sure, it works a few weeks ago, so here is what I tried:
- Downgrade to Version 1.2.2 and 1.2.1
- changed log_format of nginx from $time_local to $time_iso8601 (same
Exception with another date format)
I'm not as familiar with graylog extractors, cause I'm quite new to this
topic. Thats why I need help to locate and perhaps to solve this problem.
Anybody got an idea?
Thanks in advance
Christian
--
You received this message because you are subscribed to the Google Groups
"Graylog Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to [email protected].
To view this discussion on the web visit
https://groups.google.com/d/msgid/graylog2/1b1dfdc4-cd6e-468b-a1d5-6afcacac48bd%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
