Simon Elliston Ball created METRON-1639:
-------------------------------------------
Summary: Grok Parser does not handle missing year well in syslog
rfc3164 timestamps
Key: METRON-1639
URL: https://issues.apache.org/jira/browse/METRON-1639
Project: Metron
Issue Type: Improvement
Affects Versions: 0.5.0
Reporter: Simon Elliston Ball
Assignee: Simon Elliston Ball
The grok parser does not handle timestamp fields in rfc3164 format well, since
the format omits a year from the date, the year defaults to 1970. We should
either switch this to default year to current, or create a "dateFormat" config
option "syslog" which runs the SyslogUtils parser used in other parsers on the
captured fields for the field specified in "timestampField" config.
This capability should also reflect the timezone for the sensor, which is not
currently applied to Grok parsing but is honoured in parsers like
BasicASAParser. Note that it is not universally applied across all parsers, but
probably should be.
"Mmm dd hh:mm:ss" is the canonical date format in rfc3164, with options to
include a timezone and year. We currently handle this and variants found in the
wild in
SyslogUtils::parseTimestampToEpochMillis, which also accounts for timezone
based on a Clock parameter. This function assumes that any date more than 4
days in the future is in the past, which seems acceptable and consistent for
our purposes and covers the possibility of year end discrepancies.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
