Nick Allen created METRON-1926:
----------------------------------
Summary: Parser Validation Does Not Indicate Cause of Failure
Key: METRON-1926
URL: https://issues.apache.org/jira/browse/METRON-1926
Project: Metron
Issue Type: Bug
Reporter: Nick Allen
The BasicParser performs some validation of the messages produced by a parser.
It ensure that a parser implementation produces a message with a 'timestamp'
and an 'original_string'.
When the BasicParser causes message parsing to fail, the error message that is
produced and sent to the error topic does not contain any information
indicating why the message failed to parse. The only indication are log
statements made at the trace level.
For example, using a new regex parser implementation, the BasicParser was
causing valid messages to fail because there was no timestamp added by this
parser. The error message produced does not indicate why the message failed to
parse.
{code:java}
{
"failed_sensor_type": "regex",
"hostname": "node1",
"raw_message":
"{\"dst_process_id\":\"11672\",\"dst_process_name\":\"sshd\",\"source.type\":\"regex\",\"device_name\":\"deviceName\",\"original_string\":\"<38>Jun
20 15:01:17 deviceName sshd[11672]: Accepted publickey for prod from
22.22.22.22 port 55555 ssh2\",\"event_info\":\"Accepted
publickey\",\"ip_src_port\":\"55555\",\"dst_user_id\":\"prod\",\"app_protocol\":\"ssh2\",\"guid\":\"edaee82d-02fb-4ec9-9412-5912fa8d4a6f\",\"syslogpriority\":\"38\",\"timestamp_device_original\":\"Jun
20 15:01:17\",\"ip_src_addr\":\"22.22.22.22\"}",
"error_hash":
"51d323ef83d03c4a8f9b858f7779cb882f3e61925909b66bc61348756c201057",
"error_type": "parser_invalid",
"guid": "7cbc9553-feaf-4b23-9468-01b6599299bd",
"source.type": "error",
"timestamp": 1543518747945
}{code}
The message should contain a message or exception field telling me why it
failed to parse.
The additional validation is [performed
here|https://github.com/apache/metron/blob/2ee6cc7e0b448d8d27f56f873e2c15a603c53917/metron-platform/metron-parsers/src/main/java/org/apache/metron/parsers/BasicParser.java#L34-L46].
This issue was [discovered
here|https://github.com/apache/metron/pull/1245#discussion_r239608145].
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
