[ https://issues.apache.org/jira/browse/METRON-1501?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16412013#comment-16412013 ]
ASF GitHub Bot commented on METRON-1501: ---------------------------------------- Github user ottobackwards commented on a diff in the pull request: https://github.com/apache/metron/pull/972#discussion_r176854208 --- Diff: metron-platform/metron-parsers/README.md --- @@ -45,6 +45,33 @@ There are two general types types of parsers: * `ERROR` : Throw an error when a multidimensional map is encountered * `jsonpQuery` : A [JSON Path](#json_path) query string. If present, the result of the JSON Path query should be a list of messages. This is useful if you have a JSON document which contains a list or array of messages embedded in it, and you do not have another means of splitting the message. * A field called `timestamp` is expected to exist and, if it does not, then current time is inserted. + +## Parser Error Routing + +Currently, we have a few mechanisms for either deferring processing of +messages or marking messages as invalid. + +### Invalid + --- End diff -- I would says 'there are two reasons a message will be marked as invalid' and * Failing global blah * Failing the parser's...... > Parser messages that fail to validate are dropped silently > ---------------------------------------------------------- > > Key: METRON-1501 > URL: https://issues.apache.org/jira/browse/METRON-1501 > Project: Metron > Issue Type: Bug > Reporter: Casey Stella > Priority: Major > > Currently we have two concepts of dealing with messages that are not going to > be passed through: > * Messages that fail global validation > * Messages that fail the parser's validation function > * Messages that are filtered out > > Currently in the case of messages that fail global validation, we send them > to the error queue. > In the case of messages that fail the parser's validation function, we drop > them silently. > In the case of messages that are filtered out, we drop them silently as well. > > Given the use-cases behind filtering (e.g. a kafka topic with multiple types > of messages where we want to pass along only one type), it makes sense to > drop them silently. However, in the case of the messages that fail the > parser's validation function, we should like to know that and treat those > similarly to those messages that fail global validation. -- This message was sent by Atlassian JIRA (v7.6.3#76005)