[
https://issues.apache.org/jira/browse/METRON-578?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15690762#comment-15690762
]
ASF GitHub Bot commented on METRON-578:
---------------------------------------
Github user justinleet commented on the issue:
https://github.com/apache/incubator-metron/pull/363
Thanks a lot for reviewing this, Otto.
Regarding the WriterBoltIntegrationTest, the topics looks good. I had to
do a bit of digging to confirm them, because I didn't realize they're kinda
hidden in ParserTopologyBuilder, but it looks like ENRICHMENT_TOPIC,
DEFAULT_PARSER_INVALID_TOPIC, and DEFAULT_PARSER_ERROR_TOPIC are what you want
and what you have.
The reason this PR differs is because enrichment is specifically managed by
the test.yaml, so the way components get hooked up is different and required me
to define bolts to actually write the errors to Kafka topics.
> Missing error handling bolts for enrichment and threat intel
> ------------------------------------------------------------
>
> Key: METRON-578
> URL: https://issues.apache.org/jira/browse/METRON-578
> Project: Metron
> Issue Type: Improvement
> Affects Versions: 0.2.1BETA
> Reporter: Justin Leet
> Assignee: Justin Leet
>
> TL;DR - we need to add error handling to enrichments/threat intel
> Metron has parsers, enrichment + threat intel, and indexing topologies
> currently. Parsers and and enrichment have bolts that write to error topics
> in Kafka
> # indexing_error
> # parser_error
> # parser_invalid
> The GenericEnrichmentBolt handles errors gracefully by passing along failed
> enrichment tuples un-enriched and additionally emitting the tuple to an
> "error" stream, however there is currently no plumbing to handle the error
> stream.
> {code:java}
> } catch (Exception e) {
> LOG.error("[Metron] Unable to enrich message: " + rawMessage, e);
> JSONObject error = ErrorUtils.generateErrorMessage("Enrichment problem:
> " + rawMessage, e);
> if (key != null) {
> collector.emit(enrichmentType, new Values(key, enrichedMessage,
> subGroup));
> }
> collector.emit("error", new Values(error));
> }
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
