[
https://issues.apache.org/jira/browse/METRON-1832?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Nick Allen updated METRON-1832:
-------------------------------
Attachment: error-message.json
> Recurrent Large Indexing Error Messages
> ---------------------------------------
>
> Key: METRON-1832
> URL: https://issues.apache.org/jira/browse/METRON-1832
> Project: Metron
> Issue Type: Bug
> Reporter: Nick Allen
> Priority: Major
> Attachments: error-message.json
>
>
> If any index destination like HDFS, Elasticsearch, or Solr goes down while
> the Indexing topology is running, an error message is created and sent back
> to the user-defined error topic. By default, this is defined to also be the
> 'indexing' topic.
> The Indexing topology then consumes this error message and attempts to write
> it again. If the index destination is still down, another error occurs and
> another error message is created that encapsulates the original error
> message. That message is then sent to the 'indexing' topic, which is later
> consumed, yet again, by the Indexing topology.
> These error messages will continue to be recycled and grow larger and larger
> as each new error message encapsulates all previous error messages in the
> "raw_message" field.
> Once the index destination recovers, one giant error message will finally be
> written that contains massively duplicated, useless information which can
> further negatively impact performance of the index destination.
> See attachment for example of once of these error messages.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
