[
https://issues.apache.org/jira/browse/FLUME-2390?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14158647#comment-14158647
]
Edward Sargisson commented on FLUME-2390:
-----------------------------------------
[~xnag] Err... No. If the sink can't figure out a way to write the data then it
will block and wait for you to fix it. There is currently no poison pill
handling.
> Flume-ElasticSearch Data gets posted multiple times when one of the event
> fail validation at elastic search sink for JSON Data
> ------------------------------------------------------------------------------------------------------------------------------
>
> Key: FLUME-2390
> URL: https://issues.apache.org/jira/browse/FLUME-2390
> Project: Flume
> Issue Type: Bug
> Components: Sinks+Sources
> Affects Versions: v1.4.0
> Environment: CDH4.5
> Reporter: Deepak Subhramanian
>
> Hi,
> I am using Elastic Search Sink to post JSON data. I used the temporary fix
> mentioned in https://issues.apache.org/jira/browse/FLUME-2126 to get JSON
> data posted to elastic search. When one of the message fail validation at
> ElasticSearch mapping for JSON data ( For example - getting empty message) ,
> Flume seems to post the entire batch again and again until I restart Flume.
> Because of that no of events went from an avg of 100 to avg of 2000 per 10
> minutes. As a temporary fix I set a header in my FlumeHTTP Source for non
> valid JSON and used a interceptor to send data to multiple ESSINKS which has
> different index names.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
