Ashish Paliwal created FLUME-2661:
-------------------------------------
Summary: Support timeout in ES Sink
Key: FLUME-2661
URL: https://issues.apache.org/jira/browse/FLUME-2661
Project: Flume
Issue Type: Bug
Components: Sinks+Sources
Affects Versions: v1.5.1
Reporter: Ashish Paliwal
Credits to Otis, From ML
Hi,
I don't have a very good bug report, but I want to share one problem with
Flume we observed the other day. Specifically, with Elasticsearch Sink and
AWS ELB, but others may have this problem, too.
In short, ELB has this notion of a Spillover where it can reject requests
when it can't handle the load. Unfortunately, it doesn't actually signal
this by closing the connection, so Flume's connection remains open, I
believe in some java.net....... readSocket(....) method.
To work around this we had to add a socket timeout in our custom version of
Flume's Elasticsearch Sink.
I think it's worth fixing this in Flume and checking if other Sinks need
this sort of timeout.
HTH
Otis
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
