Before the thread fades in-memory, here is the JIRA https://issues.apache.org/jira/browse/FLUME-2661
On Thu, Apr 2, 2015 at 6:26 AM, Ashish <[email protected]> wrote: > Is it possible to share the method and class name? It would be > somewhere in client package I guess. > I shall do the rest, else would need to re-invent the wheel :) > > On Wed, Apr 1, 2015 at 3:30 AM, Otis Gospodnetic > <[email protected]> wrote: >> Hi, >> >> Unfortunately don't have a patch, though I think this is just 1 method that >> needs to be added. Don't think ELB provides the info needed, so one needs >> to just have timeout. >> >> Otis >> -- >> Monitoring * Alerting * Anomaly Detection * Centralized Log Management >> Solr & Elasticsearch Support * http://sematext.com/ >> >> >> On Tue, Mar 31, 2015 at 4:25 PM, Hari Shreedharan <[email protected] >>> wrote: >> >>> I am not very familiar with ELB, but if you do have a patch that will make >>> Flume work better - I’ll be glad to review it. I think the HDFS, HBase, >>> Morphline, Solr, Thrift, and Avro sinks will need it (similar logic, but >>> different impl). >>> >>> >>> >>> >>> Thanks, Hari >>> >>> On Tue, Mar 31, 2015 at 9:27 AM, Ashish <[email protected]> wrote: >>> >>> > Does ELB provide any kind of status code when it rejects the request >>> > or it just doesn't send any response? >>> > Perhaps adding a JIRA would be great. Doesn't ELB times out connection >>> > which are Idle for more than 60 sec? >>> > thanks >>> > ashish >>> > On Tue, Mar 31, 2015 at 7:50 PM, Otis Gospodnetic >>> > <[email protected]> wrote: >>> >> Hi, >>> >> >>> >> I don't have a very good bug report, but I want to share one problem >>> with >>> >> Flume we observed the other day. Specifically, with Elasticsearch Sink >>> and >>> >> AWS ELB, but others may have this problem, too. >>> >> >>> >> In short, ELB has this notion of a Spillover where it can reject >>> requests >>> >> when it can't handle the load. Unfortunately, it doesn't actually >>> signal >>> >> this by closing the connection, so Flume's connection remains open, I >>> >> believe in some java.net....... readSocket(....) method. >>> >> >>> >> To work around this we had to add a socket timeout in our custom >>> version of >>> >> Flume's Elasticsearch Sink. >>> >> >>> >> I think it's worth fixing this in Flume and checking if other Sinks need >>> >> this sort of timeout. >>> >> >>> >> HTH >>> >> >>> >> Otis >>> >> -- >>> >> Monitoring * Alerting * Anomaly Detection * Centralized Log Management >>> >> Solr & Elasticsearch Support * http://sematext.com/ >>> > -- >>> > thanks >>> > ashish >>> > Blog: http://www.ashishpaliwal.com/blog >>> > My Photo Galleries: http://www.pbase.com/ashishpaliwal >>> > > > > -- > thanks > ashish > > Blog: http://www.ashishpaliwal.com/blog > My Photo Galleries: http://www.pbase.com/ashishpaliwal -- thanks ashish Blog: http://www.ashishpaliwal.com/blog My Photo Galleries: http://www.pbase.com/ashishpaliwal
