That will probably work. From my earlier mail: "I assume it would be connection based round robin? Might work just fine, but probably best to the use built-in support."
On Thu, Nov 15, 2012 at 11:27 AM, Rahul Ravindran <[email protected]> wrote: > HAProxy has a TCP mode where it round robins TCP connections. Does it need > to understand something specific about the wire protocol used by Flume? > > ________________________________ > From: Brock Noland <[email protected]> > To: [email protected]; Rahul Ravindran <[email protected]> > Sent: Wednesday, November 14, 2012 6:20 PM > > Subject: Re: Flume hops behind HAProxy > >> It would be round robin but not sticky sessions( so each request could >> Goto any random flume hop) > > Does HAProxy understand the protocol? To round robin requests like > this it needs to understand the communication protocol, which I > suppose would work if you were using the HTTPSource. > > On Wed, Nov 14, 2012 at 4:46 PM, Rahul Ravindran <[email protected]> wrote: >> HAProxy is widely deployed already in our deployment and Ops is familiar >> with dealing with it for hosts which go down etc. >> >> ________________________________ >> From: "Camp, Roy" <[email protected]> >> To: "[email protected]" <[email protected]> >> Sent: Wednesday, November 14, 2012 2:15 PM >> Subject: RE: Flume hops behind HAProxy >> >> Out of curiosity, what is the use case vs using the built in load >> balancing? >> >> >> >> -----Original Message----- >> From: Rahul Ravindran [mailto:[email protected]] >> Sent: Wednesday, November 14, 2012 1:49 PM >> To: [email protected] >> Cc: [email protected] >> Subject: Re: Flume hops behind HAProxy >> >> It would be round robin but not sticky sessions( so each request could >> Goto >> any random flume hop) >> >> Sent from my phone.Excuse the terseness. >> >> On Nov 14, 2012, at 1:33 PM, Brock Noland <[email protected]> wrote: >> >>> I assume it would be connection based round robin? Might work just >>> fine, but probably best to the use built-in support. >>> >>> On Wed, Nov 14, 2012 at 2:46 PM, Rahul Ravindran <[email protected]> >>> wrote: >>>> Resending given I sent it during off-hours. >>>> >>>> ________________________________ >>>> From: Rahul Ravindran <[email protected]> >>>> To: "[email protected]" <[email protected]> >>>> Sent: Tuesday, November 13, 2012 5:52 PM >>>> Subject: Flume hops behind HAProxy >>>> >>>> Hi, >>>> Before I try it, I wanted to check if there were any known issues >>>> with this. We will have multiple flume agents sending an Avro stream >>>> each to a smaller set of intermediate flume hops. Are there any >>>> issues/concerns around having the flume agents send their streams to >>>> an HAProxy which will round robin between the different flume hops. >>>> Any issue around the transaction mechanism with this setup? >>>> >>>> I know that there is a selector mechanism in Flume to do this, but >>>> our operations extensively use HAProxy, and are most familiar with it. >>>> >>>> Thanks, >>>> ~Rahul. >>> >>> >>> >>> -- >>> Apache MRUnit - Unit testing MapReduce - >>> http://incubator.apache.org/mrunit/ >> >> > > > > -- > Apache MRUnit - Unit testing MapReduce - http://incubator.apache.org/mrunit/ > > -- Apache MRUnit - Unit testing MapReduce - http://incubator.apache.org/mrunit/
