There is support for failover and load balancing. Take a look at sink processors here http://flume.apache.org/FlumeUserGuide.html#flume-sink-processors You could have failover sinks configured to send it a different agent(i,e collector).
-roshan On Tue, Oct 16, 2012 at 10:09 AM, Pankaj Gupta <[email protected]>wrote: > Hi, > > > We are investigating using Flume for log aggregation in our organization > and have this dilemma over going with Flume NG or OG. We really like the > centralized configuration management in OG but that's not a must have. What > we need most is reliability in face of machines going down (which is quite > common on AWS). How is that handled in Flume NG. In OG, I believe, it was > possible to define failover semantics so that messages could be routed to > another collector if one collector went down. In Flume OG there is no > concept of a special collector but collector is just an agent as I > understand. I'm wondering how failover semantics are defined for these > agents. It would be great if someone could elaborate on that. > > We also want to use a relatively new version of Hadoop e.g. 1.0.3 or > higher. Is it possible to use Flume NG (or OG) with the new versions of > Hadoop? > > Thanks in advance, > > Pankaj > > -- > > > *P* | (415) 677-9222 ext. 205 *F *| (415) 677-0895 | [email protected] > > Pankaj Gupta | Software Engineer > > *BrightRoll, Inc. *| Smart Video Advertising | www.brightroll.com > > > United States | Canada | United Kingdom | Germany > > > We're > hiring<http://newton.newtonsoftware.com/career/CareerHome.action?clientId=8a42a12b3580e2060135837631485aa7> > ! > >
