What I did was modified the agent to register itself with ZK if its source listens to a port. I then have it setup so that the the agent will check ZK if the sink defines a config called "cluster". This is used to discover the host to talk to. If the sink defines cluster, the host/port will be replaced.
This was all done in the agent itself so i could inherit all source/sinks out there. I was thinking of forking the FailoverSinkProcessor so on failover discover a new host to talk to and create a new sink. On Mon, Aug 13, 2012 at 1:11 PM, Ralph Goers <[email protected]>wrote: > I'd be interested in knowing how you dynamically located the new host. I > would also imagine you could create a "smart" sink that could failover > itself. > > Ralph > > On Aug 13, 2012, at 11:33 AM, David Capwell wrote: > > > I am looking at flume for data ingestion on AWS. I was wondering how > people handle dynamic hostnames in flume? > > > > I have created my own ConfigurationProvider and looking to mimic > FailoverSinkProcessor but on failover to dynamically find a new host to > talk to. > > > > Is there a better route to go than the one I went down? > > > > Thanks for your time reading this email. > >
