You would use something like puppet or chef to push such config files around.
Regards, Harish On Wed, Oct 10, 2012 at 2:19 PM, Camp, Roy <[email protected]> wrote: > You have to manually start each node with its specific configuration. > However, you can put the configuration for all your different setups into > one config file, but you will still need to place a copy of it on every > machine. Simply define which agent config to use with the --name param > when starting. **** > > ** ** > > Thanks,**** > > ** ** > > Roy**** > > ** ** > > ** ** > > *From:* Juan Gentile [mailto:[email protected]] > *Sent:* Wednesday, October 10, 2012 9:54 AM > *To:* [email protected] > *Subject:* Re: Flume-ng - Distributed**** > > ** ** > > Thank you both very much, I've been reading the documentation you sent me > and this brings another question, is there a way to submit my flume > configuration to a cluster or I have to manually start up each node with > the its specific configuration?**** > > ** ** > > Thank you!**** > > On Wed, Oct 10, 2012 at 1:51 AM, Mike Percy <[email protected]> wrote:**** > > +1 on what Roy said, with a minor terminology quibble: in Flume NG the > Avro collector component is called the Avro Source.**** > > ** ** > > Also, here are links to the docs with working image links and table of > contents:**** > > ** ** > > http://flume.apache.org/FlumeUserGuide.html**** > > http://flume.apache.org/FlumeDeveloperGuide.html**** > > ** ** > > Regards,**** > > Mike**** > > ** ** > > On Tue, Oct 9, 2012 at 5:52 PM, Camp, Roy <[email protected]> wrote:**** > > You would run a flume-ng instance on each node with an avro-sink. Then on > your collector machine you will run another flume-ng instance with an > avro-collector.**** > > **** > > If you run more than one collector you can setup sink groups and define > that it does failover or load balancing.**** > > **** > > The concept of a flume master from flume 0.9.x does not exist on > flume-ng. I personally use the node and collector configs in the same > config file under a different agent name, and then keep them synced on all > machines. **** > > **** > > These two docs are pretty helpful:**** > > > https://github.com/apache/flume/blob/trunk/flume-ng-doc/sphinx/FlumeUserGuide.rst > > https://github.com/apache/flume/blob/trunk/flume-ng-doc/sphinx/FlumeDeveloperGuide.rst > **** > > **** > > Thanks,**** > > **** > > Roy**** > > **** > > **** > > **** > > **** > > *From:* Juan Gentile [mailto:[email protected]] > *Sent:* Tuesday, October 09, 2012 11:04 AM > *To:* [email protected] > *Subject:* Flume-ng - Distributed**** > > **** > > Hi,**** > > **** > > I'm new to Flume-ng, I'd like to ask you if you can tell me how I can > accomplish to have an agent distributed in a cluster. I've have developed > my own source and sink version that reads from a queue and the sink stores > the messages read to hdfs. If I want to have this running in multiple > instances, do I have to submit it on each node?**** > > **** > > This is my conf file:**** > > agent1.channels.channel1.type = memory**** > > agent1.channels.channel1.capacity = 1000**** > > agent1.channels.channel1.transactionCapacity = 1000**** > > **** > > agent1.sources.source1.channels = channel1**** > > agent1.sources.source1.type = MySource**** > > **** > > agent1.sinks.sink1.channel = channel1**** > > agent1.sinks.sink1.type = MySink**** > > **** > > agent1.channels = channel1**** > > agent1.sources = source1**** > > agent1.sinks = sink1**** > > **** > > **** > > I see that there is the concept of 'master' a 'node' in the previous > version of flume, do I have something similar here?**** > > **** > > Thanks,**** > > Juan**** > > ** ** > > ** ** >
