Hi, I would use puppet, chef, scp, or Cloudera Manager to manage the configuration files of the flume agents. No sense in writing something custom just to copy files around.
Brock On Wed, Oct 3, 2012 at 9:39 AM, Kim, Jongkook <[email protected]> wrote: > Hi Brock, > > Without a central place for agent config, how to update flume.conf files on > hundreds of agents? > > We are trying to give user an ability to update agent setup (previously a > logical node) real-time. > > About 10 - 30 logical nodes are created per agent and they can be updated or > deleted anytime. > > Do you have any architectural design of this? > > > Thanks. > -----Original Message----- > From: Brock Noland [mailto:[email protected]] > Sent: Wednesday, October 03, 2012 10:31 AM > To: [email protected] > Cc: *DE Global E2E Sponge Stakeholders > Subject: Re: Flume NG for enterprise environment > > Hi, > > I can confirm that many organizations are using Flume NG in a production > enterprise environment. You are correct the concept of a master no longer > exists. > > Brock > > On Tue, Oct 2, 2012 at 4:02 PM, Kim, Jongkook <[email protected]> wrote: >> Hi, >> >> >> >> I'm using flume 0.94 in CDH3 environment and thinking to migrate to CDH4. >> >> >> >> The old flume has a master instance to control agents and collectors, >> so we developed a configuration UI to communicate with a flume master. >> >> >> >> From this UI, user can add, edit, and delete logical nodes. >> >> >> >> Does anyone use flume NG on enterprise environment? >> >> >> >> Without the master concept, how do you manage all agents and collectors. >> >> >> >> If you need to read more files on a machine on flume NG, do you >> manually update flume.conf file and restart the agent? >> >> >> >> Thanks. >> >> >> >> >> >> >> >> > > > > -- > Apache MRUnit - Unit testing MapReduce - http://incubator.apache.org/mrunit/ -- Apache MRUnit - Unit testing MapReduce - http://incubator.apache.org/mrunit/
