I think if the requirement is to reconfigure the config file by adding, removing or provisioning and existing agent, you don’t really have to make it too complicated. Just modify the content of your .conf file programmatically via UI.
The flume agent reloads the conf file periodically, like every 15 seconds or something so your changes will be propagated. This is the simplest way without any RPC. thanks, -Terrey On Sep 28, 2014, at 9:58 PM, [email protected] wrote: > Yes Blade, > > My requirement is either programmatically re-configure config file or add new > source/sink definitions to the already running agent. > > Will RPC client serves this purpose?Any examples will help me explore more on > it. > > Thanks, > Manohar. > > From: Blade Liu [mailto:[email protected]] > Sent: Sunday, September 28, 2014 2:05 PM > To: [email protected] > Subject: Re: Programmatically configuring new source/sink into Flume agent > > I guess you want to reconfig and run flume agents on-the-fly. Using RPC is > intended to restart agents or handle new config? > > > Thanks, > Blade > > 2014-09-27 7:58 GMT+08:00 terreyshih <[email protected]>: > HI, Manohar: > > I don’t understand your examples. copying/moving file from one directory to > another is not what source / sinks do. > > thx > > On Sep 26, 2014, at 5:24 AM, [email protected] wrote: > > > Hi All, > > I have a use-case where in I get source and sink details from user via UI > page, these config details I want to feed to flum so that programmatically it > starts processing newly added request for source/sink. When I was looking at > developer guide > https://flume.apache.org/FlumeDeveloperGuide.html#rpc-clients-avro-and-thrift > there is mention of using RPC client to programmatically get handle to flume > instance, now I wanted to know how can I programmatically feed these user > details as and when they arrive ? > > Example – one definition may look like copy files from NFs mount to HDFS and > one more might look like copy zipped files placed in other directory on local > system to HDFS sink etc. > > Thanks, > Manohar. > > > > > > > Please consider the environment before printing this e-mail > > Disclaimer: This communication is for the exclusive use of the intended > recipient(s) and shall not attach any liability on the originator or ITC > Infotech India Ltd./its Holding company/ its Subsidiaries/ its Group > Companies. If you are the addressee, the contents of this e-mail are intended > for your use only and it shall not be forwarded to any third party, without > first obtaining written authorization from the originator or ITC Infotech > India Ltd./ its Holding company/its Subsidiaries/ its Group Companies. It > may contain information which is confidential and legally privileged and the > same shall not be used or dealt with by any third party in any manner > whatsoever without the specific consent of ITC Infotech India Ltd./ its > Holding company/ its Subsidiaries/ its Group Companies.
