Hi Robert, Looks like what you want is more like a global configuration, and based on my limited knowledge in Apache Flume, it is missing now. BR, Shang From: Roberto Coluccio <[email protected]> Reply-To: "[email protected]" <[email protected]> Date: Thursday, December 1, 2016 at 2:59 AM To: "[email protected]" <[email protected]> Subject: Re: Define and leverage custom constants in agent configuration
Hi Silvio, thanks for your suggestion. It gives me a workaround, but it's not exactly what I was hoping/asking about. I'm going to assume what I'd want to do being just not doable. Thank you, Roberto On 29/11/2016 09:26, Silvio Di gregorio wrote: with the "static interceptor" you can add key/value pairs in your flume event header: agentOne.sources.s1.interceptors = interOne agentOne.sources.s1.interceptors.interOne.type = static agentOne.sources.s1.interceptors.interOne.key = costante agentOne.sources.s1.interceptors.interOne.value = quello_che_vuoi I hope it is useful have nice day Silvio 2016-11-28 18:26 GMT+01:00 Roberto Coluccio <[email protected]<mailto:[email protected]>>: Hello folks, I was wondering if it's possible to define custom constants in a Flume agent configuration file, so to avoid errors and replicated stuff when not needed. An example to make my point clear: assume I want to launch an agent of type myAgent configured with file channel and hdfs sink. I want the file channel to persist data and checkpoint into "~/flume/myAgent1/data" and "~/flume/myAgent1/checkpoint" Also, I want the hdfs sink to write files with prefix "myAgent1" In this scenario, I should harcorde the "myAgent1" string 3 times, one for each configuration parameter. This is error prone and kinda frustrating. Is there a way to define a constant like "myConstant" in the configuration file, and leverage it as parameter in the other configuration lines? e.g. myConstant = myAgent1 myAgent.channels.myChannel.dataDirs = ~/flume/${myConstant}/data myAgent.sinks.mySink.hdfs.filePrefix = ${myConstant} It seems like the use of flume events header keys, but I don't want to add interceptors or other components just to have parameters in my config file. Thanks for your help. Best regards, Roberto -- [cid:[email protected]] Roberto Coluccio Solution Designer Direzione Ricerca e Innovazione [email protected]<mailto:[email protected]> Interno 604135 | Esterno +39 0510435135 Engineering Ingegneria Informatica spa Via Marconi, 10, 40122 Bologna www.eng.it<http://www.eng.it/>
