Hi Roberto,
Do you mean something like the proposed change at
https://issues.apache.org/jira/browse/FLUME-2993 ? Would that fit your
requirements?
Thank you,
Donat
On Dec 1, 2016 7:33 PM, "Shang Wu (CS)" <shang...@symantec.com> wrote:
> Hi Robert,
>
> Looks like what you want is more like a global configuration, and based on
> my limited knowledge in Apache Flume, it is missing now.
>
> BR,
>
> Shang
>
> *From: *Roberto Coluccio <roberto.coluc...@eng.it>
> *Reply-To: *"user@flume.apache.org" <user@flume.apache.org>
> *Date: *Thursday, December 1, 2016 at 2:59 AM
> *To: *"user@flume.apache.org" <user@flume.apache.org>
> *Subject: *Re: Define and leverage custom constants in agent configuration
>
>
>
> Hi Silvio,
>
> thanks for your suggestion. It gives me a workaround, but it's not exactly
> what I was hoping/asking about.
>
> I'm going to assume what I'd want to do being just not doable.
>
> Thank you,
>
> Roberto
>
>
>
> On 29/11/2016 09:26, Silvio Di gregorio wrote:
>
> with the "static interceptor" you can add key/value pairs in your flume
> event header:
>
>
>
> agentOne.sources.s1.interceptors = interOne
>
> agentOne.sources.s1.interceptors.interOne.type = static
>
> agentOne.sources.s1.interceptors.interOne.key = costante
>
> agentOne.sources.s1.interceptors.interOne.value = quello_che_vuoi
>
>
>
> I hope it is useful
>
> have nice day
>
> Silvio
>
>
>
> 2016-11-28 18:26 GMT+01:00 Roberto Coluccio <roberto.coluc...@eng.it>:
>
> Hello folks,
>
> I was wondering if it's possible to define custom constants in a Flume
> agent configuration file, so to avoid errors and replicated stuff when not
> needed. An example to make my point clear:
>
> assume I want to launch an agent of type *myAgent* configured with file
> channel and hdfs sink.
>
> I want the file channel to persist data and checkpoint into
> "~/flume/myAgent1/data" and "~/flume/myAgent1/checkpoint"
>
> Also, I want the hdfs sink to write files with prefix "myAgent1"
>
> In this scenario, I should harcorde the "myAgent1" string 3 times, one for
> each configuration parameter. This is error prone and kinda frustrating.
>
> Is there a way to define a constant like "myConstant" in the configuration
> file, and leverage it as parameter in the other configuration lines?
>
> e.g.
>
> myConstant = myAgent1
> myAgent.channels.myChannel.dataDirs = ~/flume/${myConstant}/data
> myAgent.sinks.mySink.hdfs.filePrefix = ${myConstant}
>
> It seems like the use of flume events header keys, but I don't want to add
> interceptors or other components just to have parameters in my config file.
>
> Thanks for your help.
>
> Best regards,
>
> Roberto
>
>
>
>
>
> --
>
>
>
>
>
>
>
>
>
> *Roberto Coluccio*
>
> Solution Designer
> Direzione Ricerca e Innovazione
> roberto.coluc...@eng.it
>
> Interno 604135 | Esterno +39 0510435135 <+39%20051%20043%205135>
>
>
>
> *Engineering Ingegneria Informatica spa*
> Via Marconi, 10, 40122 Bologna
>
> www.eng.it
>
>
>
