Hello folks,
I was wondering if it's possible to define custom constants in a Flume
agent configuration file, so to avoid errors and replicated stuff when
not needed. An example to make my point clear:
assume I want to launch an agent of type /myAgent/ configured with file
channel and hdfs sink.
I want the file channel to persist data and checkpoint into
"~/flume/myAgent1/data" and "~/flume/myAgent1/checkpoint"
Also, I want the hdfs sink to write files with prefix "myAgent1"
In this scenario, I should harcorde the "myAgent1" string 3 times, one
for each configuration parameter. This is error prone and kinda frustrating.
Is there a way to define a constant like "myConstant" in the
configuration file, and leverage it as parameter in the other
configuration lines?
e.g.
myConstant = myAgent1
myAgent.channels.myChannel.dataDirs = ~/flume/${myConstant}/data
myAgent.sinks.mySink.hdfs.filePrefix = ${myConstant}
It seems like the use of flume events header keys, but I don't want to
add interceptors or other components just to have parameters in my
config file.
Thanks for your help.
Best regards,
Roberto