[
https://issues.apache.org/jira/browse/FLUME-1941?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13603713#comment-13603713
]
Connor Woodson commented on FLUME-1941:
---------------------------------------
I think using {'s might make it overly complicated due to the exec source and
the hdfs path names; what about using "__" (two _), either in front or maybe
either side of the variable name (as that is something which I assume would
normally never be in a configuration).
{noformat}
agent.vars = variable
agent.vars.variable = cookies
agent.sinks.hdfs.hdfs.path = http://www.__variable__.com
{noformat}
The easiest route is to prevent variables from referencing themselves; not sure
of a good work around from that. If something detected as a variable hasn't
been defined, then throw a warning but treat that code as not a variable (and
if a line contains an opening two "_" but not a closing, probably again just
throw a warning but keep the text).
For the prototypes et al., intstead of creating a new section for defining
prototypes what about giving sources/channels/sinks a "parent" attribute (I
don't know how well it'd work for interceptors / serializers), so you'd have a
configuration like so:
{noformat}
agent.sinks.s1.type = HDFS
...
agent.sinks.s2.parent = s1
{noformat}
Rules would probably be that you can't have both a parent and a type; and to
prevent an infinite reference case, the parent sink cannot have "parent"
defined.
To allow for recursive inheritance, it would depend on how the configuration
code works; I haven't taken a look at it so I don't know. The parent of a sink
either has no parent defined, as before, or it will need to already have been
fully defined i.e. has taken all properties from its parent sink, thus making
it no longer have a parent, and thus fulfilling the previous case.
> Support defaults or inheritance in configs
> ------------------------------------------
>
> Key: FLUME-1941
> URL: https://issues.apache.org/jira/browse/FLUME-1941
> Project: Flume
> Issue Type: New Feature
> Components: Configuration
> Reporter: Mike Percy
> Fix For: v1.4.0
>
>
> Proposal to support defaults or inheritance in configs.
> The idea is to create a "prototypal" component config, such as a source or
> sink, which is not necessarily instantiated but is used to avoid repetitive
> configurations when creating multiple components of the same type. A great
> example of this is users who define 5 HDFS sinks to increase write
> parallelism, but they each contain many of the same configuration parameters
> and differ only in their name and path.
> Basic idea:
> {noformat}
> agent.sinks.sink-proto-1.type = my-sink
> agent.sinks.sink-proto-1.path = /var/log/foo/bar
> agent.sinks.sink-proto-1.serializer = MySerializer$Builder
> agent.sinks.sink-proto-1.credentials = mpercy
> agent.sinks.sink-1.__prototype__ = sink-proto-1
> agent.sinks.sink-1.path = /var/log/baz/blam
> agent.sinks.sink-2.__prototype__ = sink-proto-1
> agent.sinks.sink-2.path = /var/log/glerp/bazinga
> {noformat}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
