[
https://issues.apache.org/jira/browse/FLUME-865?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13208240#comment-13208240
]
Arvind Prabhakar commented on FLUME-865:
----------------------------------------
@Jarcec - thanks, I think we will need your help considering this issue is a
sizable one. Please go ahead and file the sub-task to break off the
pluggability of sink processor and related configuration group mechanism.
@Juhani, sorry for the delay in responding. My comments on your points follow:
* Handing sink specific configuration: One advantage of using namespaced
hierarchical configuration is that you get independent namespaces which can be
defined by the component as necessary. So anything that is needed by the
failover processor could be dictated by its implementation and easily
assimilated pulled into the configuration. For example:
host1.sinks.groups = group1
host1.sinks.groups.group1.sinks = sink1 sink2
host1.sinks.groups.group1.processor.type = failover
host1.sinks.groups.group1.processor.priority.sink1 = 5
host1.sinks.groups.group1.processor.priority.sink2 = 10
In this example - everything below the host1.sinks.groups.group1.processor
becomes a property context for the processor itself. It is then free to define
how it expects the configuration to be laid out in that namespace.
* How to handle groups: Your suggestion of making groups as first class objects
within AgentConfiguration seems appropriate to me. Regarding your confusion wrt
to the loadSinks() method being part of the properties file configuration
provider - yes, that is due to there being only one configuration provider for
now. Eventually, we will refactor out the process of parsing and loading
configuration from the actual wiring of components. I recently filed FLUME-969
for this main purpose.
* Leftover sinks: I agree that we could wire left-over sinks with default
singleton groups which exposes a processer interface to the runner just like
any other group.
> Implement failover sink
> ------------------------
>
> Key: FLUME-865
> URL: https://issues.apache.org/jira/browse/FLUME-865
> Project: Flume
> Issue Type: New Feature
> Components: Sinks+Sources
> Affects Versions: NG alpha 2
> Reporter: Jarek Jarcec Cecho
> Assignee: Juhani Connolly
> Fix For: v1.1.0
>
> Attachments: FLUME-865.patch
>
>
> It would be nice if the flume-ng would have ability to failover to different
> sink in case that the active one is not responding (e.g. before failing the
> transaction).
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
