[
https://issues.apache.org/jira/browse/FLUME-865?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13207406#comment-13207406
]
Juhani Connolly commented on FLUME-865:
---------------------------------------
@Arvind
My proposed config(and the current existing one in this patch) would look like
the following
host1.sinks = avro1 avro2
host1.channels = ch1
host1.sinks.avro1.type = avro
host1.sinks.avro1.channel = ch1
host1.sinks.avro1.runner.type = failover
host1.sinks.avro1.runner.name = fail1
host1.sinks.avro1.runner.priority = 1
host1.sinks.avro2.type = avro
host1.sinks.avro2.channel = ch1
host1.sinks.avro2.runner.type = failover
host1.sinks.avro2.runner.name = fail1
host1.sinks.avro2.runner.priority = 2
As you can see there is some repetition with needing to specify the type
repeatedly. If the runner type is not specified it would default to a normal
SinkRunner.
However with this approach we can specify per-sink settings such as priority or
weight, and we can use the existing code for runner configuration. Adding the
concept of groups adds a whole lot of configuration overhead and feels unwieldy
to me.
I understand you don't want to maintain the core runner logic in multiple
classes, hence my suggestion of turning sinkrunner into an abstract class that
other classes override. To enforce the logic we could make the core functions
final and thus avoid creation of class confetti... Effectively the processing
logic would be abstracted out, as well as adding and removing sinks. It would
also be possible to add these into a sink processor. In this case I guess that
the runner would just own the processor with no access to the sinks, and the
processor would be responsible for handling all the sinks.
If we want to avoid having multiple runners we could change the config
runner.type to
host1.sinks.avro1.runner.processor = failover
> Implement failover sink
> ------------------------
>
> Key: FLUME-865
> URL: https://issues.apache.org/jira/browse/FLUME-865
> Project: Flume
> Issue Type: New Feature
> Components: Sinks+Sources
> Affects Versions: NG alpha 2
> Reporter: Jarek Jarcec Cecho
> Assignee: Juhani Connolly
> Fix For: v1.1.0
>
> Attachments: FLUME-865.patch
>
>
> It would be nice if the flume-ng would have ability to failover to different
> sink in case that the active one is not responding (e.g. before failing the
> transaction).
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
