[
https://issues.apache.org/jira/browse/FLUME-1201?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13281819#comment-13281819
]
Jarek Jarcec Cecho commented on FLUME-1201:
-------------------------------------------
I'm also having concerns about compound channels. Current design is quite
straightforward, easy to understand and (with comparison to flume-og) working.
I'm afraid that having compound channels would introduce a lot of confusion in
the user based and might lead to user issues and bad impression from flume in
general.
However the idea of a memory (=fast) channel that would spill data do disk in
case that it would went full (because of issue on another agent down the road)
occurred to me as well, because that's exactly what scribe is doing and what my
current employer want to do :-) I'm volunteering myself to write such
"SpillableChannel" if we agree that it's acceptable.
Jarcec
> Create a buffer channel, that stores overflow from a fast, low capacity
> channel to a slower high capacity channel
> -----------------------------------------------------------------------------------------------------------------
>
> Key: FLUME-1201
> URL: https://issues.apache.org/jira/browse/FLUME-1201
> Project: Flume
> Issue Type: New Feature
> Components: Channel
> Reporter: Juhani Connolly
> Assignee: Juhani Connolly
> Fix For: v1.2.0
>
>
> As it stands, users need to make a choice between either slower channels or
> faster ones. The current "middle ground" consists of RecoverableFileChannel
> which still stores everything to disk. MemoryChannel on the other hand has
> limited capacity or a high memory footprint.
> I propose to implement a buffer channel, somewhat like the buffer store in
> scribed.
> It would normally act as a proxy to the primary channel. Should this channel
> be unable to receive events(normally because it is at capacity, but perhaps
> some future channels may have other failure cases) it would switch states to
> buffering, storing new events to the secondary channel.
> In buffering, items continue to be read from the primary channel, and it
> attempts to "refill" itself from the secondary. Once the secondary is found
> to be empty, operation switches back to streaming mode, with items going
> directly to the primary.
> The main objective of this would thus be to have a high throughput channel as
> the primary mode of operation, allowing it to switch over when takes are not
> keeping up with puts.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
