[
https://issues.apache.org/jira/browse/FLUME-1201?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13281929#comment-13281929
]
Joe Crobak commented on FLUME-1201:
-----------------------------------
I'm with Jarcec. A SpillableChannel is exactly what we'd like. Particularly in
EC2, where IO performance is variable, it'd be nice to bypass disk IO whenever
possible.
> Create a buffer channel, that stores overflow from a fast, low capacity
> channel to a slower high capacity channel
> -----------------------------------------------------------------------------------------------------------------
>
> Key: FLUME-1201
> URL: https://issues.apache.org/jira/browse/FLUME-1201
> Project: Flume
> Issue Type: New Feature
> Components: Channel
> Reporter: Juhani Connolly
> Assignee: Juhani Connolly
> Fix For: v1.2.0
>
>
> As it stands, users need to make a choice between either slower channels or
> faster ones. The current "middle ground" consists of RecoverableFileChannel
> which still stores everything to disk. MemoryChannel on the other hand has
> limited capacity or a high memory footprint.
> I propose to implement a buffer channel, somewhat like the buffer store in
> scribed.
> It would normally act as a proxy to the primary channel. Should this channel
> be unable to receive events(normally because it is at capacity, but perhaps
> some future channels may have other failure cases) it would switch states to
> buffering, storing new events to the secondary channel.
> In buffering, items continue to be read from the primary channel, and it
> attempts to "refill" itself from the secondary. Once the secondary is found
> to be empty, operation switches back to streaming mode, with items going
> directly to the primary.
> The main objective of this would thus be to have a high throughput channel as
> the primary mode of operation, allowing it to switch over when takes are not
> keeping up with puts.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
