[
https://issues.apache.org/jira/browse/FLUME-1995?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13664406#comment-13664406
]
Israel Ekpo commented on FLUME-1995:
------------------------------------
Hi Edward,
Thank you for the feedback.
After reading the internal architecture document, I do see your point.
I will explore another option then as this would lead to performance and disk
space issues down the line.
My reason for queuing outside of the same computer as the Flume agent is that
there are uses cases where the data is very important and if there happens to
be a non-recoverable hardware failure on the agent's machine where the file
channel is located, it would be easier to restart another agent with the same
configuration from a separate machine since the events would still be available
elsewhere.
We can discuss more alternatives offline (regarding what the queuing solution
should be) but I still think there is a need for a high-performing channel that
queues the events outside of the machine where the agent is running.
> CassandraChannel - A Distributed Channel Backed By Apache Cassandra as a
> Persistent Store for Events
> ----------------------------------------------------------------------------------------------------
>
> Key: FLUME-1995
> URL: https://issues.apache.org/jira/browse/FLUME-1995
> Project: Flume
> Issue Type: New Feature
> Components: Channel
> Affects Versions: v1.4.0
> Reporter: Israel Ekpo
> Assignee: Israel Ekpo
>
> Apache Cassandra Channel
> The events received by this channel are queued up in Cassandra to be picked
> up later when sinks send pickup requests to the channel.
> This type of channel is suitable for use cases where recoverability in the
> event of a hardware failure on the agent machine is important.
> The Cassandra cluster can be located on a remote machine.
> Cassandra also supports replication which could back up and replicate the
> events further to other nodes.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
