Hari Shreedharan created FLUME-2500:
---------------------------------------
Summary: Add a channel that uses Kafka
Key: FLUME-2500
URL: https://issues.apache.org/jira/browse/FLUME-2500
Project: Flume
Issue Type: Bug
Reporter: Hari Shreedharan
Assignee: Hari Shreedharan
Here is the rationale:
- Kafka does give a HA channel, which means a dead agent does not affect the
data in the channel - thus reducing delay of delivery.
- Kafka is used by many companies - it would be a good idea to use Flume to
pull data from Kafka and write it to HDFS/HBase etc.
This channel is not going to be useful for cases where Kafka is not already
used, since it brings is operational overhead of maintaining two systems, but
if there is Kafka in use - this is good way to integrate Kafka and Flume.
Here is an a scratch implementation:
https://github.com/harishreedharan/flume/blob/kafka-channel/flume-ng-channels/flume-kafka-channel/src/main/java/org/apache/flume/channel/kafka/KafkaChannel.java
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
