[
https://issues.apache.org/jira/browse/FLUME-2523?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14191375#comment-14191375
]
Hari Shreedharan commented on FLUME-2523:
-----------------------------------------
Thanks for writing these docs!
A few minor comments:
- The example shows kafka-1:9092 etc. That should be a full hostname, correct?
(for both ZK and broker list).
- Mention that class that is used when parseAsFlumeEvent is true. This would be
required if we just want Flume to write to Kafka and not remove it from the
channel - so other consumers are able to remove the data (Log4j->Flume->Kafka
etc). The class is org.apache.flume.source.avro.AvroFlumeEvent provided by the
flume-ng-sdk artifact.
- Worth mentioning how multiple channels must use the same topic and group to
ensure that when one agent fails another can remove the data.
- readSmallestOffset matters only if parseAsFlumeEvent is false. We should also
mention why it is better to keep it to false (or not set it at all) if
parseAsFlumeEvent is false.
- Mention that producer.type will always be sync and other properties which we
override anyway - even if the customer has overriden it.
- Mention that by using Kafka Channel -> HDFS/HBase/Solr sink, Kafka+Flume
provides low latency, fault-tolerant writes to the corresponding system.
> Document Kafka channel
> ----------------------
>
> Key: FLUME-2523
> URL: https://issues.apache.org/jira/browse/FLUME-2523
> Project: Flume
> Issue Type: Task
> Components: Docs
> Reporter: Gwen Shapira
> Assignee: Gwen Shapira
> Attachments: FLUME-2523.0.patch
>
>
> FLUME-2500 adds a Kafka channel. We need to document its usage.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
