Arkadiusz Firus created FLUME-2848:
--------------------------------------
Summary: Kafka Source could not work with Kafka Sink
Key: FLUME-2848
URL: https://issues.apache.org/jira/browse/FLUME-2848
Project: Flume
Issue Type: Bug
Components: Sinks+Sources
Affects Versions: v1.6.0
Environment: IMHO any
Reporter: Arkadiusz Firus
KafkaSource stores information from which topic record was read. This
information is stored in event header with key "topic" (see
org.apache.flume.source.kafka.KafkaSource line 108).
KafkaSink can be instructed to store record in a different topic than the one
in configuration (dynamic destination). To do that "topic" header have to be
set in event (see org.apache.flume.sink.kafka.KafkaSink line 107).
When this two features are combined we have infinite loop - KafkaSink writes
record to topic which is read by KafkaSource.
Two solve the problem fields:
org.apache.flume.sink.kafka.KafkaSink.TOPIC_HDR
org.apache.flume.source.kafka.KafkaSourceConstants.TOPIC
need to have different values.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
