[
https://issues.apache.org/jira/browse/FLUME-2789?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Alex Tian updated FLUME-2789:
-----------------------------
Description:
In my scenario, I need to send messages from a kafka source to a kafka sink ,
in other workds, transfering messages from a topic A to another topic B.
As a result, I find my kafka sink always sends messages back to topic A.
The reason in the codes is:
KafkaSink.java:
****************************************************************************************
107 if ((eventTopic = headers.get(TOPIC_HDR)) == null) {
108 eventTopic = topic;
109 }
****************************************************************************************
If the source is a Kafka source, eventTopic won't be null because it is the
source topic from the msg header. Therefore, the kafka sink will send the
message to its source topic by mistake.
was:
In my scenario, I need to send messages from a kafka source to a kafka sink ,
in other workds, transfering messages from a topic A to another topic B.
As a result, I find my kafka sink always sends messages back to topic A.
The reason in the codes is:
KafkaSink.java:
****************************************************************************************
107 if ((eventTopic = headers.get(TOPIC_HDR)) == null) {
108 eventTopic = topic;
109 }
****************************************************************************************
If the source is a Kafka source, eventTopic won't be null because it is the
source topic from the msg header. Therefore, the kafka sink will send the
message to its source topic by mistake.
I think we should remove line 107 and line 109 (Making "eventTopic = topic"
anyway)
> Kafka sink sends messages to the source topic by mistake
> ---------------------------------------------------------
>
> Key: FLUME-2789
> URL: https://issues.apache.org/jira/browse/FLUME-2789
> Project: Flume
> Issue Type: Bug
> Components: Sinks+Sources
> Affects Versions: v1.6.0
> Environment: Flume 1.6
> Reporter: Alex Tian
> Original Estimate: 1h
> Remaining Estimate: 1h
>
> In my scenario, I need to send messages from a kafka source to a kafka sink ,
> in other workds, transfering messages from a topic A to another topic B.
> As a result, I find my kafka sink always sends messages back to topic A.
> The reason in the codes is:
> KafkaSink.java:
> ****************************************************************************************
> 107 if ((eventTopic = headers.get(TOPIC_HDR)) == null) {
> 108 eventTopic = topic;
> 109 }
> ****************************************************************************************
> If the source is a Kafka source, eventTopic won't be null because it is the
> source topic from the msg header. Therefore, the kafka sink will send the
> message to its source topic by mistake.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
