[
https://issues.apache.org/jira/browse/FLUME-2789?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Alex Tian updated FLUME-2789:
-----------------------------
Description:
In my scenario, I need to send messages from a kafka source to a kafka sink ,
in other workds, transfering messages from a topic A to another topic B.
As a result, I find my kafka sink always sends messages back to topic A.
The reason in the codes is:
KafkaSink.java:
****************************************************************************************
107 if ((eventTopic = headers.get(TOPIC_HDR)) == null) {
108 eventTopic = topic;
109 }
****************************************************************************************
If the source is a Kafka source, eventTopic won't be null because it is the
source topic from the msg header. Therefore, the kafka sink will send the
message to its source topic by mistake.
I think we should remove line 107 and line 109 (Making "eventTopic = topic"
anyway)
was:
In my scenario, I need to send messages from a kafka source to a kafka sink ,
in other workds, transfering messages from a topic A to another topic B.
As a result, I find my kafka sink always sends messages back to topic A.
The reason in the codes is:
KafkaSink.java:
****************************************************************************************
107 if ((eventTopic = headers.get(TOPIC_HDR)) == null) {
108 eventTopic = topic;
109 }
If the source is a Kafka source, eventTopic won't be null because it is the
source topic from the msg header. Therefore, the kafka sink will send the
message to its source topic by mistake.
I think we should remove line 107 and line 109 (Making "eventTopic = topic"
anyway)
> Kafka sink sends messages to the source topic by mistake
> ---------------------------------------------------------
>
> Key: FLUME-2789
> URL: https://issues.apache.org/jira/browse/FLUME-2789
> Project: Flume
> Issue Type: Bug
> Components: Sinks+Sources
> Affects Versions: v1.6.0
> Environment: Flume 1.6
> Reporter: Alex Tian
> Original Estimate: 1h
> Remaining Estimate: 1h
>
> In my scenario, I need to send messages from a kafka source to a kafka sink ,
> in other workds, transfering messages from a topic A to another topic B.
> As a result, I find my kafka sink always sends messages back to topic A.
> The reason in the codes is:
> KafkaSink.java:
> ****************************************************************************************
> 107 if ((eventTopic = headers.get(TOPIC_HDR)) == null) {
> 108 eventTopic = topic;
> 109 }
> ****************************************************************************************
> If the source is a Kafka source, eventTopic won't be null because it is the
> source topic from the msg header. Therefore, the kafka sink will send the
> message to its source topic by mistake.
> I think we should remove line 107 and line 109 (Making "eventTopic = topic"
> anyway)
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
