[
https://issues.apache.org/jira/browse/FLUME-2799?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15098913#comment-15098913
]
Roshan Naik commented on FLUME-2799:
------------------------------------
- I don't think it is wise to block this requirement on Kafka 0.9. This ability
seems useful in its own right.
- Functionally, It does seem to overlap with notion of interceptors even if its
not the intention.. JMS convertors, deal more with body and less with headers.
- If each source implements its own converters. It is better to have a common
reusable convertor system shared by others sources. Which can then bring into
question the need for interceptors.
- Although well motivated, It feels excessive to introduce convertors in this
ticket which deals with merely adding couple headers.
- My thoughts
+ Documentation needs update
+ Make the new headers optional (and disabled by default) so that existing
users don
't see any impact.
+ If you are willing to simplify it to do this without convertors it would
make this a simpler review and require less debate. Unless you any other
thoughts ?
> Kafka Source - Message Offset and Partition add to headers
> ----------------------------------------------------------
>
> Key: FLUME-2799
> URL: https://issues.apache.org/jira/browse/FLUME-2799
> Project: Flume
> Issue Type: Improvement
> Components: Sinks+Sources
> Affects Versions: v1.6.0
> Reporter: Michael Andre Pearce (IG)
> Priority: Minor
> Labels: easyfix, patch
> Fix For: v1.7.0
>
> Attachments: FLUME-2799-0.patch
>
> Original Estimate: 24h
> Remaining Estimate: 24h
>
> Currently Kafka source only persists the original kafka message's topic into
> the Flume event headers.
> For downstream interceptors and sinks that may want to have available to them
> the partition and the offset , we need to add these.
> Also it is noted that the conversion from MessageAndMetaData to FlumeEvent is
> not configurable unlike other sources such as JMS.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
