[
https://issues.apache.org/jira/browse/STORM-697?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14540344#comment-14540344
]
ASF GitHub Bot commented on STORM-697:
--------------------------------------
GitHub user itachi90 opened a pull request:
https://github.com/apache/storm/pull/550
STORM-697: Support for Emitting Kafka Message Offset and Partition
Allow kafkaSpout to emit message offset and partition so that user are able
to track tuple using this information.
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/Symantec/storm master
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/storm/pull/550.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #550
----
commit 98ab16caece75f451cdeaa8f006eb5eb13435389
Author: Yichi Zhang <[email protected]>
Date: 2015-05-12T17:51:00Z
STORM-697: Support for Emitting Kafka Message Offset and Partition
----
> Support for Emitting Kafka Message Offset and Partition
> -------------------------------------------------------
>
> Key: STORM-697
> URL: https://issues.apache.org/jira/browse/STORM-697
> Project: Apache Storm
> Issue Type: Improvement
> Components: storm-kafka
> Reporter: Matthew Tieman
> Assignee: Matthew Tieman
> Original Estimate: 4h
> Remaining Estimate: 4h
>
> It would be nice expose the offset and partition of messages consumed from
> Kafka to the Scheme generating the Tuples. This is useful for
> auditing/replaying data from arbitrary points on a Kafka topic. There are a
> few ways to accomplish this.
> One is to define a new scheme, say MessageAndMetadataScheme, and add another
> case in KafkaUtils.generateTuples to determine if the config's schema is an
> instance of MessageAndMetadataScheme.
> Another is to overload KafkaUtils.generateTuples to accept and instance of
> MessageAndRealOffset. Next, add a flag to SpoutConfig indicating that the
> scheme should accept a byte array of MessageAndRealOffset, this flag will be
> used to determine which generateTuples method to call.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
