[
https://issues.apache.org/jira/browse/NIFI-2298?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15386131#comment-15386131
]
ASF GitHub Bot commented on NIFI-2298:
--------------------------------------
GitHub user olegz opened a pull request:
https://github.com/apache/nifi/pull/687
NIFI-2298 added Kafka consume attributes to a FlowFile
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/olegz/nifi NIFI-2298
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/nifi/pull/687.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #687
----
commit f0b62d477882b65fba84b94a0e5067749a5f6ee4
Author: Oleg Zhurakousky <[email protected]>
Date: 2016-07-20T16:18:55Z
NIFI-2298 added Kafka consume attributes to a FlowFile
----
> Add missing futures for ConsumeKafka
> ------------------------------------
>
> Key: NIFI-2298
> URL: https://issues.apache.org/jira/browse/NIFI-2298
> Project: Apache NiFi
> Issue Type: Improvement
> Components: Extensions
> Affects Versions: 0.7.0
> Reporter: sumanth chinthagunta
> Assignee: Oleg Zhurakousky
> Labels: kafka
> Fix For: 1.0.0, 0.8.0
>
>
> The new ConsumeKafka processor is missing some capabilities that were
> present in old getKafka processor.
> 1. New ConsumeKafka is not writing critical Kafka attributes i.e.,
> kafka.key, kafka.offset, kafka.partition etc into flowFile attributes.
> Old getKafka processor:
> {quote}
> Standard FlowFile Attributes
> Key: 'entryDate'
> Value: 'Sun Jul 17 15:17:00 CDT 2016'
> Key: 'lineageStartDate'
> Value: 'Sun Jul 17 15:17:00 CDT 2016'
> Key: 'fileSize'
> Value: '183'
> FlowFile Attribute Map Content
> Key: 'filename'
> Value: '19709945781167274'
> Key: 'kafka.key'
> Value: '\{"database":"test","table":"sc_job","pk.systemid":1\}'
> Key: 'kafka.offset'
> Value: '1184010261'
> Key: 'kafka.partition'
> Value: '0'
> Key: 'kafka.topic'
> Value: ‘data'
> Key: 'path'
> Value: './'
> Key: 'uuid'
> Value: '244059bb-9ad9-4d74-b1fb-312eee72124a'
> {quote}
>
> New ConsumeKafka processor :
> {quote}
> Standard FlowFile Attributes
> Key: 'entryDate'
> Value: 'Sun Jul 17 15:18:41 CDT 2016'
> Key: 'lineageStartDate'
> Value: 'Sun Jul 17 15:18:41 CDT 2016'
> Key: 'fileSize'
> Value: '183'
> FlowFile Attribute Map Content
> Key: 'filename'
> Value: '19710046870478139'
> Key: 'path'
> Value: './'
> Key: 'uuid'
> Value: '349fbeb3-e342-4533-be4c-424793fa5c59’
> {quote}
> 2. getKafka/petKafka are compatible with Kafka 0.8.x and 0.9.x .
> Please make new PublishKafka/ConsumeKafka processors based on Kafka 0.10
> version.
> 3. Support subscribing to multiple topics i.e., topic: topic1,topic2
> 4. Support configurable Serializer/DeSerializer for String, JSON , Avro etc.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
