[
https://issues.apache.org/jira/browse/APEXMALHAR-2187?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15419625#comment-15419625
]
devendra tagare commented on APEXMALHAR-2187:
---------------------------------------------
Hi,
Taking this up.
Will post the feasibility of doing this based on the offset persistence
semantics of the Kafka consumer client and then go ahead with the
implementation.
Thanks,
Dev
cc : [~siyuanhe]
> Kafka Input Operator supports retry for loading initial offset
> --------------------------------------------------------------
>
> Key: APEXMALHAR-2187
> URL: https://issues.apache.org/jira/browse/APEXMALHAR-2187
> Project: Apache Apex Malhar
> Issue Type: New Feature
> Reporter: Sunil
> Assignee: devendra tagare
>
> Kafka input operator loads initial consumer offset from Kafka/Zookeeper and
> if it doesn't exist falls back to earliest or latest based on configuration.
> In the current implementation the offset load method doesn't retry and in the
> event like timeout can reset the consumer offset. This can result in data
> loss or duplicates.
> Kafka input operator should have an attribute for retrying the initial offset
> loading instead of falling over to reset on first attempt.
>
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
