[
https://issues.apache.org/jira/browse/FLINK-2974?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15000360#comment-15000360
]
ASF GitHub Bot commented on FLINK-2974:
---------------------------------------
Github user StephanEwen commented on the pull request:
https://github.com/apache/flink/pull/1341#issuecomment-155785654
I think this breaks the checkpoint atomicity. The fetcher (which emits the
data to the source context) will not lock on the checkpoint lock any more, but
on an arbitrary new object created by the offset committer.
Why can't the offset committer simply use the checkpoint lock? Blocking the
checkpoint lock for an `Arrays.copyOfRange()` call seems pretty okay.
> Add periodic offset commit to Kafka Consumer if checkpointing is disabled
> -------------------------------------------------------------------------
>
> Key: FLINK-2974
> URL: https://issues.apache.org/jira/browse/FLINK-2974
> Project: Flink
> Issue Type: Improvement
> Components: Kafka Connector
> Reporter: Robert Metzger
>
> Flink only writes the offsets from the consumer into ZK if checkpointing is
> enabled.
> We should have a similar feature to Kafka's autocommit in our consumer.
> Issue reported by user:
> http://stackoverflow.com/questions/33501574/flink-kafka-why-am-i-losing-messages
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
