[
https://issues.apache.org/jira/browse/FLINK-5697?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16379815#comment-16379815
]
Thomas Weise edited comment on FLINK-5697 at 2/28/18 5:52 AM:
--------------------------------------------------------------
We have implemented periodic watermark support in a customization of
FlinkKinesisConsumer via FLINK-8648. The consumer accepts an optional instance
of AssignerWithPeriodicWatermarks and uses the configuration from
ExecutionConfig#setAutoWatermarkInterval for the timer interval. It also
addresses the issue described in FLINK-5479 with an (optional) interval
property for the user to specify after how much time since the last record a
shard is considered idle and should not hold back the watermark. If there is
interest, I would contribute these changes to the current Flink Kinesis
connector.
was (Author: thw):
We have implemented periodic watermark support in a customization of
FlinkKinesisConsumer via FLINK-8648. The consumer accepts an optional instance
of AssignerWithPeriodicWatermarks and uses the configuration from
ExecutionConfig#setAutoWatermarkInterval for the timer interval. It also
addresses the issue described in FLINK-5479 with an (optional) interval
property for the user to specify after how much time since the last a shard is
considered idle and should not hold back the watermark. If there is interest, I
would contribute these changes to the current Flink Kinesis connector.
> Add per-shard watermarks for FlinkKinesisConsumer
> -------------------------------------------------
>
> Key: FLINK-5697
> URL: https://issues.apache.org/jira/browse/FLINK-5697
> Project: Flink
> Issue Type: New Feature
> Components: Kinesis Connector, Streaming Connectors
> Reporter: Tzu-Li (Gordon) Tai
> Priority: Major
>
> It would be nice to let the Kinesis consumer be on-par in functionality with
> the Kafka consumer, since they share very similar abstractions. Per-partition
> / shard watermarks is something we can add also to the Kinesis consumer.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
