[
https://issues.apache.org/jira/browse/FLINK-5697?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15970738#comment-15970738
]
Tzu-Li (Gordon) Tai edited comment on FLINK-5697 at 4/17/17 6:40 AM:
---------------------------------------------------------------------
I'd say it still makes sense. If I understand how Kinesis resharding works
correctly, merging or splitting shards actually just results in new empty
shards with new key ranges, while the previous shards that were merged or split
are simple closed (i.e., records that were collected by Kinesis before the
resharding are consumed from the old closed shards). Therefore, we can still
assume simple event time patterns per-shard, as there is no merging or
splitting of the Kinesis records taking place and their event time pattern is
kept intact.
was (Author: tzulitai):
I'd say it still makes sense. If I understand how Kinesis resharding works
correctly, merging or splitting shards actually just results in new empty
shards with new key ranges, while the previous shards that were merged or split
are simple closed (i.e., records that were collected by Kinesis before the
resharding are consumed from the old closed shards). Therefore, we can still
assume simple event time patterns per-shard, as there is no merging or
splitting of the Kinesis records taking place.
> Add per-shard watermarks for FlinkKinesisConsumer
> -------------------------------------------------
>
> Key: FLINK-5697
> URL: https://issues.apache.org/jira/browse/FLINK-5697
> Project: Flink
> Issue Type: New Feature
> Components: Kinesis Connector, Streaming Connectors
> Reporter: Tzu-Li (Gordon) Tai
>
> It would be nice to let the Kinesis consumer be on-par in functionality with
> the Kafka consumer, since they share very similar abstractions. Per-partition
> / shard watermarks is something we can add also to the Kinesis consumer.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
