[
https://issues.apache.org/jira/browse/FLINK-5697?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15970738#comment-15970738
]
Tzu-Li (Gordon) Tai edited comment on FLINK-5697 at 4/17/17 6:40 AM:
---------------------------------------------------------------------
I'd say it still makes sense. If I understand how Kinesis resharding works
correctly, merging or splitting shards actually just results in new empty
shards with new key ranges, while the previous shards that were merged or split
are simple closed (i.e., records that were collected by Kinesis before the
resharding are consumed from the old closed shards). Therefore, we can still
assume simple event time patterns per-shard, as there is no merging or
splitting of the Kinesis records taking place.
was (Author: tzulitai):
I'd say it still makes sense. If I understand how Kinesis resharding works
correctly, merging or splitting shards always result in new empty shards with
new key ranges, and the previous shards that were merged or split are simple
closed (i.e., records that were collected by Kinesis before the resharding are
consumed from the old closed shards). Therefore, we can still assume simple
event time patterns per-shard, as there is no merging or splitting of the
Kinesis records taking place.
> Add per-shard watermarks for FlinkKinesisConsumer
> -------------------------------------------------
>
> Key: FLINK-5697
> URL: https://issues.apache.org/jira/browse/FLINK-5697
> Project: Flink
> Issue Type: New Feature
> Components: Kinesis Connector, Streaming Connectors
> Reporter: Tzu-Li (Gordon) Tai
>
> It would be nice to let the Kinesis consumer be on-par in functionality with
> the Kafka consumer, since they share very similar abstractions. Per-partition
> / shard watermarks is something we can add also to the Kinesis consumer.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
