[
https://issues.apache.org/jira/browse/FLINK-11164?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Aljoscha Krettek reassigned FLINK-11164:
----------------------------------------
Assignee: Aljoscha Krettek
> Refreshing expired shard iterator fails if no records were retrieved from
> shard since ShardConsumer was started in FlinkKinesisConsumer
> ---------------------------------------------------------------------------------------------------------------------------------------
>
> Key: FLINK-11164
> URL: https://issues.apache.org/jira/browse/FLINK-11164
> Project: Flink
> Issue Type: Bug
> Components: Kinesis Connector
> Reporter: Tzu-Li (Gordon) Tai
> Assignee: Aljoscha Krettek
> Priority: Major
>
> Originally reported by:
> [http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/Encountered-the-following-Expired-Iterator-exception-in-getRecords-using-FlinkKinesisConsumer-td25093.html]
> In {{ShardConsumer.getRecords(...)}}, we catch any expired iterator
> exceptions so that in the case no data was written to the Kinesis shard, we
> last iterator which eventually expires doesn't just fail the job, by doing:
> {code:java}
> shardItr = kinesis.getShardIterator(subscribedShard,
> ShardIteratorType.AFTER_SEQUENCE_NUMBER.toString(),
> lastSequenceNum.getSequenceNumber());
> {code}
> The problem is that, if absolutely no records were retrieved at all from the
> shard since the {{ShardConsumer}} was started, then {{lastSequenceNumber}}
> would still be a sentinel value (e.g. {{EARLIEST_SEQUENCE_NUMBER}},
> {{LATEST_SEQUENCE_NUMBER}}, etc.) instead of an actual value. This isn't
> recognized by Kinesis, and would fail the job.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
