Tzu-Li (Gordon) Tai created FLINK-6503:
------------------------------------------
Summary: Refactor KinesisDataFetcher to separate concerns for
shard discovery
Key: FLINK-6503
URL: https://issues.apache.org/jira/browse/FLINK-6503
Project: Flink
Issue Type: Improvement
Components: Kinesis Connector
Reporter: Tzu-Li (Gordon) Tai
Currently, shard discovery is done within the `KinesisDataFetcher`'s loop. I
propose to extract that from the fetcher and encapsulate that in a separate
class to separate concerns.
As can be seen in
https://github.com/apache/flink/pull/3001/files#diff-0b702244caed9a73b7b3e1dc8a9cbaebR461,
one downside if we do not do this refactor is apparent when adding
rescalability of the Kinesis consumer. That required shards lookup operations
prioir to running the fetcher, which resulted in the need to expose shard
fetching logic external to the `KinesisDataFetcher`. This should be properly
done be separating concerns of shard discovery from the fetcher.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
