[GitHub] flink issue #2432: [FLINK-4514][kinesis-connector] Handle unexpected Expired...
Github user tzulitai commented on the issue: https://github.com/apache/flink/pull/2432 Merging ... --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] flink issue #2432: [FLINK-4514][kinesis-connector] Handle unexpected Expired...
Github user tzulitai commented on the issue: https://github.com/apache/flink/pull/2432 Thanks for the confirmation. The build will probably fail again due to an unused import: https://travis-ci.org/tzulitai/flink/jobs/156191304, which was just hotfixed, so we need to rebase again. But I think it's ok to merge this now, because the tests for all the connectors had passed the last run before it was rebased on the bucketed rolling sink. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] flink issue #2432: [FLINK-4514][kinesis-connector] Handle unexpected Expired...
Github user rmetzger commented on the issue: https://github.com/apache/flink/pull/2432 +1 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] flink issue #2432: [FLINK-4514][kinesis-connector] Handle unexpected Expired...
Github user tzulitai commented on the issue: https://github.com/apache/flink/pull/2432 Max seems to have just hotfixed the failing flink-mesos tests. Rebasing this PR on latest master. Merging this once Travis turns green. I'll open a separate JIRA to improve the fetch interval implementation. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] flink issue #2432: [FLINK-4514][kinesis-connector] Handle unexpected Expired...
Github user rmetzger commented on the issue: https://github.com/apache/flink/pull/2432 I think a check for the interval to be lower than 5 minutes is sufficient. Setting the limit to 4.5 min seems to be a bit too strict. You never know if some advanced users want to cover a very specific use case. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] flink issue #2432: [FLINK-4514][kinesis-connector] Handle unexpected Expired...
Github user tzulitai commented on the issue: https://github.com/apache/flink/pull/2432 The latest commit sets the check to be less than 5 minutes. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] flink issue #2432: [FLINK-4514][kinesis-connector] Handle unexpected Expired...
Github user tzulitai commented on the issue: https://github.com/apache/flink/pull/2432 I think it'll also make sense to limit the config setting `ConsumerConfigConstants.SHARD_GETRECORDS_INTERVAL_MILLIS` to be lower than the shard iterator expire time, otherwise the shard iterator will definitely timeout on the next `getRecords()`. AWS documentation says the expire is 5 minutes (http://docs.aws.amazon.com/kinesis/latest/APIReference/API_GetShardIterator.html), I propose to set the limit to be 4.5 min, although I don't expect any user would actually set such a high value. Adding this now... --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] flink issue #2432: [FLINK-4514][kinesis-connector] Handle unexpected Expired...
Github user tzulitai commented on the issue: https://github.com/apache/flink/pull/2432 @rmetzger I know it might be a bit of a rush, but could you have a quick look at this too? It's not a critical blocker, but might as well would be good to make it into the 1.1.2 patch freeze. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
