[
https://issues.apache.org/jira/browse/FLINK-7011?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16064447#comment-16064447
]
ASF GitHub Bot commented on FLINK-7011:
---------------------------------------
GitHub user tzulitai opened a pull request:
https://github.com/apache/flink/pull/4190
[FLINK-7011] [kafka] Harden Kafka testStartFromKafkaCommitOffsets ITCases
Hardens `testStartFromKafkaCommitOffsets` in Kafka ITCases.
**Description of what the test does:**
The case verifies that whatever offset was committed to Kafka, Flink reads
it correctly and can use that as the correct starting point for exactly-once.
It is done in an end-to-end manner, verifying that the commit logic and read
offset logic is coherent.
**Problem:**
The previous implementation was too strict. It tries 3 times to fetch some
committed offsets. If none is fetched, the test fails. This PR changes it so
that we retry infinitely until some offsets are committed. In the case that no
offsets are ever committed due to incorrect offset commit logic, the test
timeouts can guard against that.
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/tzulitai/flink FLINK-7011
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/flink/pull/4190.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #4190
----
commit 264172d3b788ff73aade75116e8342c132df24c0
Author: Tzu-Li (Gordon) Tai <[email protected]>
Date: 2017-06-27T07:53:06Z
[FLINK-7011] [kafka] Harden Kafka testStartFromKafkaCommitOffsets ITCases
----
> Instable Kafka testStartFromKafkaCommitOffsets failures on Travis
> -----------------------------------------------------------------
>
> Key: FLINK-7011
> URL: https://issues.apache.org/jira/browse/FLINK-7011
> Project: Flink
> Issue Type: Bug
> Components: Kafka Connector, Tests
> Affects Versions: 1.3.1, 1.4.0
> Reporter: Tzu-Li (Gordon) Tai
> Assignee: Tzu-Li (Gordon) Tai
>
> Example:
> https://s3.amazonaws.com/archive.travis-ci.org/jobs/246703474/log.txt?X-Amz-Expires=30&X-Amz-Date=20170627T065647Z&X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=AKIAJRYRXRSVGNKPKO5A/20170627/us-east-1/s3/aws4_request&X-Amz-SignedHeaders=host&X-Amz-Signature=dbfc90cfc386fef0990325b54ff74ee4d441944687e7fdaa73ce7b0c2b2ec0ea
> In general, the test {{testStartFromKafkaCommitOffsets}} implementation is a
> bit of an overkill. Before continuing with the test, it writes some records
> just for the sake of committing offsets to Kafka and waits for some offsets
> to be committed (which leads to the instability), whereas we can do that
> simply using the test base's {{OffsetHandler}}.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
