MartijnVisser opened a new pull request, #19994:
URL: https://github.com/apache/flink/pull/19994
## What is the purpose of the change
Updated Kafka Clients to resolve the issue where Flink is unable to commit
its offset back to Kafka in case of Kafka Broker becoming unavailable. This
should be resolved when the broker comes back up, but due to KAFKA-13563 that
doesn't work. Since that fix has only become available with Kafka Clients
3.1.1, this commit updates the Kafka Clients dependency from 2.8.4 to 3.1.1.
No interfaces needed to be adjusted.
It was necessary to change some of our Bash e2e tests since they still
relied on the Zookeeper parameter which has been removed in this version.
The other necessary change was adjusting the `KafkaConsumerTestBase` class
since the level of exception is changed in the new Kafka Clients so
exception.getCause().getMessage() throws an NPE in the test case.
## Brief change log
* Updated Kafka Clients dependency
* Adjusted Bash e2e tests to connect to the brokers instead of zookeeper
* Small change to `KafkaConsumerTestBase` due to change in level of exception
## Verifying this change
This change is a trivial rework / code cleanup without any test coverage.
## Does this pull request potentially affect one of the following parts:
- Dependencies (does it add or upgrade a dependency): yes
- The public API, i.e., is any changed class annotated with
`@Public(Evolving)`: no
- The serializers: no
- The runtime per-record code paths (performance sensitive): no
- Anything that affects deployment or recovery: JobManager (and its
components), Checkpointing, Kubernetes/Yarn, ZooKeeper: no
- The S3 file system connector: no
## Documentation
- Does this pull request introduce a new feature? no
- If yes, how is the feature documented? not applicable
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
