[
https://issues.apache.org/jira/browse/FLINK-41116?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Purushottam Sinha updated FLINK-41116:
--------------------------------------
Description: Bump {{kafka.version}} from 4.2.0 to 4.4.0. Requires adapting
{{FlinkKafkaInternalProducer}} to the {{TransactionalRequestResult#await}}
signature change in 4.3.0, so the connector no longer works with
{{kafka-clients}} older than 4.3.0. (was: Bump {{kafka.version}} from 4.2.0 to
4.4.0 (latest release; 4.3.x is also affected by the API change below).
Changes needed:
* {{FlinkKafkaInternalProducer#flushNewPartitions}} calls
{{TransactionalRequestResult#await()}}. Since 4.3.0 the only variant is
{{await(long, TimeUnit, String)}}. Use {{await(Long.MAX_VALUE,
TimeUnit.MILLISECONDS, ...)}} to keep the previous wait-forever behavior.
* {{org.jetbrains:annotations}} no longer converges in the e2e test module
({{kafka-clients}} -> {{zstd-jni}} wants 24.1.0, {{testcontainers}} ->
{{duct-tape}} wants 17.0.0). Add a {{dependencyManagement}} pin (24.1.0).
The other internals the connector reaches into by reflection
({{transactionalId}}, {{currentState}}, {{newPartitionsInTransaction}},
{{txnPartitionMap}}, {{transactionStarted}}, {{producerIdAndEpoch}},
{{transitionTo}}, {{addPartitionsToTransactionHandler}}) still exist in 4.4.0.
{{ProducerIdAndEpoch}} moved to {{org.apache.kafka.common.utils.internals}},
which is handled because the connector resolves it from the field type.
Compatibility note: {{await(long, TimeUnit, String)}} does not exist in 4.2.x,
so the connector can no longer be used with {{kafka-clients}} older than 4.3.0.
Verified on 4.4.0: all {{flink-connector-kafka}} unit tests and ITCases pass
(sink 147, rest 398), and {{-Pcheck-convergence}} passes.
{{KafkaWriterFaultToleranceITCase#testFlushExceptionWhenKafkaUnavailable}} is
flaky on both 4.2.0 and 4.4.0 (1 of 3 runs each). The e2e tests were not run.)
> Connectors/Kafka: Upgrade kafka-clients to 4.4.0
> ------------------------------------------------
>
> Key: FLINK-41116
> URL: https://issues.apache.org/jira/browse/FLINK-41116
> Project: Flink
> Issue Type: Improvement
> Components: Connectors / Kafka
> Reporter: Purushottam Sinha
> Priority: Minor
> Labels: pull-request-available
>
> Bump {{kafka.version}} from 4.2.0 to 4.4.0. Requires adapting
> {{FlinkKafkaInternalProducer}} to the {{TransactionalRequestResult#await}}
> signature change in 4.3.0, so the connector no longer works with
> {{kafka-clients}} older than 4.3.0.
--
This message was sent by Atlassian Jira
(v10.3.26#10030026)
