Shekhar Prasad Rajak created KAFKA-20738:
--------------------------------------------
Summary: Producer 2PC init sets v6-only InitProducerId fields but
request builder caps version at v5
Key: KAFKA-20738
URL: https://issues.apache.org/jira/browse/KAFKA-20738
Project: Kafka
Issue Type: Bug
Components: core
Affects Versions: 4.2.1
Reporter: Shekhar Prasad Rajak
Assignee: Shekhar Prasad Rajak
Fix For: 4.3.0
Kafka producer exposes transaction.two.phase.commit.enable=true, but a producer
cannot currently initialize a 2PC transaction through the public API.
TransactionManager sets Enable2Pc=true in
InitProducerIdRequestData, which is only valid in InitProducerId v6. However,
InitProducerIdRequest.Builder is constructed with the default stable-version
range, so NetworkClient selects v5 and serialization
fails.
Example:
Properties props = new Properties();
props.put(ProducerConfig.BOOTSTRAP_SERVERS_CONFIG, bootstrap);
props.put(ProducerConfig.TRANSACTIONAL_ID_CONFIG, "txn-2pc");
props.put(ProducerConfig.ENABLE_IDEMPOTENCE_CONFIG, "true");
props.put(ProducerConfig.TRANSACTION_TWO_PHASE_COMMIT_ENABLE_CONFIG, "true");
KafkaProducer<byte[], byte[]> producer = new KafkaProducer<>(props);
producer.initTransactions();
Current behavior:
UnsupportedVersionException:
Attempted to write a non-default enable2Pc at version 5
so here :
KafkaProducer
-> TransactionManager.initializeTransactions()
-> InitProducerIdRequestData.setEnable2Pc(true)
-> new InitProducerIdRequest.Builder(data)
-> AbstractRequest.Builder(ApiKeys.INIT_PRODUCER_ID) allows stable versions
only
-> NetworkClient selects InitProducerId v5
-> serialization fails because Enable2Pc is v6-only
Expected behavior:
When Enable2Pc or KeepPreparedTxn is set, the InitProducerId builder should
allow v6. NetworkClient should then select v6 if the broker advertises it
through ApiVersions. If the broker does not support v6, the
producer should fail cleanly with a clear unsupported-version error.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
