Kirk True created KAFKA-15767:
---------------------------------
Summary: Redesign TransactionManager to avoid use of ThreadLocal
Key: KAFKA-15767
URL: https://issues.apache.org/jira/browse/KAFKA-15767
Project: Kafka
Issue Type: Improvement
Components: clients, producer
Affects Versions: 3.6.0
Reporter: Kirk True
Assignee: Kirk True
Fix For: 3.7.0
A {{TransactionManager}} instance is created by the {{KafkaProducer}} and
shared with the {{Sender}} thread. The {{TransactionManager}} has internal
states through which it transitions as part of its initialization, transaction
management, shutdown, etc. It contains logic to ensure that those state
transitions are valid, such that when an invalid transition is attempted, it is
handled appropriately.
The issue is, the definition of "handled appropriately" depends on which thread
is making the API call that is attempting an invalid transition. The
application thread expects that the invalid transition will generate an
exception. However, the sender thread expects that the invalid transition will
"poison" the entire {{TransactionManager}} instance.
So as part of the implementation of KAFKA-14831, we needed a way to change
logic in the {{TransactionManager}} on a per-thread basis, so a {{ThreadLocal}}
instance variable was added to the {{TransactionManager}} to keep track of
this. However, the use of ThreadLocal instance variables is generally
discouraged because of their tendency for memory leaks, shared state across
multiple threads, and not working with virtual threads.
The initial implementation attempt of KAFKA-14831 used a context object, passed
in to each method, to affect the logic. However, the number of methods that
needed to be changed to accept this new context object grew until most of the
methods in {{TransactionManager}} needed to be updated. Thus all the affected
call sites needed to be updated, resulting in a much larger change than
anticipated.
The task here is to remove the use of the {{ThreadLocal}} instance variable,
let the application thread and {{Sender}} thread keep their desired behavior,
but keep the change to a minimum.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
