ChenXi created KAFKA-21481:
------------------------------

             Summary: ListTransactions can return transactions from 
no-longer-owned partitions
                 Key: KAFKA-21481
                 URL: https://issues.apache.org/jira/browse/KAFKA-21481
             Project: Kafka
          Issue Type: Bug
          Components: core
            Reporter: ChenXi


h1. Problem and reproduction

TransactionStateManager.listTransactionStates iterates the entire
transactionMetadataCache and returns Errors.NONE. The cache entry is removed 
only
by TransactionCoordinator.onResignation, which runs after the replica has 
already
become a follower. During that window, a broker can return transactions for a
__transaction_state partition it no longer leads.

Affected source in trunk:

TransactionStateManager.scala:331-392 (listTransactionStates), especially :380:
{code:scala}
transactionMetadataCache.foreachEntry { (_, cache) =>
  // partition id discarded; no leadership check
  cache.metadataPerTransactionalId.forEach { ... }
}
{code}
TransactionCoordinator.scala:299-310 only checks the process-wide isActive flag:
{code:scala}
if (!isActive.get()) {
  ... COORDINATOR_NOT_AVAILABLE
} else {
  txnManager.listTransactionStates(...)
}
{code}
The leadership helper exists but is used only by the point-lookup path:
{code:scala}
// TransactionStateManager.scala:403-405
private def isCoordinatorFor(partitionId: Int, coordinatorEpoch: Int): Boolean 
= ...

// TransactionStateManager.scala:423-424 (getAndMaybeAddTransactionState only)
case Some(cacheEntry) if !isCoordinatorFor(partitionId, 
cacheEntry.coordinatorEpoch) =>
  Left(Errors.NOT_COORDINATOR)
{code}
Cache removal still happens in onResignation after leadership loss:
TransactionCoordinator.scala:497-505.

Reproduction:

1. Load a transaction metadata cache entry for partition P at coordinator epoch 
E.
2. Make the local replica a follower for P, or move its leader epoch away from 
E,
while keeping the cache entry present (the pre-onResignation state).
3. Call ListTransactions on that broker.
4. The response has error NONE and still includes P's transactional IDs. 
Expected:
P is skipped.

This can be covered by a unit test in TransactionStateManagerTest using the
existing partitionLeadership mock.

h1. Relationship to KAFKA-21181 and difference

This is a follow-up, not a duplicate. KAFKA-21181 added isCoordinatorFor and
applied it only to getAndMaybeAddTransactionState, returning NOT_COORDINATOR for
point lookups, EndTxn, AddPartitions and InitProducerId.

The listTransactionStates path was not updated. It still uses
foreachEntry

{ (_, cache) => ... }

, loses the partition id and never calls
isCoordinatorFor.

The expected semantics differ deliberately: point requests should return
NOT_COORDINATOR so the client rediscovers the coordinator; ListTransactions is a
cluster-wide administrative aggregation and should skip stale partitions while
continuing to return transactions from partitions the broker still owns.

*Proposed fix:* call isCoordinatorFor before reading each cache entry in
listTransactionStates, add regression tests, and document that no wire/API/KIP
change is required.



--
This message was sent by Atlassian Jira
(v10.3.26#10030026)

Reply via email to