Sean Quah created KAFKA-21402:
---------------------------------
Summary: Transaction coordinator installs partially loaded
transaction state when loading __transaction_state fails
Key: KAFKA-21402
URL: https://issues.apache.org/jira/browse/KAFKA-21402
Project: Kafka
Issue Type: Bug
Affects Versions: 0.11.0.0
Reporter: Sean Quah
Assignee: Sean Quah
If \{{TransactionStateManager.loadTransactionMetadata}} throws while reading a
\{{__transaction_state}} partition, it logs the error and returns the partially
built map:
{code:scala}
} catch {
case t: Throwable => error(s"Error loading transactions from
transaction log $topicPartition", t)
}
}
loadedTransactions
{code}
{\{loadTransactionsForTxnTopicPartition}} then installs the partial map as the
coordinator's cache and completes the load. For any transactional ID with
records past the failure point, the in-memory cache holds an older state than
the log does. Any transition the coordinator makes from that stale state is
appended to the log and supersedes the correct, newer records. The stale state
is then persisted, and it survives coordinator failover and reloads.
Observed case (TV2):
# Loading failed part-way with \{{OutOfMemoryError}}. The partial map held a
transactional ID as \{{ONGOING}} at epoch 22969, a transaction that had
completed hours earlier.
# The coordinator timed it out and wrote \{{PREPARE_ABORT}} at epoch 22970.
# The data partition had already seen the producer at epoch 24018, so it
rejected the marker with \{{InvalidProducerEpochException}}.
# \{{PREPARE_ABORT}} is now the latest record for the ID, so every later
coordinator hits the same failure. The transaction is stuck in
\{{PrepareAbort}}, and \{{InitProducerId}} returns \{{CONCURRENT_TRANSACTIONS}}
indefinitely.
Relates to KAFKA-21349.
--
This message was sent by Atlassian Jira
(v10.3.26#10030026)
