[
https://issues.apache.org/jira/browse/KAFKA-20753?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Bill Bejeck resolved KAFKA-20753.
---------------------------------
Resolution: Fixed
> Transactional RocksDB store fails to reopen with transactional status marker
> in EOS
> -----------------------------------------------------------------------------------
>
> Key: KAFKA-20753
> URL: https://issues.apache.org/jira/browse/KAFKA-20753
> Project: Kafka
> Issue Type: Bug
> Components: streams
> Affects Versions: 4.4.0
> Reporter: Bill Bejeck
> Assignee: Nicholas Telford
> Priority: Blocker
> Fix For: 4.4.0
>
>
> -With {{processing.guarantee=exactly_once_v2}} and
> {{enable.transactional.statestores=true,}} a rebalance that adds a
> StreamThread can transiently co-locate the same task's active and standby on
> one-
> -instance (different threads). The freshly-assigned active opens the
> persistent RocksDB store while the live sibling standby still holds it open,
> so the store-open status check rejects it and the task enters a
> {{TaskCorrupted → re-init → corrupt}} loop.-
> When {{{}enable.transactional.statestores=true{}}}, the KIP-1035 open/closed
> status marker written to the RocksDB offsets column family on {{open()}}
> and {{close()}} was being routed through
> {{{}TransactionalDBAccessor.put(){}}},
> which stages writes in the in-memory {{{}RocksDBTransactionBuffer{}}}. The
> buffer is discarded without committing on store close, so the CLOSED
> marker was silently lost. The on-disk status remained permanently at
> OPEN, causing every subsequent {{open()}} under EOS
> ({{{}ignoreInvalidState=false{}}}) to throw
> {{{}ProcessorStateException("Invalid state during store open..."){}}}, which
> escalated to
> {{{}TaskCorruptedException{}}}. Because the wipe-and-restore cycle would
> reproduce the same condition, this created an unrecoverable loop
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
