Hi Ivan, Giuseppe, Greg, and all,
I am following up on the commitment I made in the KIP-1150 VOTE thread.
A little history so it is easy to follow:
- 2026-01-26 (KIP-1150 vote): I objected on two design gaps -
missing storage GC, and the exactly-once (EOS) coordinator-sync gap - and
asked to pause.
- 2026-02-25 (KIP-1150 vote): I withdrew the storage-leak objection
after the KIP-1163 reconciliation loop was added, then withdraw my
objection entirely and voted +1 (non-binding), on the understanding
that the EOS details would be finalized in KIP-1164.
*I am now moving those EOS questions here, as promised.*
*RESOLVED (acknowledging on the record)* - *Orphan files / storage leak: *the
WAL file management and Orphan files sections, the owner-handover protocol,
and DisklessDescribeFiles (HasLiveBatches) address this. Thank you.
*- Idempotent produce*: DisklessCommitFile carrying
ProducerId/ProducerEpoch/BaseSequence/LastSequence with an IsDuplicate
result is exactly right.
*STILL OPEN - Exactly-once (LSO + transaction protocol)*KIP-1164 states
that "Some operations remain out of scope of this KIP, such as the ones for
transaction management," while DisklessCommitFile says it performs
"transactional checks." Those two lines are in slight tension, and that gap
is where read_committed correctness lives. One small thing to align first:
the DC role text says it manages "producer and transaction state," which
reads against the "out of scope" note - a sentence on where the boundary
sits would help.
*Concretely, checked against 4.4.0-SNAPSHOT:*
* 1. No LSO in the read path*. DisklessFindBatchesResponse
returns LogStartOffset and HighWatermark, but no LastStableOffset. In
classic Kafka a read_committed consumer only reads up to the LSO, and
the LSO is computed on the partition leader from in-memory producer state:
ProducerStateManager.firstUnstableOffset() derived from the
ongoingTxns and unreplicatedTxns maps
(storage/src/main/java/org/apache/kafka/storage/internals/log/ProducerStateManager.java).
KIP-1150 removes the leader, so where is ongoing-transaction state
tracked, how does the Diskless Coordinator compute the LSO, and how is it
returned to the fetching broker?
2. No aborted-transaction data. A read_committed Fetch response must
carry the list of aborted transactions so the consumer can skip
them (FetchResponse AbortedTransactions, v4+).
DisklessFindBatches returns batch coordinates but no aborted-transaction
metadata. Where does that come from in the diskless path?
3. No Transaction-Coordinator to Diskless-Coordinator
protocol.DisklessCommitFile carries ProducerId/ProducerEpoch for
idempotence, but there are no transaction control-marker records
(commit/abort) and no defined handshake with the Transaction Coordinator.
When the Transaction Coordinator writes a commit/abort marker, what flow
lets the DC confirm it has sequenced all data batches for that
producer/epoch before the marker takes effect?
4. The delayed-data race. DisklessCommitFile assigns offsets at
commit time, and a producer's upload and commit are separate steps. If a
broker stalls after uploading a transaction's data batch but before
committing its coordinates, and the commit marker is sequenced at the DC
first, does the DC hold until the data is committed, or can it advance past
a transaction whose data is not yet visible? That would be a read_committed
isolation violation if unhandled.
I am not asking to design all of the EOS in this KIP. But "out of scope"
plus "transactional checks" leaves the correctness-critical piece undefined
across every diskless KIP. Could we either (a) specify the LSO computation,
the aborted-transaction path, and the Txn-Coordinator/DC protocol here,
or (b) name the specific KIP that will own them and mark
DisklessCommitFile's "transactional checks" as provisional until that KIP
lands? I would rather we not vote on the coordinator contract with that
piece pointing at a document that does not exist yet.
Happy to help review whichever KIP takes this on - it is also close to
the KIP-1279 transaction-marker work I have been in.
Regards,
Vaquar Khan
LinkedIn - https://www.linkedin.com/in/vaquar-khan-b695577/
