[
https://issues.apache.org/jira/browse/KAFKA-20716?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=18101291#comment-18101291
]
Yunseop Eom commented on KAFKA-20716:
-------------------------------------
Updated PR #22937 with the maintainer-requested end-to-end regression test and
pushed commit 3e78ad59a1. The test deterministically aligns the data leader
with the transaction coordinator, waits for the follower to replicate the
transactional record, stops the follower before the transaction marker is
committed, performs an unclean election, and verifies that a read_committed
consumer returns only the earlier non-transactional record and remains at
offset 1.
Verification passed in both Raft-Isolated and Raft-Combined variants:
- ./gradlew :server:test --tests
org.apache.kafka.server.UncleanLeaderElectionTest --no-build-cache
--console=plain
- ./gradlew :server:test --tests
org.apache.kafka.server.UncleanLeaderElectionTest.testUncleanLeaderElectionCanLeaveReadCommittedConsumerAtLastStableOffset
--no-build-cache --console=plain
- ./gradlew :server:spotlessCheck :server:checkstyleMain :server:checkstyleTest
:server:spotbugsMain
PR: https://github.com/apache/kafka/pull/22937
> LSO stuck after unclean leader election
> ---------------------------------------
>
> Key: KAFKA-20716
> URL: https://issues.apache.org/jira/browse/KAFKA-20716
> Project: Kafka
> Issue Type: Bug
> Reporter: Luke Chen
> Priority: Major
>
> When a topic has an unclean leader election, the new leader might contain txn
> data without COMMIT/ABORT markers. However, the data in __transaction_state
> shows the transaction is committed/aborted, so the transaction timeout will
> not expire here. This causes the LSO stuck and READ_COMMITTED will never
> proceed.
>
> reproduce steps:
> 1. Create a cluster with 2 brokers
> 2. Create a topic with unclean leader election enabled
> {code:java}
> bin/kafka-topics.sh --create --topic t1 --bootstrap-server localhost:9091
> --replication-factor 2 --config unclean.leader.election.enable=true {code}
> 3. write a txn record to the topic t1, but don't commit it, yet.
> 4. Before the record committed in step (3), shutdown the follower broker
> (suppose it's broker 2)
> 5. commit the step (3) record.
> 6. Now, the the topic t1-0 in broker 1 contains [offset 0 (data) and offset 1
> (commit)], but broker 2 only contains [offset 0 (data)]
> 7. shutdown broker 1, so both broker 1 and 2 are down, but broker 2 is not
> the last leader or ELR
> 8. start up broker 2, unclean leader election triggered
> 9. start up broker 1, log truncation on t1-0, so the log becomes [offset 0
> (data)]
> 10. appending more non-txn data to t1-0
> 11. consume with READ_COMMITTED, it'll return nothing.
>
> We never document anywhere about unclean leader election unsupported in
> transaction feature, I think this should be supported and we have to find out
> a solution for it.
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
