[
https://issues.apache.org/jira/browse/KAFKA-858?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Neha Narkhede updated KAFKA-858:
--------------------------------
Comment: was deleted
(was: -This is a corner case bug that shows up only if, after the controlled
shutdown is issued to the broker and before the broker actually shuts down, the
high watermark checkpoint thread wakes up and writes the high watermarks.
- Since we depend on allPartitions map to checkpoint high watermarks, I think
the safest and least intrusive fix is to never remove partitions from that map.
If we do, then we run into a risk of overwriting high watermark values for a
partition.
- Stop replica request is half baked right now, but we don't need to focus on
fixing it since those changes will be pretty big, intrusive and risky.
- Apart from the fix, I cleaned up logging in two places -
1. adding/removing fetchers to print partition the way the rest of the code
does. This is required for ease of grepping for a particular partition
2. state change log during leader/follower request to remove redundant entries)
> High watermark values can be overwritten during controlled shutdown
> -------------------------------------------------------------------
>
> Key: KAFKA-858
> URL: https://issues.apache.org/jira/browse/KAFKA-858
> Project: Kafka
> Issue Type: Bug
> Components: replication
> Affects Versions: 0.8
> Reporter: Neha Narkhede
> Assignee: Neha Narkhede
> Priority: Blocker
> Labels: kafka-0.8, p1
>
> Race condition between controlled shutdown, actual process shutdown and high
> watermark checkpoint thread frequency can cause high watermark values for a
> subset of partitions to be overwritten. So even if the controller sends a
> complete list of partitions to the broker, the highwatermark for some
> partitions is still 0. This causes the follower to fetch from the leader's
> start offset.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
