[
https://issues.apache.org/jira/browse/KAFKA-46?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Neha Narkhede updated KAFKA-46:
-------------------------------
Attachment: kafka-46-draft.patch
Attaching a draft patch for message replication. This is just to give a high
level overview of the changes involved and is by no means ready to be
committed. So no need for a detailed review. There are probably a few bugs
lurking around. Since the changes are pretty significant, I was hoping to get
some early feedback.
1. Added ReplicaFetcherThread that reads data from the leader and appends to
local replica log
2. Added highwatermark maintenance at the leader and the follower. The
highwatermark is checkpointed in the partition directory in a file named
highwatermark.
3. Added ISR maintenance logic on the leader. This involves possibly expanding
the ISR while handling a fetch request from a follower.
4. Also added ISRExpirationThread that tracks the highwatermark update time for
all partitions that the leader owns and shrinks it if (hw update time +
keep.in.sync.time.ms) < current time.
5. Note that to get this patch going, I had to put in code to cover KAFKA-302.
I would encourage a more detailed review for the becomeLeader() and
becomeFollower() APIs. We would either like to check it in from this patch, or
if Prashanth has some patch, review that one too.
I will probably add v1 patch with unit tests early next week.
Also, I would like to check this in parts, if possible. Starting with probably
KAFKA-302, then the actual message replication logic. But that is open for
discussion as well.
> Commit thread, ReplicaFetcherThread for intra-cluster replication
> -----------------------------------------------------------------
>
> Key: KAFKA-46
> URL: https://issues.apache.org/jira/browse/KAFKA-46
> Project: Kafka
> Issue Type: Bug
> Reporter: Jun Rao
> Assignee: Neha Narkhede
> Attachments: kafka-46-draft.patch
>
>
> We need to implement the commit thread at the leader and the fetcher thread
> at the follower for replication the data from the leader.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
