[
https://issues.apache.org/jira/browse/KAFKA-1000?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Tejas Patil updated KAFKA-1000:
-------------------------------
Description:
This Jira is for tracking the phase #2 of Offset Management [0]. After
KAFKA-657 [1], we have a protocol for consumers to commit and fetch offsets
from brokers. Currently, consumers are not using this API and directly talking
with Zookeeper.
Joel and I have been working on this.
[0] : https://cwiki.apache.org/confluence/display/KAFKA/Offset+Management
[1] : https://issues.apache.org/jira/browse/KAFKA-657
was:
This Jitra corresponds to the phase #2 of Offset Management [0]. After
KAFKA-657 [1], we have a protocol for consumers to commit and fetch offsets
from brokers. Currently, consumers are not using this API and directly talking
with Zookeeper.
This Jira will involve following:
1. Consumers should now use the offset commit API
2. Add a special topic in kafka for storing offsets
3. Pluggable offset commit logic so that users can pick the existing ZK based
storage or inbuilt storage for offsets.
4. Leader brokers would now maintain an additional hash table of offsets for
the group-topic-partitions that they lead
5. Consumers send their offset fetch requests to brokers. Receiving broker may
or may not be the leader for the requested group-topic-partitions so would have
to forward the request to respective leader brokers.
Joel and I have been working on this and I would share the initial patch and a
writeup covering the nitty-gritty of the implementation.
[0] : https://cwiki.apache.org/confluence/display/KAFKA/Offset+Management
[1] : https://issues.apache.org/jira/browse/KAFKA-657
> Inbuilt consumer offset management
> ----------------------------------
>
> Key: KAFKA-1000
> URL: https://issues.apache.org/jira/browse/KAFKA-1000
> Project: Kafka
> Issue Type: New Feature
> Components: consumer
> Affects Versions: 0.8.1
> Reporter: Tejas Patil
> Assignee: Tejas Patil
> Priority: Minor
> Labels: features
>
> This Jira is for tracking the phase #2 of Offset Management [0]. After
> KAFKA-657 [1], we have a protocol for consumers to commit and fetch offsets
> from brokers. Currently, consumers are not using this API and directly
> talking with Zookeeper.
> Joel and I have been working on this.
> [0] : https://cwiki.apache.org/confluence/display/KAFKA/Offset+Management
> [1] : https://issues.apache.org/jira/browse/KAFKA-657
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
