[
https://issues.apache.org/jira/browse/KAFKA-657?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
David Arthur updated KAFKA-657:
-------------------------------
Attachment: KAFKA-657v8.patch
Attaching v8
70. I have consolidated (offset, metadata, error) into a case class
OffsetMetadataAndError
71. Added InvalidOffset=-1 and NoMetadata="" to OffsetMetadataAndError as
constants
72. Done
73. Fixed by using OffsetMetadataAndError
74. Done
75. Removed: redundant test cases, unused imports, misspellings
76. Done
77. Added metadata to OffsetCommitRequest and OffsetFetchResponse. Currently
not doing anything with the metadata
Other changes:
* Added "offset.metadata.max.size" to KafkaConfig, default 1024
* Pass KafkaConfig from KafkaServer to KafkaApis (through the constructor). Not
sure if there is a more elegant way to do this, but this works
* Added OffsetMetadataTooLargeException
* Better error handling (maybe?) in the handle* methods. Instead of returning
UnknownCode always, I handle specific cases and the catch-all uses
ErrorMapping.codeFor instead of just UnknownCode
Also, I found a bug in ApiUtils.writeShortString while testing this code. I
opened up KAFKA-680 with a fix attached. Until that fix is incorporated,
OffsetCommitTest.testLargeMetadataPayload will fail.
> Add an API to commit offsets
> ----------------------------
>
> Key: KAFKA-657
> URL: https://issues.apache.org/jira/browse/KAFKA-657
> Project: Kafka
> Issue Type: New Feature
> Reporter: Jay Kreps
> Labels: project
> Attachments: KAFKA-657v1.patch, KAFKA-657v2.patch, KAFKA-657v3.patch,
> KAFKA-657v4.patch, KAFKA-657v5.patch, KAFKA-657v6.patch, KAFKA-657v7.patch,
> KAFKA-657v8.patch
>
>
> Currently the consumer directly writes their offsets to zookeeper. Two
> problems with this: (1) This is a poor use of zookeeper, and we need to
> replace it with a more scalable offset store, and (2) it makes it hard to
> carry over to clients in other languages. A first step towards accomplishing
> that is to add a proper Kafka API for committing offsets. The initial version
> of this would just write to zookeeper as we do today, but in the future we
> would then have the option of changing this.
> This api likely needs to take a sequence of
> consumer-group/topic/partition/offset entries and commit them all.
> It would be good to do a wiki design on how this would work and consensus on
> that first.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
