[
https://issues.apache.org/jira/browse/KAFKA-2076?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14389839#comment-14389839
]
Jiangjie Qin commented on KAFKA-2076:
-------------------------------------
Return a map of time->offset would be useful.
I cannot think of use cases where people want to know the value of earliest
offset. But it is covered by the map anyway.
I like the interface. It looks cleaner.
One concern I have is that most of the time user only cares about log end
offsets which is the only continuously updated offset. Using this interface, in
order to get the LEO, they have to get the entire map. This is probably OK if
we have coarse granularity for timestamps, but could have some overhead if we
have finer granularity, because the LEO might potentially be queried frequently
and the returned map might be large.
Maybe we can add an interface:
{code}
PartitionOffsets offsetTimes(TopicPartition tp, int numOffsets)
{code}
So it returns the numOffsets entries of time->offset back traced from LEO. So
people set numOffsets=1 will only get LEO.
And I think batched request might be better. Otherwise for a consumer consuming
from many partitions, it has to send many separate requests. It is not
efficient and might also complicate the user code.
> Add an API to new consumer to allow user get high watermark of partitions.
> --------------------------------------------------------------------------
>
> Key: KAFKA-2076
> URL: https://issues.apache.org/jira/browse/KAFKA-2076
> Project: Kafka
> Issue Type: Improvement
> Reporter: Jiangjie Qin
>
> We have a use case that user wants to know how far it is behind a particular
> partition on startup. Currently in each fetch response, we have high
> watermark for each partition, we only keep a global max-lag metric. It would
> be better that we keep a record of high watermark per partition and update it
> on each fetch response. We can add a new API to let user query the high
> watermark.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
