I've created a patch to expose the high end watermark, having this exact
requirement.
Still waiting for it to be accepted, but are using this in production at
the moment and it works quite nicely:
https://issues.apache.org/jira/browse/KAFKA-1977
On Sat, 9 May 2015 at 18:43 Charlie Knudsen
Hi,
Any update on the above patch?
Hoping you might be able to review it soon.
Thanks.
On 23 February 2015 at 21:21, Will Funnell w.f.funn...@gmail.com wrote:
Hey guys,
I created a patch based on your feedback.
Let me know what you think.
https://issues.apache.org/jira/browse/KAFKA
. (It is not correlated with the consumer's offset).
On Thu, Feb 19, 2015 at 08:58:10PM +, Will Funnell wrote:
So at what point does the log end offset change? When you commit?
On 19 February 2015 at 18:47, Joel Koshy jjkosh...@gmail.com wrote:
If I consumed up to the log end offset
the end of the committed messages in the log
(the last thing the consumer has access to). It isn't the same as the
cleaner point but is always later than it so it would work just as well.
-Jay
On Thu, Feb 19, 2015 at 8:54 AM, Will Funnell w.f.funn...@gmail.com
wrote:
I'm not sure if I
a configurable SLA for the
reader
process to catch up.
2. Make the log end offset available more easily in the consumer.
-Jay
On Wed, Feb 18, 2015 at 10:18 AM, Will Funnell
w.f.funn...@gmail.com
wrote:
We are currently using Kafka 0.8.1.1
We are currently using Kafka 0.8.1.1 with log compaction in order to
provide streams of messages to our clients.
As well as constantly consuming the stream, one of our use cases is to
provide a snapshot, meaning the user will receive a copy of every message
at least once.
Each one of these
, Will Funnell w.f.funn...@gmail.com
wrote:
We are currently using Kafka 0.8.1.1 with log compaction in order to
provide streams of messages to our clients.
As well as constantly consuming the stream, one of our use cases is to
provide a snapshot, meaning the user will receive
