Hi Taylor,
I've added a mention about Kafka's lack of an index to the client/driver
doc, since it might confuse new users. I'll include your methods on how to
cope when I write more end-user documentation.
FWIW, we ended up going with option 1, storing the history in a DB. Unlike
your N-messages
Great work!
Having a solid doc/spec for the wire protocol would also likely decrease
the need to have as many clients in-tree. I think at this point that
would be a good thing.
On 11/23/2011 06:32 PM, David Ormsbee wrote:
> Hi folks,
>
> Inspired by the Wire Format wiki entry, I recently create
One thing we should make clear somewhere is that while Kafka has a history
mechanism, it doesn't provide an index.
I probably moved forward in my implementation (and selection) to use Kafka
for 3-4 weeks before realizing that I would not be able to efficiently
query Kafka for the N-1000th message.
I thought that in this context, "driver" would extend to cover the
Clojure-friendly wrapper, and "client programs" would be reserved for
something higher level still, like command-line or GUI interfaces,
utilities that do certain complex operations (e.g. loading a text file
as a series of messages,
This terminology is a little confusing to me, what is the concrete
difference between a driver and a client? Are we saying that the current
scala client code is a "driver" and say, a Clojure-friendly wrapper for the
scala or java driver would be an example of a "client"? If so do we need to
call ou
Hi Taylor,
Yeah, Joe brought up the need for this distinction as well. When I
move the doc over to the wiki, I'll try to consistently use "driver"
to clear up ambiguities. The bits that are more higher-level client
oriented are really just there for context, to explain why the network
protocol is
Just wanted to add my $0.02 - I'm glad David wrote this - excellent job sir!
My comment is this (I think it might have already been mentioned, however I
will re-iterate it): the document as is covers two audiences - those that
are writing Kafka "drivers" and those that are writing clients that pu
HI Jay,
> 1. Would you be willing to add this to the kafka wiki so we could make
> this the official howto doc?
Absolutely.
> 2. It might be good to add a "how to contribute your client" section.
> This would be hard to write right now because we haven't given anyone any
> guidelines f
Dave,
This is really great. Two things:
1. Would you be willing to add this to the kafka wiki so we could make
this the official howto doc?
2. It might be good to add a "how to contribute your client" section.
This would be hard to write right now because we haven't given anyone any
Hi Joe,
> One thing I wanted to float is "client" vs "client" meaning a new (or
> existing) language implementing a new API vs wrapping an existing API and
> confusion between the two.
>
> I think your post is geared towards the former which is really awesome
> because it helps devs jump into Kafk
Hi Dave, this is really helpful, useful, informative and cool.
One thing I wanted to float is "client" vs "client" meaning a new (or
existing) language implementing a new API vs wrapping an existing API and
confusion between the two.
I think your post is geared towards the former which is really
Thanks a lot Dave, this is quite helpful.
- pyr
On Wed, 23 Nov 2011 18:32:12 -0500
David Ormsbee wrote:
> Hi folks,
>
> Inspired by the Wire Format wiki entry, I recently created a draft of
> "Writing a Client for Kafka":
>
> http://readthedocs.org/docs/brod/en/latest/spec.html
>
> I tri
Hi folks,
Inspired by the Wire Format wiki entry, I recently created a draft of
"Writing a Client for Kafka":
http://readthedocs.org/docs/brod/en/latest/spec.html
I tried to make it the document that I wish we had at Datadog when we
were total Kafka newbies writing client code. It's still very
13 matches
Mail list logo
