hmmmm .... i would like to repeat the suggestion i made earlier:
i would keep all clients -- except the scala reference implementation --
out of the kafka distribution.
this is the model that redis follows, for example, and it works really
well.
no need to officially "accept" clients either.
tim
On 2011-11-29, at 8:51 AM, Jay Kreps wrote:
Dave wrote an excellent guide to implementing a kafka client here:
http://readthedocs.org/docs/brod/en/latest/spec.html <d...@datadoghq.com
>
One point that was raised in the discussion was that we don't
currently
have a lot of standardization or documentation for clients. For
example
there is no documentation on the site, inconsistent or no examples
in the
source, and inconsistent or no integration tests for the clients.
Overall
our approach to clients has been to be somewhat liberal in what we
except
on the assumption that something is better than nothing.
Unfortunately the
clients are somewhat harmed by lack of a clear model to follow.
I was thinking it might be good to have a standard for these and try
to
move the clients towards these standards and only take new clients
that
meet the standards. Here is what I was thinking should be a minimum
for us
to take a client:
- A patch for documentation
http://svn.apache.org/repos/asf/incubator/kafka/site that adds
quickstart style instructions for usage. We should make the existing
quickstart link a list with an entry for each language.
- An integration test suite which can be run against a running
broker/zk
on the default port and localhost. Not sure of the best format for
these,
but maybe just a shell script that can be invoked by doing
clients/<lang>/test.sh (the tests themselves can be in the client
language,
this just gives a uniform way to run them as part of a regression
suite).
This script should exit with a non-zero code if one or more tests
fail and
print diagnostic information.
- An entry in the examples examples/src/main/<lang> directory which
gives complete code for a working example.
- An owner for this client who would be willing to maintain this
code
going forward. This probably isn't a hugely time consuming job,
but it is
good to have a contact person who can be point for questions or
fixes.
- A self-assessment of the "production readiness" of the client. I
think
it is fine to take proof-of-concept code since it gives people
something to
work from, but it is good to be able to give people some
assessment of how
production-ready it is so people don't run into problems and have a
negative reaction to the project as a whole.
Any other criteria we should add?
-Jay
--
http://tim.lossen.de
