Dave wrote an excellent guide to implementing a kafka client here: http://readthedocs.org/docs/brod/en/latest/spec.html <d...@datadoghq.com>
One point that was raised in the discussion was that we don't currently have a lot of standardization or documentation for clients. For example there is no documentation on the site, inconsistent or no examples in the source, and inconsistent or no integration tests for the clients. Overall our approach to clients has been to be somewhat liberal in what we except on the assumption that something is better than nothing. Unfortunately the clients are somewhat harmed by lack of a clear model to follow. I was thinking it might be good to have a standard for these and try to move the clients towards these standards and only take new clients that meet the standards. Here is what I was thinking should be a minimum for us to take a client: - A patch for documentation http://svn.apache.org/repos/asf/incubator/kafka/site that adds quickstart style instructions for usage. We should make the existing quickstart link a list with an entry for each language. - An integration test suite which can be run against a running broker/zk on the default port and localhost. Not sure of the best format for these, but maybe just a shell script that can be invoked by doing clients/<lang>/test.sh (the tests themselves can be in the client language, this just gives a uniform way to run them as part of a regression suite). This script should exit with a non-zero code if one or more tests fail and print diagnostic information. - An entry in the examples examples/src/main/<lang> directory which gives complete code for a working example. - An owner for this client who would be willing to maintain this code going forward. This probably isn't a hugely time consuming job, but it is good to have a contact person who can be point for questions or fixes. - A self-assessment of the "production readiness" of the client. I think it is fine to take proof-of-concept code since it gives people something to work from, but it is good to be able to give people some assessment of how production-ready it is so people don't run into problems and have a negative reaction to the project as a whole. Any other criteria we should add? -Jay
