Hi Stones, Thanks for your feedback ! Maintaining documentation is an ongoing effort and any feedback on how to improve it is useful.
>> Also there's no clear way to get the latest Kafka release (no maven yet). That is because Kafka hasn't had its first incubator release yet. We are almost there, it should be available in the next few weeks. We have a download link from our home page, that points you to non-incubator releases - http://incubator.apache.org/kafka/downloads.html >> For compression, there's no example to show how to use it. Compression will be released in 0.7 which is not out yet. We will update the documentation once the release is complete. >> What serializer.class should I use when compression.codec=1 (gzip) is used ? >> Why Kafka doesn't throw exception when no serializer.class is configured? Compression codec is unrelated to the serializer. The serializer merely converts data in a particular format to a kafka Message. The default serializer is DefaultEncoder which is a no-op serializer, since the default data is assumed to be a Kafka message. You raised a good point. The producer shell and consumer shell use the StringEncoder by default, so specifying any other serializer should raise an error. This is an improvement to the ProducerShell. Please can you file a JIRA for this ? Thanks, Neha On Wed, Dec 7, 2011 at 10:39 AM, Stone <stones....@gmail.com> wrote: > Hi > > Kafka is just what we need for log aggregation and dispatching. I am play > with it recently. But found that the docs is pretty poor. I got the > following links to find docs and sources of Kafka. It's either lack of docs > or inconsistent between different sources or even broken ( eg. some > examples on [1]). > > [1] http://incubator.apache.org/kafka/index.html > [2] https://cwiki.apache.org/confluence/display/KAFKA/Index > [3] https://github.com/apache/kafka > [4] https://github.com/kafka-dev/kafka > > Also there's no clear way to get the latest Kafka release (no maven yet). > Spent some time and finally find this > > http://people.apache.org/~nehanarkhede/kafka-0.7.0-incubating-candidate-7/link > to get the lastest lib. > > For compression, there's no example to show how to use it. I tried with the > following config it works. > > val props = new Properties(); > props.put("zk.connect", "127.0.0.1:2181"); > props.put("producer.type", "async"); > props.put("batch.size", "50") > props.put("serializer.class", "kafka.serializer.StringEncoder"); > props.put("compression.codec", "1") //gzip > val config = new ProducerConfig(props); > > But if I remove the serializer.class config : props.put("serializer.class", > "kafka.serializer.StringEncoder"); The consumer-shell can no longer get the > messages published by producer, so it's like there's something wrong, but > no exception got. > > What serializer.class should I use when compression.codec=1 (gzip) is used > ? Why Kafka doesn't throw exception when no serializer.class is configured > ? > > Lack of documentation is frustrating and I have to spend many time digging > the Kafka source code. > > Thanks for your help. > > Best Regards, > Stone >
