Neha, Thanks for your explanation.
I added a jira issue about the missing serializer.class bug: https://issues.apache.org/jira/browse/KAFKA-224 . Best Regards, Stone On Thu, Dec 8, 2011 at 2:56 AM, Neha Narkhede <neha.narkh...@gmail.com>wrote: > Hi Stones, > > Thanks for your feedback ! Maintaining documentation is an ongoing effort > and any feedback on how to improve it is useful. > > >> Also there's no clear way to get the latest Kafka release (no maven > yet). > > That is because Kafka hasn't had its first incubator release yet. We are > almost there, it should be available in the next few weeks. > We have a download link from our home page, that points you to > non-incubator releases - http://incubator.apache.org/kafka/downloads.html > > >> For compression, there's no example to show how to use it. > > Compression will be released in 0.7 which is not out yet. We will update > the documentation once the release is complete. > > >> What serializer.class should I use when compression.codec=1 (gzip) is > used ? > >> Why Kafka doesn't throw exception when no serializer.class is > configured? > > Compression codec is unrelated to the serializer. The serializer merely > converts data in a particular format to a kafka Message. The default > serializer is DefaultEncoder which is a no-op serializer, since the default > data is assumed to be a Kafka message. > > You raised a good point. The producer shell and consumer shell use the > StringEncoder by default, so specifying any other serializer should raise > an error. > This is an improvement to the ProducerShell. Please can you file a JIRA for > this ? > > Thanks, > Neha > > > On Wed, Dec 7, 2011 at 10:39 AM, Stone <stones....@gmail.com> wrote: > > > Hi > > > > Kafka is just what we need for log aggregation and dispatching. I am play > > with it recently. But found that the docs is pretty poor. I got the > > following links to find docs and sources of Kafka. It's either lack of > docs > > or inconsistent between different sources or even broken ( eg. some > > examples on [1]). > > > > [1] http://incubator.apache.org/kafka/index.html > > [2] https://cwiki.apache.org/confluence/display/KAFKA/Index > > [3] https://github.com/apache/kafka > > [4] https://github.com/kafka-dev/kafka > > > > Also there's no clear way to get the latest Kafka release (no maven yet). > > Spent some time and finally find this > > > > > http://people.apache.org/~nehanarkhede/kafka-0.7.0-incubating-candidate-7/link > > to get the lastest lib. > > > > For compression, there's no example to show how to use it. I tried with > the > > following config it works. > > > > val props = new Properties(); > > props.put("zk.connect", "127.0.0.1:2181"); > > props.put("producer.type", "async"); > > props.put("batch.size", "50") > > props.put("serializer.class", "kafka.serializer.StringEncoder"); > > props.put("compression.codec", "1") //gzip > > val config = new ProducerConfig(props); > > > > But if I remove the serializer.class config : > props.put("serializer.class", > > "kafka.serializer.StringEncoder"); The consumer-shell can no longer get > the > > messages published by producer, so it's like there's something wrong, but > > no exception got. > > > > What serializer.class should I use when compression.codec=1 (gzip) is > used > > ? Why Kafka doesn't throw exception when no serializer.class is > configured > > ? > > > > Lack of documentation is frustrating and I have to spend many time > digging > > the Kafka source code. > > > > Thanks for your help. > > > > Best Regards, > > Stone > > >
