Hi,
Anybody could help to answer below question? If compression type could be modified through command " bin/kafka-console-producer.sh --producer.config <config file>"? Thanks! Best Regards Johnny -----Original Message----- From: ZHU Hua B Sent: 2016年10月17日 14:52 To: users@kafka.apache.org; Radoslaw Gruchalski Subject: RE: A question about kafka Hi, Thanks for your reply! OK, I got it. And, there is a parameter named compression.type in config/producer.properties, which is same usage as "--compression-codec " I think. I modify compression.type in config/producer.properties firstly, then run console producer with option "--producer.config <config file>" and send message, but the compression codec could not change as modification. Do you know the reason about it? Thanks! # bin/kafka-console-producer.sh Read data from standard input and publish it to Kafka. Option Description ------ ----------- --producer.config <config file> Producer config properties file. Note that [producer-property] takes precedence over this config. # bin/kafka-console-producer.sh --producer.config config/producer.properties --broker-list localhost:9092 --topic test Best Regards Johnny -----Original Message----- From: Hans Jespersen [mailto:h...@confluent.io] Sent: 2016年10月17日 14:29 To: users@kafka.apache.org; Radoslaw Gruchalski Subject: RE: A question about kafka Because the producer-property option is used to set other properties that are not compression type. //h...@confluent.io -------- Original message --------From: ZHU Hua B <hua.b....@alcatel-lucent.com> Date: 10/16/16 11:20 PM (GMT-08:00) To: Radoslaw Gruchalski <ra...@gruchalski.com>, users@kafka.apache.org Subject: RE: A question about kafka Hi, Thanks for your reply! If console producer only allows for compression codec argument, why we could found option —producer-property defined in ConsoleProducer.scala? And we could find the usage also if we running console producer? The version we used is Kafka 0.10.0.0. Thanks! # ./kafka-console-producer.sh Read data from standard input and publish it to Kafka. Option Description ------ ----------- --compression-codec [compression-codec] The compression codec: either 'none', 'gzip', 'snappy', or 'lz4'.If specified without value, then it defaults to 'gzip' --producer-property <producer_prop> A mechanism to pass user-defined properties in the form key=value to the producer. --producer.config <config file> Producer config properties file. Note that [producer-property] takes precedence over this config. --property <prop> A mechanism to pass user-defined properties in the form key=value to the message reader. This allows custom configuration for a user- defined message reader. Best Regards Johnny From: Radoslaw Gruchalski [mailto:ra...@gruchalski.com] Sent: 2016年10月17日 14:02 To: ZHU Hua B; users@kafka.apache.org Subject: RE: A question about kafka Hi, I believe the answer is in the code. This is where the --compression-codec is processed: https://github.com/apache/kafka/blob/trunk/core/src/main/scala/kafka/tools/ConsoleProducer.scala#L143 and this is —producer-property: https://github.com/apache/kafka/blob/trunk/core/src/main/scala/kafka/tools/ConsoleProducer.scala#L234 The usage is here: https://github.com/apache/kafka/blob/trunk/core/src/main/scala/kafka/tools/ConsoleProducer.scala#L114 The answer is: The console producer allows for compression codec only with —compression-codec argument. – Best regards, Radek Gruchalski ra...@gruchalski.com<mailto:ra...@gruchalski.com> On October 17, 2016 at 7:46:41 AM, ZHU Hua B (hua.b....@alcatel-lucent.com<mailto:hua.b....@alcatel-lucent.com>) wrote: Hi, Anybody could help to answer this question? Thanks! Best Regards Johnny -----Original Message----- From: ZHU Hua B Sent: 2016年10月14日 16:41 To: users@kafka.apache.org<mailto:users@kafka.apache.org> Subject: [COMMERCIAL] A question about kafka Hi, I have a question about kafka, could you please help to have a look? I want to send a message from producer with snappy compression codec. So I run the command "bin/kafka-console-producer.sh --compression-codec snappy --broker-list localhost:9092 --topic test", after that I checked the data log, compresscodec is SnappyCompressionCodec as expectation. Then I tried another command "bin/kafka-console-producer.sh --producer-property compression.type=snappy --broker-list localhost:9092 --topic test" to send a message from producer with snappy also, but this time I found compresscodec is NoCompressionCodec in data log. I double checked the option of "kafka-console-producer.sh" as below, seems both "--compression-codec" and "--producer-property" could be used to set up compression codec, why the second way failed? The "key=value" I found in config/producer.properties, so I think it is regular. Thanks! # bin/kafka-console-producer.sh --help --compression-codec [compression-codec] The compression codec: either 'none','gzip', 'snappy', or 'lz4'.If specified without value, then it defaults to 'gzip'. --producer-property <producer_prop> A mechanism to pass user-defined properties in the form key=value to the producer. --producer.config <config file> Producer config properties file. Note that [producer-property] takes precedence over this config. Best Regards Johnny