CCCLL commented on code in PR #23813:
URL: https://github.com/apache/kafka/pull/23813#discussion_r4278658407


##########
tools/src/main/java/org/apache/kafka/tools/ConsoleProducer.java:
##########
@@ -392,7 +392,9 @@ Properties producerProps() throws IOException {
 
             
props.putAll(parseKeyValueArgs(options.valuesOf(commandPropertyOpt)));
             props.put(BOOTSTRAP_SERVERS_CONFIG, 
options.valueOf(bootstrapServerOpt));
-            props.put(COMPRESSION_TYPE_CONFIG, compressionCodec());
+            if (options.has(compressionCodecOpt) || 
!props.containsKey(COMPRESSION_TYPE_CONFIG)) {
+                props.put(COMPRESSION_TYPE_CONFIG, compressionCodec());

Review Comment:
   Before this change such a value was silently replaced by `none`, so the tool 
started and sent the records uncompressed. Now it reaches `ProducerConfig` like 
any other producer property, and the tool exits with a `ConfigException` before 
sending anything:
   
   ```
   org.apache.kafka.common.config.ConfigException: Invalid value foo for 
configuration compression.type: String must be one of: none, gzip, snappy, lz4, 
zstd
   ```
   
   An empty value (`--command-property compression.type=` or 
`compression.type=` in the config file) fails the same way. This is what 
already happens for `--compression-codec foo` or `--command-property acks=foo`. 
I added `testInvalidCompressionTypeFromCommandProperty` for it.



##########
tools/src/test/java/org/apache/kafka/tools/ConsoleProducerTest.java:
##########
@@ -388,6 +388,59 @@ public void testCommandPropertyOverridesConfig() throws 
IOException {
         assertEquals(16384, 
producerConfig.getInt(ProducerConfig.BATCH_SIZE_CONFIG));
     }
 
+    @Test
+    public void testDefaultCompressionType() throws IOException {

Review Comment:
   Right, `none` is also the producer's default, so that test could not catch a 
regression here. I removed it.



##########
tools/src/test/java/org/apache/kafka/tools/ConsoleProducerTest.java:
##########
@@ -388,6 +388,59 @@ public void testCommandPropertyOverridesConfig() throws 
IOException {
         assertEquals(16384, 
producerConfig.getInt(ProducerConfig.BATCH_SIZE_CONFIG));
     }
 
+    @Test
+    public void testDefaultCompressionType() throws IOException {
+        ConsoleProducerOptions opts = new 
ConsoleProducerOptions(BOOTSTRAP_SERVER_VALID_ARGS);
+        ProducerConfig producerConfig = new 
ProducerConfig(opts.producerProps());
+
+        assertEquals("none", 
producerConfig.getString(ProducerConfig.COMPRESSION_TYPE_CONFIG));
+    }
+
+    @Test
+    public void testCompressionTypeFromCommandProperty() throws IOException {
+        String[] args = new String[]{
+            "--bootstrap-server", "localhost:9092",
+            "--topic", "test",
+            "--command-property", "compression.type=zstd"
+        };
+
+        ConsoleProducerOptions opts = new ConsoleProducerOptions(args);
+        ProducerConfig producerConfig = new 
ProducerConfig(opts.producerProps());
+
+        assertEquals("zstd", 
producerConfig.getString(ProducerConfig.COMPRESSION_TYPE_CONFIG));
+    }
+
+    @Test
+    public void testCompressionTypeFromCommandConfig() throws IOException {
+        File propsFile = 
ToolsTestUtils.tempPropertiesFile(Map.of("compression.type", "lz4"));
+
+        String[] args = new String[]{
+            "--bootstrap-server", "localhost:9092",
+            "--topic", "test",
+            "--command-config", propsFile.getAbsolutePath()
+        };
+
+        ConsoleProducerOptions opts = new ConsoleProducerOptions(args);
+        ProducerConfig producerConfig = new 
ProducerConfig(opts.producerProps());
+
+        assertEquals("lz4", 
producerConfig.getString(ProducerConfig.COMPRESSION_TYPE_CONFIG));
+    }
+
+    @Test
+    public void testCompressionCodecOverridesCommandProperty() throws 
IOException {
+        String[] args = new String[]{
+            "--bootstrap-server", "localhost:9092",
+            "--topic", "test",
+            "--command-property", "compression.type=zstd",
+            "--compression-codec", "lz4"
+        };
+
+        ConsoleProducerOptions opts = new ConsoleProducerOptions(args);
+        ProducerConfig producerConfig = new 
ProducerConfig(opts.producerProps());
+
+        assertEquals("lz4", 
producerConfig.getString(ProducerConfig.COMPRESSION_TYPE_CONFIG));
+    }
+

Review Comment:
   Added tests for a bare `--compression-codec` (it means `gzip` and still 
takes precedence over `compression.type` from `--command-property`) and for the 
deprecated `--producer-property` and `--producer.config`.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]

Reply via email to