CCCLL commented on code in PR #23813:
URL: https://github.com/apache/kafka/pull/23813#discussion_r4278658407
##########
tools/src/main/java/org/apache/kafka/tools/ConsoleProducer.java:
##########
@@ -392,7 +392,9 @@ Properties producerProps() throws IOException {
props.putAll(parseKeyValueArgs(options.valuesOf(commandPropertyOpt)));
props.put(BOOTSTRAP_SERVERS_CONFIG,
options.valueOf(bootstrapServerOpt));
- props.put(COMPRESSION_TYPE_CONFIG, compressionCodec());
+ if (options.has(compressionCodecOpt) ||
!props.containsKey(COMPRESSION_TYPE_CONFIG)) {
+ props.put(COMPRESSION_TYPE_CONFIG, compressionCodec());
Review Comment:
Before this change such a value was silently replaced by `none`, so the tool
started and sent the records uncompressed. Now it reaches `ProducerConfig` like
any other producer property, and the tool exits with a `ConfigException` before
sending anything:
```
org.apache.kafka.common.config.ConfigException: Invalid value foo for
configuration compression.type: String must be one of: none, gzip, snappy, lz4,
zstd
```
An empty value (`--command-property compression.type=` or
`compression.type=` in the config file) fails the same way. This is what
already happens for `--compression-codec foo` or `--command-property acks=foo`.
I added `testInvalidCompressionTypeFromCommandProperty` for it.
##########
tools/src/test/java/org/apache/kafka/tools/ConsoleProducerTest.java:
##########
@@ -388,6 +388,59 @@ public void testCommandPropertyOverridesConfig() throws
IOException {
assertEquals(16384,
producerConfig.getInt(ProducerConfig.BATCH_SIZE_CONFIG));
}
+ @Test
+ public void testDefaultCompressionType() throws IOException {
Review Comment:
Right, `none` is also the producer's default, so that test could not catch a
regression here. I removed it.
##########
tools/src/test/java/org/apache/kafka/tools/ConsoleProducerTest.java:
##########
@@ -388,6 +388,59 @@ public void testCommandPropertyOverridesConfig() throws
IOException {
assertEquals(16384,
producerConfig.getInt(ProducerConfig.BATCH_SIZE_CONFIG));
}
+ @Test
+ public void testDefaultCompressionType() throws IOException {
+ ConsoleProducerOptions opts = new
ConsoleProducerOptions(BOOTSTRAP_SERVER_VALID_ARGS);
+ ProducerConfig producerConfig = new
ProducerConfig(opts.producerProps());
+
+ assertEquals("none",
producerConfig.getString(ProducerConfig.COMPRESSION_TYPE_CONFIG));
+ }
+
+ @Test
+ public void testCompressionTypeFromCommandProperty() throws IOException {
+ String[] args = new String[]{
+ "--bootstrap-server", "localhost:9092",
+ "--topic", "test",
+ "--command-property", "compression.type=zstd"
+ };
+
+ ConsoleProducerOptions opts = new ConsoleProducerOptions(args);
+ ProducerConfig producerConfig = new
ProducerConfig(opts.producerProps());
+
+ assertEquals("zstd",
producerConfig.getString(ProducerConfig.COMPRESSION_TYPE_CONFIG));
+ }
+
+ @Test
+ public void testCompressionTypeFromCommandConfig() throws IOException {
+ File propsFile =
ToolsTestUtils.tempPropertiesFile(Map.of("compression.type", "lz4"));
+
+ String[] args = new String[]{
+ "--bootstrap-server", "localhost:9092",
+ "--topic", "test",
+ "--command-config", propsFile.getAbsolutePath()
+ };
+
+ ConsoleProducerOptions opts = new ConsoleProducerOptions(args);
+ ProducerConfig producerConfig = new
ProducerConfig(opts.producerProps());
+
+ assertEquals("lz4",
producerConfig.getString(ProducerConfig.COMPRESSION_TYPE_CONFIG));
+ }
+
+ @Test
+ public void testCompressionCodecOverridesCommandProperty() throws
IOException {
+ String[] args = new String[]{
+ "--bootstrap-server", "localhost:9092",
+ "--topic", "test",
+ "--command-property", "compression.type=zstd",
+ "--compression-codec", "lz4"
+ };
+
+ ConsoleProducerOptions opts = new ConsoleProducerOptions(args);
+ ProducerConfig producerConfig = new
ProducerConfig(opts.producerProps());
+
+ assertEquals("lz4",
producerConfig.getString(ProducerConfig.COMPRESSION_TYPE_CONFIG));
+ }
+
Review Comment:
Added tests for a bare `--compression-codec` (it means `gzip` and still
takes precedence over `compression.type` from `--command-property`) and for the
deprecated `--producer-property` and `--producer.config`.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]