peferron commented on issue #7039: Kafka Ingestion Spec to allow kafka topic with gzip compression URL: https://github.com/apache/incubator-druid/issues/7039#issuecomment-461913177 Do you mean that the Kafka messages themselves are compressed with gzip, either via a producer or broker configuration? If that's the case, then Druid can automatically handle decompression, just like most Kafka consumers. Try completely removing your `indexSpec` and giving it another shot. `dimensionCompression` and `metricCompression` inside the `indexSpec` do not refer to how Druid decompresses messages from Kafka—they refer to how Druid compresses dimension and metric columns *after* consuming messages from Kafka. Decompression speed is usually critical there, which is why the doc lists `LZ4` and `LZF` as possible formats, but not `gzip`, which would be too slow.
---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected] With regards, Apache Git Services --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
