[
https://issues.apache.org/jira/browse/KAFKA-1499?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14158091#comment-14158091
]
Joel Koshy commented on KAFKA-1499:
-----------------------------------
[~jkreps] Regarding the two approaches, both are susceptible to people
forgetting (or misunderstanding) the configs. Providing a
broker.compression.enabled property which defaults to false helps avoid an
already deployed scenario from being affected. Producers continue to set
whatever compression.type they already use and that is unaffected at the
broker. The issues with this as you point out are an additional config to deal
with and forgetting to turn it on which would be confusing when people use
per-topic overrides.
On the other hand, the other approach of not having the config and assuming
that broker compression (or decompression) is always enabled is better when
people use per-topic overrides but is slightly dangerous/inconvenient if an
existing deployment needs compression enabled and upgrades and does not set it
to a suitable compression type.
How about the following: right now all of our server configs are always present
- either explicitly specified or default. In this instance it is better to make
broker.compression.type an optional config. i.e., if it is is explicitly
specified use it otherwise, assume that broker compression/decompression is
disabled. So when appending messages to the log: if
(broker-compression-type-config is specified) use that; else if
(topic-compression-type-override is specified) use that; else use whatever
compression type the producer sent the message with.
> Broker-side compression configuration
> -------------------------------------
>
> Key: KAFKA-1499
> URL: https://issues.apache.org/jira/browse/KAFKA-1499
> Project: Kafka
> Issue Type: New Feature
> Reporter: Joel Koshy
> Assignee: Manikumar Reddy
> Labels: newbie++
> Fix For: 0.8.2
>
> Attachments: KAFKA-1499.patch, KAFKA-1499.patch,
> KAFKA-1499_2014-08-15_14:20:27.patch, KAFKA-1499_2014-08-21_21:44:27.patch,
> KAFKA-1499_2014-09-21_15:57:23.patch, KAFKA-1499_2014-09-23_14:45:38.patch,
> KAFKA-1499_2014-09-24_14:20:33.patch, KAFKA-1499_2014-09-24_14:24:54.patch,
> KAFKA-1499_2014-09-25_11:05:57.patch
>
> Original Estimate: 72h
> Remaining Estimate: 72h
>
> A given topic can have messages in mixed compression codecs. i.e., it can
> also have a mix of uncompressed/compressed messages.
> It will be useful to support a broker-side configuration to recompress
> messages to a specific compression codec. i.e., all messages (for all
> topics) on the broker will be compressed to this codec. We could have
> per-topic overrides as well.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
