[
https://issues.apache.org/jira/browse/KAFKA-6387?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16316820#comment-16316820
]
Randall Hauch commented on KAFKA-6387:
--------------------------------------
There are a number of cases where this behavior might actually change how the
producers and consumers used for connectors are configured. Consider a worker
configuration includes the following configuration properties:
{code}
retries=2
max.partition.fetch.bytes=262144
{code}
This configuration file does not define
"{{consumer.max.partition.fetch.bytes}}" or
"{{producer.max.partition.fetch.bytes}}", so currently these would default to
1048576. However, after this proposed change if implemented the
"{{consumer.max.partition.fetch.bytes}}" or
"{{producer.max.partition.fetch.bytes}}" values would be the inherited value of
262144, not the default 1048576.
In short, implementing this change would break backward compatibility. We could
implement a configuration switch that controls whether the configurations are
inherited, but this adds complexity to the already-complex configuration
mechanism.
> Worker's producer and consumer configs should inherit from worker configs
> -------------------------------------------------------------------------
>
> Key: KAFKA-6387
> URL: https://issues.apache.org/jira/browse/KAFKA-6387
> Project: Kafka
> Issue Type: Improvement
> Components: KafkaConnect
> Affects Versions: 1.0.0
> Reporter: Randall Hauch
> Labels: needs-kip
>
> Currently, the worker configuration file defines the connection properties
> for the three separate types of connections being made to the Kafka cluster:
> # the worker group membership,
> # producers for source connectors,
> # the consumers for sink connectors.
> The configs are namespaced because to properly support things like
> interceptors where the configs for 2 and 3 would conflict (same config name,
> different value).
> However, it would be beneficial when such control is not required for the
> producers and consumers to inherit the top-level configurations yet be able
> to override them with the {{producer.}} and {{consumer.}} namespaced
> configurations. This way the producer- and consumer-specific configurations
> need only be specified if/when they need to override the top-level
> configurations. This may be necessary, for example, to have different ACLs
> than the connector tasks compared to the producers and consumers.
> This will require a minimal KIP to explain the new behavior.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
