zhangyue19921010 commented on pull request #10551:
URL: https://github.com/apache/druid/pull/10551#issuecomment-737150275
Hi @pphust
> ```
> final Map<String, Object> consumerConfigs =
KafkaConsumerConfigs.getConsumerProperties();
> final Properties props = new Properties();
> addConsumerPropertiesFromConfig(props, sortingMapper,
consumerProperties);
> props.putAll(consumerConfigs);
> ```
>
> According to above code, consumerProperties(from KafkaSupervisorIOConfig)
will be overwritten by consumerConfigs which is set in KafkaConsumerConfigs
class.
> It looks a bit strange for me that user custom setting is NOT prior to
static setting. Is it a good idea if we just change the order like below? Druid
user will get more flexibility on kafka connect setting and I think the new
isolation.level=read_uncommitted will work well also.
>
> ```
> final Map<String, Object> consumerConfigs =
KafkaConsumerConfigs.getConsumerProperties();
> final Properties props = new Properties();
> **props.putAll(consumerConfigs);**
> addConsumerPropertiesFromConfig(props, sortingMapper,
consumerProperties);
> ```
I also considered the feasibility of this solution before. What I am
concerned is that some configs such as ` props.put("auto.offset.reset",
"none") ` and `props.put("enable.auto.commit", "false")` are important for
Druid to provide exactly once ingestion guarantees. If these properties are
made user-configurable, it is possible that users may unintentionally break the
`exactly-once`. So it may be more appropriate to keep the existing priorities
for important parameters and flex other configs like `isolation.level` through
`getOrDefault()`
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
