[GitHub] [kafka] chia7712 commented on a change in pull request #8663: KAFKA-9985: Sink connector may exhaust broker when writing in DLQ

2020-05-13 Thread GitBox 


chia7712 commented on a change in pull request #8663:
URL: https://github.com/apache/kafka/pull/8663#discussion_r424860254



##
File path: 
connect/runtime/src/main/java/org/apache/kafka/connect/runtime/SinkConnectorConfig.java
##
@@ -96,6 +101,25 @@ public static void validate(Map props) {
 throw new ConfigException("Must configure one of " +
 SinkTask.TOPICS_CONFIG + " or " + 
SinkTask.TOPICS_REGEX_CONFIG);
 }
+
+if (hasDlqTopicConfig) {
+String dlqTopic = props.get(DLQ_TOPIC_NAME_CONFIG).trim();
+if (hasTopicsConfig) {
+List topics = parseTopicsList(props);
+if (topics.contains(dlqTopic)) {
+throw new ConfigException(DLQ_TOPIC_NAME_CONFIG + " has a 
topic name which is already in " +

Review comment:
   Should we log the topic name for this exception? For example, ```has a 
topic name (xxx) which```





This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [kafka] chia7712 commented on a change in pull request #8663: KAFKA-9985: Sink connector may exhaust broker when writing in DLQ

2020-05-13 Thread GitBox 


chia7712 commented on a change in pull request #8663:
URL: https://github.com/apache/kafka/pull/8663#discussion_r424859533



##
File path: 
connect/runtime/src/main/java/org/apache/kafka/connect/runtime/SinkConnectorConfig.java
##
@@ -108,6 +132,20 @@ public static boolean hasTopicsRegexConfig(Map props) {
 return topicsRegexStr != null && !topicsRegexStr.trim().isEmpty();
 }
 
+public static boolean hasDlqTopicConfig(Map props) {
+String dqlTopicStr = props.get(DLQ_TOPIC_NAME_CONFIG);
+return dqlTopicStr != null && !dqlTopicStr.trim().isEmpty();
+}
+
+public static List parseTopicsList(Map props) {
+List topics = (List) 
ConfigDef.parseType(TOPICS_CONFIG, props.get(TOPICS_CONFIG), Type.LIST);
+return topics
+.stream()
+.filter(topic -> !topic.isEmpty())
+.distinct()

Review comment:
   How about returning a Set instead of List? 
   
   ```
   return topics
   .stream()
   .filter(topic -> !topic.isEmpty())
   .collect(Collectors.toSet());
   ```





This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [kafka] chia7712 commented on a change in pull request #8663: KAFKA-9985: Sink connector may exhaust broker when writing in DLQ

2020-05-13 Thread GitBox 


chia7712 commented on a change in pull request #8663:
URL: https://github.com/apache/kafka/pull/8663#discussion_r424859533



##
File path: 
connect/runtime/src/main/java/org/apache/kafka/connect/runtime/SinkConnectorConfig.java
##
@@ -108,6 +132,20 @@ public static boolean hasTopicsRegexConfig(Map props) {
 return topicsRegexStr != null && !topicsRegexStr.trim().isEmpty();
 }
 
+public static boolean hasDlqTopicConfig(Map props) {
+String dqlTopicStr = props.get(DLQ_TOPIC_NAME_CONFIG);
+return dqlTopicStr != null && !dqlTopicStr.trim().isEmpty();
+}
+
+public static List parseTopicsList(Map props) {
+List topics = (List) 
ConfigDef.parseType(TOPICS_CONFIG, props.get(TOPICS_CONFIG), Type.LIST);
+return topics
+.stream()
+.filter(topic -> !topic.isEmpty())
+.distinct()

Review comment:
   why not just return a Set instead of List? 
   
   ```
   return topics
   .stream()
   .filter(topic -> !topic.isEmpty())
   .collect(Collectors.toSet());
   ```





This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org