tombentley commented on code in PR #11781:
URL: https://github.com/apache/kafka/pull/11781#discussion_r896737764


##########
connect/runtime/src/main/java/org/apache/kafka/connect/runtime/Worker.java:
##########
@@ -1327,30 +1334,39 @@ public WorkerTask doBuild(Task task,
                     connectorClientConfigOverridePolicy, kafkaClusterId);
             KafkaProducer<byte[], byte[]> producer = new 
KafkaProducer<>(producerProps);
 
-            TopicAdmin topicAdmin;
+            // Prepare to create a topic admin if the task requires one, but 
do not actually create an instance
+            // until/unless one is needed
+            final AtomicReference<TopicAdmin> topicAdmin = new 
AtomicReference<>();
+            final Supplier<TopicAdmin> topicAdminCreator = () -> 
topicAdmin.updateAndGet(existingAdmin -> {
+                if (existingAdmin != null) {
+                    return existingAdmin;
+                }
+                Map<String, Object> adminOverrides = 
adminConfigs(id.connector(), "connector-adminclient-" + id, config,
+                        sourceConfig, connectorClass, 
connectorClientConfigOverridePolicy, kafkaClusterId, ConnectorType.SOURCE);
+                Admin adminClient = Admin.create(adminOverrides);
+                return new 
TopicAdmin(adminOverrides.get(BOOTSTRAP_SERVERS_CONFIG), adminClient);
+            });
+
             Map<String, TopicCreationGroup> topicCreationGroups;
             if (config.topicCreationEnable() && 
sourceConfig.usesTopicCreation()) {
                 topicCreationGroups = 
TopicCreationGroup.configuredGroups(sourceConfig);
                 // Create a topic admin that the task can use for topic 
creation
-                Map<String, Object> adminOverrides = 
adminConfigs(id.connector(), "connector-adminclient-" + id, config,
-                        sourceConfig, connectorClass, 
connectorClientConfigOverridePolicy, kafkaClusterId, ConnectorType.SOURCE);
-                topicAdmin = new TopicAdmin(adminOverrides);
+                topicAdminCreator.get();

Review Comment:
   Looking at the tests it doesn't seem we have any that assert that the 
`Admin` actually gets closed. `WorkerSourceTaskTest` asserts this, but for this 
code we're reliant that every invocation of `doBuild` always returns a 
`WorkerSourceTask` which is always stopped. I can see this is pretty hard to 
test though.



##########
connect/runtime/src/main/java/org/apache/kafka/connect/runtime/Worker.java:
##########
@@ -1327,30 +1334,39 @@ public WorkerTask doBuild(Task task,
                     connectorClientConfigOverridePolicy, kafkaClusterId);
             KafkaProducer<byte[], byte[]> producer = new 
KafkaProducer<>(producerProps);
 
-            TopicAdmin topicAdmin;
+            // Prepare to create a topic admin if the task requires one, but 
do not actually create an instance
+            // until/unless one is needed
+            final AtomicReference<TopicAdmin> topicAdmin = new 
AtomicReference<>();
+            final Supplier<TopicAdmin> topicAdminCreator = () -> 
topicAdmin.updateAndGet(existingAdmin -> {
+                if (existingAdmin != null) {
+                    return existingAdmin;
+                }
+                Map<String, Object> adminOverrides = 
adminConfigs(id.connector(), "connector-adminclient-" + id, config,
+                        sourceConfig, connectorClass, 
connectorClientConfigOverridePolicy, kafkaClusterId, ConnectorType.SOURCE);
+                Admin adminClient = Admin.create(adminOverrides);
+                return new 
TopicAdmin(adminOverrides.get(BOOTSTRAP_SERVERS_CONFIG), adminClient);
+            });
+
             Map<String, TopicCreationGroup> topicCreationGroups;
             if (config.topicCreationEnable() && 
sourceConfig.usesTopicCreation()) {
                 topicCreationGroups = 
TopicCreationGroup.configuredGroups(sourceConfig);
                 // Create a topic admin that the task can use for topic 
creation
-                Map<String, Object> adminOverrides = 
adminConfigs(id.connector(), "connector-adminclient-" + id, config,
-                        sourceConfig, connectorClass, 
connectorClientConfigOverridePolicy, kafkaClusterId, ConnectorType.SOURCE);
-                topicAdmin = new TopicAdmin(adminOverrides);
+                topicAdminCreator.get();

Review Comment:
   The conditions under which we actually end up calling 
`topicAdminCreator.get()` are:
   
   ```java
   sourceConfig.offsetsTopic() != null && 
config.connectorOffsetsTopicsPermitted() // connector-specific store
   || config.topicCreationEnable() && sourceConfig.usesTopicCreation() // task 
needs for topic creation
   ```
   
   These are the only circumstances in which `topicAdmin.get()` would return 
non-null.
   
   So I wonder if we should just put that logic directly in `doBuild()`, and 
have `offsetStoreForRegularSourceTask()` accept a nullable `topicAdminSupplier` 
(which will be guaranteed non-null in the case where it's actually called).
   
   Wdyt?



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: jira-unsubscr...@kafka.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

Reply via email to