[
https://issues.apache.org/jira/browse/KAFKA-9228?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17883296#comment-17883296
]
Doug Whitfield commented on KAFKA-9228:
---------------------------------------
We performed the test with Mongodb connector 1.11.1 and 1.13.0
Supposedly, this should work since "compatible with Apache Kafka v2.3 and
later:
[https://www.mongodb.com/docs/kafka-connector/v1.11/compatibility/#kafka-compatibility]
although the release dates don't really line up in a way that seems to make
that possible. I have asked about this over at Mongo:
https://www.mongodb.com/community/forums/t/kafka-connect-version-compatibility/297511
Log snippet below shows that when the vault ttl not expired, the connector
1.11.1 works well with Kafka 3.7.1.
on the line 7851 (I didn't upload the full log, but we can keep the reference
here for any of my team members following the case) , is the first record
insert : \{userid=AMIR_SINK_AUTH_13_3_7_1_002} with the
username=v-DATA-nope-auth--test-cms-5min-RLdkjtfI2GPesm7hk1UI-1726737189 at
2024-09-19 09:15:20,590
[2024-09-19 09:15:20,590] TRACE
WorkerSinkTask\{id=ec002i000948-kf002i000165-AP24944-mongosink} Applying
transformations to record in topic
'ec002i000948-kf002i000165-AP24944-mongosink' partition 0 at offset 240 and
timestamp 1726737319573 with key null and value
\{userid=AMIR_SINK_AUTH_13_3_7_1_002}
(org.apache.kafka.connect.runtime.WorkerSinkTask:553)
the new username was renewed by the config provider on the line 20427 at
2024-09-19 09:24:38,909
[2024-09-19 09:24:38,909] INFO the value of the key = 'password' is 'lol'
(com.lonecppcoder.kafka.config.TTLFileConfigProvider:102)
[2024-09-19 09:24:38,909] INFO the value of the key = 'username' is
'v-DATA-nope-auth--test-cms-5min-cYsJMmuafidQTn2CS1YQ-1726737431'
(com.lonecppcoder.kafka.config.TTLFileConfigProvider:102)
After this moment on the line 22062, when the connector try to insert a new
record, the connector returned an exception auth 13 at 2024-09-19 09:25:34,295:
[2024-09-19 09:25:34,295] ERROR Failed to put into the sink the following
records: [SinkRecord\{kafkaOffset=241, timestampType=CreateTime,
originalTopic=ec002i000948-kf002i000165-AP24944-mongosink,
originalKafkaPartition=0, originalKafkaOffset=241}
ConnectRecord\{topic='ec002i000948-kf002i000165-AP24944-mongosink',
kafkaPartition=0, key=null, keySchema=null,
value={userid=AMIR_SINK_AUTH_13_3_7_1_003}, valueSchema=null,
timestamp=1726737933134, headers=ConnectHeaders(headers=)}]
(com.mongodb.kafka.connect.sink.MongoSinkTask:244)
com.mongodb.MongoCommandException: Command failed with error 13 (Unauthorized):
'command update requires authentication' on server
b98370ec-1806-4566-83f5-c38dfda12b7a-0.cdh5ng0b0v5co5c3f9qg.private.databases.appdomain.cloud:30603.
The full response is \{"ok": 0.0, "errmsg": "command update requires
authentication", "code": 13, "codeName": "Unauthorized", "$clusterTime":
{"clusterTime": {"$timestamp": {"t": 1726737929, "i": 1}}, "signature":
\{"hash": {"$binary": {"base64": "Take On Me", "subType": "00"}}, "keyId":
numbers}}, "operationTime": \{"$timestamp": {"t": 1726737320, "i": 1}}}
So via this log, we see clearly the error 13 (Unauthorized) appear only after
ttl expire. Is this a different bug? Is this fix incompletely fixed. Is ttl not
a concern of Kafka but a concern of Mongo?
> Reconfigured converters and clients may not be propagated to connector tasks
> ----------------------------------------------------------------------------
>
> Key: KAFKA-9228
> URL: https://issues.apache.org/jira/browse/KAFKA-9228
> Project: Kafka
> Issue Type: Bug
> Components: connect
> Affects Versions: 2.3.0, 2.4.0, 2.3.1, 2.3.2
> Reporter: Chris Egerton
> Assignee: Chris Egerton
> Priority: Major
> Fix For: 3.8.0, 3.7.1
>
>
> If an existing connector is reconfigured but the only changes are to its
> converters and/or Kafka clients (enabled as of
> [KIP-458|https://cwiki.apache.org/confluence/display/KAFKA/KIP-458%3A+Connector+Client+Config+Override+Policy]),
> the changes will not propagate to its tasks unless the connector also
> generates task configs that differ from the existing task configs. Even after
> this point, if the connector tasks are reconfigured, they will still not pick
> up on the new converter and/or Kafka client configs.
> This is because the {{DistributedHerder}} only writes new task configurations
> to the connect config topic [if the connector-provided task configs differ
> from the task configs already in the config
> topic|https://github.com/apache/kafka/blob/e499c960e4f9cfc462f1a05a110d79ffa1c5b322/connect/runtime/src/main/java/org/apache/kafka/connect/runtime/distributed/DistributedHerder.java#L1285-L1332],
> and neither of those contain converter or Kafka client configs.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
