C0urante commented on code in PR #13434:
URL: https://github.com/apache/kafka/pull/13434#discussion_r1155345175
##########
connect/runtime/src/main/java/org/apache/kafka/connect/runtime/AbstractHerder.java:
##########
@@ -866,4 +867,18 @@ public List<ConfigKeyInfo> connectorPluginConfig(String
pluginName) {
}
}
+ @Override
+ public void connectorOffsets(String connName, Callback<ConnectorOffsets>
cb) {
Review Comment:
> I haven't added an additional call to `ConfigBackingStore::refresh` since
we're capturing a fresh snapshot (in the `AbstractHerder`) and not using an
existing stale one so we should be reasonably up to speed on the state of the
topic (due to the work thread in the underlying Kafka based log)
I won't push for this but it is worth pointing out that the worker may have
lost contact with the config topic at some point and so we can't assume that
taking a snapshot gives us an up-to-date view of it. I do agree that no matter
what, the risks here are small and the guarantees we're providing already
should be fine for now.
> Btw on a side note, we don't seem to be making calls to
`ConfigBackingStore::refresh` even in methods like `putConnectorConfig` where
it does seem like it would be useful to do so, what do you think?
The `putConnectorConfig` example is an interesting one. The leader will
force a read-to-end of the config topic after writing connector configs or
tombstones (that delete connector configs) to it, so it's actually guaranteed
that the leader will be up-to-date on any newly-created or just-destroyed
connectors. I haven't done a comprehensive analysis of all of the herder
methods but I suspect this kind of logic holds for most of them. The only
exceptions I can think of off the top of my head are pausing/resuming a
connector, but that's low-stakes enough that a read-to-end probably isn't worth
it.
I'm going to resolve this conversation since I'm satisfied with the way
things look now. I'm happy to look at a follow-up PR or ticket if there's room
for improvement with the herder that you'd like to tackle.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: jira-unsubscr...@kafka.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
