Ah, it definitely seems like KIP-710 will address the issue we've been bitten
by most.We'll eagerly await the kafka-3.5.0 release and then see if enabling
'dedicated.mode.enable.internal.rest' is possible with Strimzi.
Thanks for the help and patience! :-)
Frank,
> I don't think forcing the API users to introduce the nonce is desirable.
I agree. That is why the nonce is a workaround, and not a proper solution.
It's something to alleviate the symptoms in the short-term until a bugfix &
upgrade can fix it.
> Have you had any ideas on how this can be
So we've just hit this issue again just with the MM2 connector and trying to
add a new mirrored topic.We're running MirrorMaker 2 in Strimzi. i.e.
"connector.class": "org.apache.kafka.connect.mirror.MirrorSourceConnector"We
have 6 worker nodes.We changed the config to add a new mirror topic. i.e
I don't think forcing the API users to introduce the nonce is desirable.For us,
it would mean reaching out to the Strimzi project to try to get that
implemented on their side, which I would imagine would be a proposal which
would meet some resistance.
Have you had any ideas on how this can be im
Frank,
The configs are being compared after ConfigProviders have been resolved.
This is happening both as a Connector config (by
ClusterConfigState::connectorConfig) and as task configs (by
ClusterConfigState::taskConfig).
This means that two configurations that have different direct contents (the
I'm still having trouble understanding how the configs could match in the code
you highlighted when we change connector and/or task config values when no keys
are being pruned by the connector implementations in question.Would capturing a
new generation value within the config itself on every s
Frank,
> I'm operating on the assumption that the connectors in question get stuck
in an inconsistent state
> Another thought... if an API exists to list all connectors in such a
state, then at least some monitoring/alerting could be put in place, right?
There is two different inconsistencies rel
Another thought... if an API exists to list all connectors in such a state,
then at least some monitoring/alerting could be put in place, right?
So I've been looking into the codebase to familiarize myself with it.I'm
operating on the assumption that the connectors in question get stuck in an
inconsistent state which causes them to prune the new task configs from those
which are "broadcast" to the workers.I see on
KafkaConfigBackingSto
Frank,
I don't think that the fix needs to necessarily follow the #12450 PR, we
can choose to start from scratch now that we know more about the issue.
If that PR is useful as a starting point, we can also include it, that is
up to you.
Greg
On Mon, Feb 6, 2023 at 10:21 AM Frank Grimes
wrote:
Hi Greg,
I actually just found the following comment on this PR for
https://issues.apache.org/jira/browse/KAFKA-13809:
https://github.com/apache/kafka/pull/12450
> we get the same behavior (KAFKA-9228 notwithstanding) by passing the original
>properties through to tasks transparently
It seems
Frank,
I think you're right that the KAFKA-9228 ticket doesn't capture every
possible reconfiguration that might result in a dropped restart.
The ticket calls out the FileStream connectors, which generate their
configurations by dropping unknown config values, which is relatively
uncommon.
This me
Hi Greg,
The "long-term inconsistency" we have observed is not with no tasks at all, but
instead with all the previously running tasks remaining in a running state but
with a previous config.
If I'm understanding the original bug report correctly, the scope of the
problem was thought to only af
Frank,
I realized I didn't respond to the title directly, sorry about that.
The reason that `ClusterConfigState::inconsistentConnectors` is not used,
is that the effect of an inconsistent connector is applied via
`ClusterConfigState::tasks`.
If a connector is inconsistent, then the tasks method wi
Frank,
The inconsistentConnectors method is related to an extremely specific
inconsistency that can happen when a worker writes some task
configurations, and then disconnects without writing a following "commit
tasks record" to the config topic.
This is a hold-over from the early days of connect f
Hi, we're investigating an issue where occasionally config changes don't
propagate to connectors/tasks.
When this occurs, the only way to ensure that the configuration takes effect is
to resize the number of tasks back down to 1 and then resize back up to the
original number of tasks.
In search
16 matches
Mail list logo
