[
https://issues.apache.org/jira/browse/CAMEL-25614?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=18127077#comment-18127077
]
Claus Ibsen commented on CAMEL-25614:
-------------------------------------
Merged to main in https://github.com/apache/camel/pull/27801. The review
follow-ups (a shared watch loop and an informer) are tracked in CAMEL-25653.
_Claude Code on behalf of davsclaus_
> camel-kubernetes - with camel.vault.kubernetes.refreshEnabled=true the
> CamelContext is no longer reloaded on a secret (config map) update after the
> watch was closed once (410 Gone); nothing is logged
> -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
>
> Key: CAMEL-25614
> URL: https://issues.apache.org/jira/browse/CAMEL-25614
> Project: Camel
> Issue Type: Bug
> Components: camel-kubernetes
> Reporter: shashank
> Assignee: shashank
> Priority: Major
> Fix For: 4.24.0
>
>
> With {{camel.vault.kubernetes.refreshEnabled=true}}
> ({{camel.vault.kubernetescm.refreshEnabled=true}} for config maps) camel-main
> schedules {{SecretsReloadTriggerTask}} ({{ConfigmapsReloadTriggerTask}}) with
> {{PeriodTaskScheduler.scheduledTask}} ({{DefaultConfigurationConfigurer}} 867
> / 886 at main a7169d7c6ab6). That method submits the task once:
> {{DefaultPeriodTaskScheduler.TaskWrapper.onTimer}} (93-98)
> {code:java}
> // submit task only once as the task can potentially keep running (until
> camel is stopped)
> if (running == null) {
> running = executorService.submit(task);
> }
> {code}
> {{SecretsReloadTriggerTask.run}} (117-156;
> {{ConfigmapsReloadTriggerTask.run}} 120-159 is the same) creates one watch,
> waits for {{onClose(WatcherException)}} and returns:
> {code:java}
> @Override
> public void onClose(WatcherException e) {
> isWatchClosed.countDown();
> }
> ...
> // Wait till watch gets closed
> isWatchClosed.await();
> ...
> watch.close();
> {code}
> The Kubernetes client reconnects a watch by itself after transient errors,
> but it closes the watch for good and calls {{onClose(WatcherException)}} when
> the API server answers 410 Gone (the resource version of the watch is too
> old). How often that happens depends on the cluster: the watch asks for
> bookmarks, which keep its resource version fresh, so it is an occasional
> event and not a regular one. It is the same event that CAMEL-25273 fixed for
> the consumers of the component; the two reload tasks were not part of it, and
> for them one such close is enough.
> After the first such close the task is finished and is never submitted again.
> The application keeps running with the secret values it has; a rotated secret
> is not followed by a reload, and nothing is logged, neither when the watch is
> closed nor later. A watch that cannot be created ({{watch(...)}} throws: API
> server not reachable at that moment, request refused) ends the task the same
> way; the exception stays in the {{Future}} of the submit, which nobody reads,
> so there is no log line either.
> h3. Reproduction
> New {{SecretsReloadTriggerTaskWatchClosedTest}} and
> {{ConfigmapsReloadTriggerTaskWatchClosedTest}} with the Kubernetes mock
> server that the module uses for its other tests. The task is scheduled the
> way camel-main does it. The first watch request is answered with a watch
> event {{ERROR}} / 410; the next one with a {{MODIFIED}} event for the
> configured secret (config map). In a second test one more watch request in
> between is answered with 403. A {{ContextReloadStrategy}} counts the reloads.
> On main these four tests fail after 20 seconds with {{the update of the
> secret after the watch was closed did not trigger a reload ==> expected:
> <true> but was: <false>}}. A third test per class stops the context while the
> task has a watch and expects the thread of the task to end; on main: {{the
> stopped task still waits for its watch to be closed ==> expected: <false> but
> was: <true>}} (on main a stopped task keeps its thread waiting for the watch).
> h3. Proposed fix
> {{run()}} of both tasks watches again until the task is stopped: one second
> after a closed watch, and with a delay that doubles up to 30 seconds while
> the watch cannot be created (one second at start, two seconds for the first
> failed attempt after a closed watch; the delays of
> {{KubernetesHelper.watchAgain}}, which CAMEL-25273 added for the consumers).
> Both cases are logged with one WARN line, without a stack trace (the stack
> trace of a watch that cannot be created is logged at DEBUG). {{doStop}} of
> the task ends the wait, so a stopped task does not keep its thread.
> Updates made while nothing watches (between the close and the next watch) are
> still not seen; the task does not list the secrets after it watched again.
> camel-kubernetes tests with the fix: 179 run, 0 failures, 6 skipped.
> How it was found: a sweep of the reconnect / re-watch code of the components
> after CAMEL-25273, and a small TLA+ model of the task (submitted once, watch
> created or refused, 410, updates). For main TLC reports that the task ends
> after one 410 or one refused watch request (invariant "the task never ends
> while it is not stopped", traces of 4 and 3 states) and that "in the end the
> secrets are watched" does not hold; with the fix both hold for two 410
> answers and two refused requests (144 states).
> Affected: main (a7169d7c6ab6; tested). camel-4.23.x, camel-4.22.x,
> camel-4.18.x and camel-4.14.x have the same {{run()}} in both tasks and the
> same {{TaskWrapper.onTimer}} (read, not run).
> Not looked at: the other vault refresh tasks that are scheduled with
> {{scheduledTask}}.
> Duplicate check (2026-10-13): JIRA text "SecretsReloadTriggerTask"
> (CAMEL-24636, CAMEL-21924, CAMEL-21771, CAMEL-21770, CAMEL-21436: none about
> the watch), "kubernetes secret refresh watch", "kubernetes vault refresh
> watch closed" (CAMEL-24636 only); GitHub pull requests
> "SecretsReloadTriggerTask" (none), "kubernetes secret refresh watch" (nothing
> related).
> _Filed with Claude Code on behalf of allthingssecurity._
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
