allthingssecurity opened a new pull request, #29893:
URL: https://github.com/apache/camel/pull/29893

   # Description
   
   [CAMEL-25653](https://issues.apache.org/jira/browse/CAMEL-25653)
   
   This is the follow-up offered in the review of #27801 (CAMEL-25614).
   
   After CAMEL-25614, `SecretsReloadTriggerTask` and 
`ConfigmapsReloadTriggerTask` had the same `run()`, `waitBeforeWatchingAgain()` 
and `doStop()`, apart from the resource type, and the retry delays were a third 
copy of the constants in `KubernetesHelper` (CAMEL-25273). The tasks also 
missed an update made between the close of a watch and the next one: when the 
API server closes a watch with 410 Gone, it no longer has the events since the 
resource version of that watch, and the next watch started from the current 
state.
   
   Changes:
   - New `KubernetesInformerLoop` in `org.apache.camel.component.kubernetes`, 
used by both tasks. It runs a fabric8 informer (`runnableInformer(0)` + 
`run()`): the informer lists the resources, watches them, and when the watch is 
closed with 410 Gone lists them again and compares the resource versions, so an 
update made while nothing watched is reported as an update. Once started, the 
informer retries a failed list or watch by itself, with the client's own 
backoff.
   - If the informer cannot be started (API server not reachable, 403, ...), or 
the client stops it for good (it does after a watch error other than 410 Gone), 
the loop starts a new one after 1 second, doubling up to 30 seconds while it 
cannot be started (the `KubernetesHelper` delays). The new informer starts from 
the resources the previous one had, so an update made while no informer ran is 
reported too (it can cause an extra reload when the first list of the new 
informer comes from a lagging API server cache).
   - `stop()` stops the running informer, also one that is still listing, and 
ends the wait, so a stopped task ends its thread at once and stops its informer 
before the task closes the Kubernetes client. A list or watch request already 
in flight at that moment can still fail on the closed client, and fabric8 logs 
it (WARN `... will retry` for a started informer, ERROR `... will stop` for one 
that was still listing) but does not retry a stopped informer.
   - What reloads the context is unchanged: an update of a resource whose name 
matches the configured list. Only a new resource version counts (fabric8 does 
not report a re-listed resource that did not change), and created or deleted 
resources do not reload, as before (the task only acted on `MODIFIED`). One 
difference: a matching resource that is deleted and created again while nothing 
watched is now seen as an update after the re-list, and reloads.
   - The informer keeps only the resources whose name matches, not all the 
Secrets or ConfigMaps of the namespace (its namespace index is removed, since 
the item store keeps only those).
   - `KubernetesHelper.WATCH_AGAIN_DELAY_MILLIS` and 
`WATCH_AGAIN_MAX_DELAY_MILLIS` are now public, and the tasks use them through 
the loop (their own copies are removed).
   - The tasks keep their public API (`isReloadEnabled`, `getStartingTime`, 
`matchSecret`); the reload source passed to `ContextReloadStrategy.onReload` is 
the task (it was the anonymous `Watcher`).
   
   What users can notice:
   - An informer lists before it watches, so with RBAC the ServiceAccount now 
needs the `list` permission on `secrets` (`configmaps`) as well as `watch`. 
Without it each attempt to start fails: fabric8 logs an ERROR with the stack 
trace (`... failed for ..., will stop`), then the task one WARN line, and it 
tries again with the delays above. This is in the secrets and config maps pages 
and in the 4.24 upgrade guide.
   - The log lines changed: they come from `KubernetesInformerLoop` (`Cannot 
watch Kubernetes Secrets, trying again in ... ms`, `The informer of Kubernetes 
Secrets stopped, watching again in ... ms`). After a 410 Gone nothing is logged 
at WARN anymore (fabric8 logs the re-list at DEBUG); a failed re-list or watch 
of a started informer is logged by fabric8 at WARN (`... failed for ..., will 
retry`).
   
   There was no informer in Camel to follow (no `inform(` or 
`SharedIndexInformer` on main); the loop uses the fabric8 API of the version on 
main (kubernetes-client 8.0.0).
   
   Tests (no cluster, no Docker: the fabric8 mock server in CRUD mode, with a 
test dispatcher `WatchControlDispatcher` that can fail the next list or watch 
request, hold the first watch and close it with a 410 Gone `ERROR` event, and 
tell when a watch is open):
   - `SecretsReloadTriggerTaskWatchClosedTest` and 
`ConfigmapsReloadTriggerTaskWatchClosedTest` (5 each), with the task scheduled 
as camel-main does and a `ContextReloadStrategy` that records the reload:
     - new `testUpdateMadeWhileTheWatchIsClosedIsNotMissed`: the resource is 
updated while the first watch is held, then the watch is closed with 410 Gone: 
a reload is expected;
     - the CAMEL-25614 scenarios, moved from the expectation mock to the CRUD 
mock (an informer lists before it watches and adds `resourceVersion` and 
`timeoutSeconds` to the watch request, so the fixed watch paths no longer 
match): an update after a 410 is followed by a reload, the same with a 403 on 
the first attempt to list or watch again (now retried by fabric8 inside the 
informer), the same with a 403 on the very first attempt (new, the loop's 
retry), and a stopped task ends its thread.
   - `KubernetesInformerLoopTest` (6): only an update of a matching resource is 
reported (an update of another resource, a created and a deleted one are not, 
and the informer calls back in order, so the first reported update tells); 
after a 410 the re-listed resources are reported only when they changed; when 
fabric8 stops the informer for good (an event that is not a resource on the 
held watch), the loop starts a new one, which reports the update made in 
between; `stop()` ends the loop: after the service stopped, while the service 
still runs (a stopped loop does not restart), and while the informer is still 
listing (no watch is made).
   - Without the change (main's two tasks and `KubernetesHelper`, same tests): 
the two `testUpdateMadeWhileTheWatchIsClosedIsNotMissed` fail (`the update of 
the secret made while the watch was closed did not trigger a reload ==> 
expected: <true> but was: <false>`, and the same for the config map); the other 
eight pass.
   - With the change: camel-kubernetes unit tests 228 run, 0 failures, 6 
skipped. The restart and the two stop tests of `KubernetesInformerLoopTest` 
fail with a loop that makes a new item store for each informer and whose 
`stop()` only ends the wait (`expected: <my-b> but was: <my-a>`, `the stopped 
loop still runs`, `the stopped loop watched the resources ==> expected: <0> but 
was: <1>`).
   - The three test classes above run 15 times in one JVM: 240 tests, 0 
failures. The waits are latches, semaphores and bounded joins, no 
`Thread.sleep`.
   
   Not changed: the consumers of the component that watch with 
`KubernetesHelper.watch`/`watchAgain` (CAMEL-25273) have the same gap after a 
410 Gone. An informer is not a drop-in there, since its first list would send 
an exchange for every existing resource; I can open a follow-up JIRA for them.
   
   What was not run: no real cluster. camel-main was not built or tested (not 
changed).
   
   # Target
   
   - [x] I checked that the commit is targeting the correct branch (Camel 4 
uses the `main` branch)
   
   # Tracking
   - [x] If this is a large change, bug fix, or code improvement, I checked 
there is a [JIRA issue](https://issues.apache.org/jira/browse/CAMEL) filed for 
the change (usually before you start working on it): 
[CAMEL-25653](https://issues.apache.org/jira/browse/CAMEL-25653).
   
   # Apache Camel coding standards and style
   
   - [x] I checked that each commit in the pull request has a meaningful 
subject line and body.
   - [ ] I have run `mvn clean install -DskipTests` locally from root folder 
and I have committed all auto-generated changes.
     (I built the core modules and camel-kubernetes with the formatter and 
import-sort plugins and ran its unit tests. I did not run the full root build. 
The two component pages changed, and their copies in camel-catalog are updated 
with them.)
   
   # AI-assisted contributions
   
   - [x] If this PR includes AI-generated code, commits have proper 
co-authorship attribution (e.g., `Co-authored-by` trailers) and the PR 
description identifies the AI tool used.
     This PR was prepared with Claude Code (Claude Opus 5.5). The commit 
carries a `Co-Authored-By` trailer.
   
   _Claude Code on behalf of allthingssecurity_
   
   🤖 Generated with [Claude Code](https://claude.com/claude-code)
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]

Reply via email to