[
https://issues.apache.org/jira/browse/CAMEL-24636?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=18112329#comment-18112329
]
Andrea Cosentino commented on CAMEL-24636:
------------------------------------------
Scope refinement after implementing the core, recorded here so the sub-tasks
are read in the right light.
The investigation turned up a second, complementary gap that this issue now
also covers. {{MainPropertiesReload}} already knows how to re-apply
{{camel.component.*}} options onto components, but it is registered as a
{{PropertiesReload}} service and is only invoked from the file-watch reload
strategies. {{DefaultContextReloadStrategy}} never looked it up, so on the
vault-triggered path a component option configured as a placeholder kept its
bootstrap-resolved value. Since the raw placeholder is not retained on the
component, a component could not have re-resolved it on its own even if it
wanted to. So the fix has two halves that depend on each other:
# re-apply the {{camel.}} options whose value is a placeholder, so the
component fields hold the newly resolved secret;
# notify {{SecretRotationAware}} beans, so anything holding a live
authenticated resource built from those fields rebuilds it.
Half 1 turns out to cover more than expected. {{computeProperties}} with
{{reload=true}} removes and re-creates the component, and
{{RouteController.reloadAllRoutes()}} already clears the endpoint registry, so
endpoints, producers and consumers are rebuilt from scratch.
That changes which component makes a useful reference implementation.
camel-kafka was originally named for that role, but it caches no live client:
the producers and consumers are rebuilt by the route reload, and its
component-level {{saslJaasConfig}} and {{sslKeystorePassword}} are refreshed by
half 1. A {{KafkaComponent.onSecretRotation()}} would have no work left to do,
so it has been dropped rather than added as dead code. camel-kafka needs no
sub-task.
The residual gap that only half 2 can close is the registry-owned pooled
resource that Camel references but does not create: a Hikari {{DataSource}}, a
pooled JMS {{ConnectionFactory}}, a shared {{HttpClientConnectionManager}}.
That work is vendor-specific and stays in the sub-tasks.
This issue therefore ships the SPI, both halves of the core wiring, tests and
documentation. The reference implementations are CAMEL-24637 (jms) and
CAMEL-24638 (jdbc/sql), which are where the SPI actually earns its place.
The SPI signature was also simplified from the
{{onSecretRotation(SecretRotationEvent)}} sketched in the description to
{{onSecretRotation(Object source)}}, mirroring the existing
{{ReloadStrategy.onReload(Object source)}}. The trigger tasks do not pass the
changed secret names through {{ContextReloadStrategy}}, so an event type would
have carried nothing the source object does not already carry, at the cost of a
second new public API type.
_Claude Code on behalf of oscerd_
> Add SecretRotationAware SPI so components can re-authenticate on context
> reload
> -------------------------------------------------------------------------------
>
> Key: CAMEL-24636
> URL: https://issues.apache.org/jira/browse/CAMEL-24636
> Project: Camel
> Issue Type: Improvement
> Components: camel-core, camel-core-api
> Reporter: Andrea Cosentino
> Assignee: Andrea Cosentino
> Priority: Major
>
> h2. Problem
> The vault components already detect that a secret changed and ask Camel to
> reload:
> * camel-aws-secrets-manager -- {{CloudTrailReloadTriggerTask}}
> * camel-azure-key-vault -- {{EventhubsReloadTriggerTask}}
> * camel-google-secret-manager -- {{PubsubReloadTriggerTask}}
> * camel-hashicorp-vault -- {{HashicorpVaultReloadTriggerTask}}
> * camel-ibm-secrets-manager -- {{IBMEventStreamReloadTriggerTask}}
> * camel-kubernetes -- {{SecretsReloadTriggerTask}} and
> {{ConfigmapsReloadTriggerTask}}
> * camel-spring-cloud-config -- {{SpringCloudConfigReloadTriggerTask}}
> They all end in the same call:
> {code:java}
> ContextReloadStrategy reload =
> camelContext.hasService(ContextReloadStrategy.class);
> if (reload != null) {
> reload.onReload(this);
> }
> {code}
> And {{DefaultContextReloadStrategy.onReload()}} (core/camel-support) does
> exactly two things:
> # {{reloadProperties()}} -- stop and start every {{PropertiesSource}}, so the
> _next_ placeholder resolution returns the new secret.
> # {{reloadRoutes()}} -- {{RouteController.reloadAllRoutes()}}, which removes
> all routes, clears the {{EndpointRegistry}}, and starts the route definitions
> again.
> Anything that is neither a route nor an endpoint is left untouched. That is
> documented behaviour in
> {{docs/user-manual/modules/ROOT/pages/context-reload.adoc}}:
> {quote}
> General services in CamelContext and java beans or Camel Processor is not
> updated.
> {quote}
> The practical consequence: the components that actually hold the
> authenticated connection keep using the *old* credentials after a rotation,
> and nothing tells them otherwise. The reload looks successful, the routes
> come back up, and the connections stay stale until the process is restarted.
> h2. Where it breaks
> *Component-level configuration is resolved once, at bootstrap.*
> {{KafkaComponent}} holds a single {{KafkaConfiguration}} and copies it into
> every endpoint ({{KafkaComponent#createEndpoint}} calls
> {{endpoint.setConfiguration(copy)}}). A property such as:
> {code:none}
> camel.component.kafka.saslJaasConfig = {{aws:broker-credentials}}
> {code}
> is resolved into that configuration object at configure time. Reloading the
> routes copies the same already-resolved, now-stale string into the new
> endpoint. The same applies to {{sslKeyPassword}} and {{sslKeystorePassword}},
> and to {{JmsComponent#setUsername}} / {{setPassword}}, which delegate to a
> shared {{JmsConfiguration}}.
> *Pooled connection holders live in the registry, not in the route.*
> {{SqlComponent}} and {{JdbcComponent}} resolve a {{DataSource}} bean;
> {{JmsComponent}} a {{ConnectionFactory}}; {{HttpComponent}} an optionally
> shared {{HttpClientConnectionManager}}. These are created once by Spring
> Boot, Quarkus or Camel Main with the credentials valid at startup. Clearing
> the endpoint registry does not recreate them, so a Hikari pool or a JMS
> connection pool goes on presenting a revoked password.
> *Nothing is listening.*
> A search across the whole tree finds exactly one consumer of
> {{CamelContextReloadingEvent}} / {{RouteReloadedEvent}} outside of core:
> camel-groovy, which flushes its compiled-script cache. No component
> re-authenticates. There is currently no contract a component could implement
> even if it wanted to.
> *Camel Main's re-configuration path is not wired to this trigger.*
> {{MainPropertiesReload}} does call
> {{autoConfigurationFromReloadedProperties()}}, which re-applies
> {{camel.component.*}} properties onto the component beans. But it is
> registered as a {{PropertiesReload}} service and is only invoked from the
> file-watch strategies ({{RouteWatcherReloadStrategy}},
> {{RouteOnDemandReloadStrategy}}, {{LoadOnDemandReloadStrategy}}).
> {{DefaultContextReloadStrategy}} never looks up {{PropertiesReload}}, so the
> vault-triggered path skips component re-configuration even on Camel Main,
> where the machinery to do it already exists.
> h2. Proposal
> Add a {{SecretRotationAware}} SPI to camel-api that a component, or any
> registry bean, can implement to be told that the secrets it captured are
> stale and that it should re-authenticate in place:
> {code:java}
> public interface SecretRotationAware {
> /**
> * Callback invoked when secrets may have been rotated.
> */
> void onSecretRotation(SecretRotationEvent event) throws Exception;
> }
> {code}
> {{DefaultContextReloadStrategy.onReload()}} then, after reloading the
> properties sources, walks the components and the registry and notifies every
> {{SecretRotationAware}} it finds. Notification failures are isolated per
> listener, so one misbehaving component cannot break the reload for everything
> else.
> Component-side implementations re-resolve their placeholders and rebuild the
> authenticated resource in place: re-create the Kafka client with the new
> {{saslJaasConfig}}, re-authenticate the JMS {{ConnectionFactory}}, evict and
> rebuild the JDBC pool, refresh the HTTP client credentials provider.
> This issue also closes the Camel Main asymmetry described above, by having
> {{DefaultContextReloadStrategy}} invoke the {{PropertiesReload}} service so
> that {{camel.component.*}} properties are re-applied on a vault-triggered
> reload exactly as they already are on a file-watch reload.
> h2. Scope of this issue
> * New {{SecretRotationAware}} SPI and supporting types in camel-api, carrying
> {{@since 4.23}}.
> * Notification wiring in {{DefaultContextReloadStrategy}}, with per-listener
> failure isolation.
> * {{PropertiesReload}} lookup on the context-reload path, so Camel Main
> re-applies component properties.
> * camel-kafka as the reference implementation, to prove the contract end to
> end.
> * Unit tests and an update to {{context-reload.adoc}}, whose current wording
> documents the limitation this removes.
> Adoption by the remaining component families is tracked in sub-tasks
> (camel-jms, camel-jdbc / camel-sql, camel-http).
> h2. Compatibility
> Purely additive and opt-in. A component that does not implement the interface
> behaves exactly as it does today, so there is no behavioural change for
> existing users and no public API signature is modified.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
