squakez commented on code in PR #6917: URL: https://github.com/apache/camel-k/pull/6917#discussion_r4385963645
########## proposals/keda-connection-beans.adoc: ########## @@ -0,0 +1,223 @@ +--- +title: Discovering connection configuration for KEDA scalers +authors: + - "@knockknock10" +creation-date: 2026-10-25 +last-updated: 2026-10-26 +status: draft +see-also: + - https://github.com/apache/camel-k/issues/6910 + - https://github.com/apache/camel-k/pull/6893 + - https://github.com/apache/camel-k/issues/6890 +replaces: [] +superseded-by: [] +--- + +[[keda-connection-beans]] += Discovering connection configuration for KEDA scalers + +== Summary + +Camel K can automatically create KEDA triggers from supported consumer endpoint URIs, but a scaler may need connection details configured outside the URI. Those details can live in Camel component properties or in a named bean (for example, a connection factory referenced by a component). The current scaler-mapping contract does not expose a resolved Camel registry or bean configuration to the mapper. + +This proposal recommends a staged approach: preserve the existing endpoint-URI mapping, make directly declared Integration/component properties available to scaler mappings, and assess a focused path for properties declared on Pipe endpoints that reference known Kamelets. Generic discovery of which bean is used as a connection factory—and how it is wired through arbitrary source code or multiple levels of indirection—is not feasible as a first iteration. If future use cases require a reusable description beyond the static configuration Camel K can see, the Camel community can assess a generic Camel Core metadata contract. + +This is a design proposal only; it does not introduce a runtime resolver or change the scaler interface. + +== Problem statement + +A KEDA scaler reads metadata to poll an external system and decide when to scale an Integration. Automatic discovery currently starts with a Camel consumer URI, but not every scaler's required metadata is present in that URI. + +== Feasibility assessment + +The configuration sources below are not equally discoverable. The first iteration should target values the operator already has in structured form, rather than attempting to reconstruct the running Camel registry from source code. + +* **Endpoint URI parameters — implementable; already supported.** The current mapper parses an endpoint URI and maps selected path and query parameters to KEDA metadata. This remains the baseline and requires no bean discovery. +* **Integration and component properties — implementable; partly covered by https://github.com/apache/camel-k/pull/6893[PR #6893].** Camel K can collect properties explicitly configured on an Integration and its Camel trait and pass a property map to each scaler mapper. This lets a component-specific mapper translate a known property such as `camel.component.amqp.broker-url`. It does not resolve a bean reference or a runtime placeholder. The related PR is open and this extension is not yet part of upstream `main`. +* **Properties configured on a Pipe endpoint that references a Kamelet — partially implementable.** The Pipe-to-Integration binding already preserves explicitly supplied Kamelet properties as Integration application properties. For known Kamelets, a mapping can translate declared parameters (for example, `broker-url`) to the corresponding scaler metadata. This requires an explicit mapping between a known Kamelet and its parameters; it does not automatically reveal the inner endpoint of every arbitrary Kamelet. +* **Named beans configured through `camel.beans.*` and references such as `#bean:<name>` — not generically implementable in the first iteration.** Individual bean properties may appear in static configuration, but the operator cannot safely infer which bean is actually used as the connection factory, or how it is used, in the general case. Bean names and property values alone do not establish that relationship. +* **Bean definitions in DSL source, factory methods, nested bean references, or external registries — not implementable in the first iteration.** Scanning source code to detect connection-factory use is complex across supported DSLs, and there may be several levels of indirection. Executing factories or instantiating beans to discover their values is out of scope. +* **Placeholders, environment variables, mounted configuration, and Kubernetes Secrets — only statically visible, non-sensitive configuration is eligible.** The first iteration should not attempt generic runtime resolution. Secret-backed credentials must remain references and be configured through the appropriate KEDA authentication settings, not copied into plain scaler metadata. + +The objective is therefore not universal connection-bean introspection. It is to increase automatic discovery where the configuration and its relationship to a known endpoint or Kamelet are explicit, while leaving unsupported or dynamic cases to explicit user configuration. + +== Current implementation and related work + +The current flow is implemented in `pkg/trait/keda.go`, `pkg/metadata/metadata.go`, and `pkg/trait/keda/mapper.go`: + +. The KEDA trait calls `metadata.ExtractAll` on the Integration's sources. +. Source inspectors extract the endpoint URIs from the Integration DSL, and the KEDA trait iterates the discovered `from` URIs. +. `MapToKedaTrigger` parses each URI into the component name, path value, and query parameters. +. The registered `ScaleMapper` maps the path and parameters to a KEDA scaler type and metadata. +. The trait merges `keda.autoMetadata` for the discovered scaler type and later creates a `ScaledObject`. + +=== Endpoint URI mapping: already implementable + +This is the existing behavior, not a new design requirement. For example, the Kafka scaler takes the topic from the URI path and maps the `brokers` and `groupId` query parameters to the Kafka scaler's `bootstrapServers` and `consumerGroup` metadata. Each mapper intentionally selects the URI fields relevant to its scaler. Values that are not present in the URI cannot be obtained through this path alone. + +The existing `ScaleMapper.Map` contract receives the endpoint path and URI parameters only. It does not receive a resolved Camel registry, bean instances, or a generic description of how a bean was constructed. + +=== Integration/component property mapping: implementable extension + +https://github.com/apache/camel-k/pull/6893[PR #6893] demonstrates the next, narrower step for the ArtemisQueue mapping. The proposed `integrationProperties(e)` helper collects Integration configuration entries and properties configured through the Camel trait, then passes that map to `MapToKedaTrigger` and the scaler mapper. The Artemis mapper can use a directly declared property such as `camel.component.amqp.broker-url` to derive the scaler's management endpoint. + +This approach uses configuration already available to Camel K and avoids querying the broker or constructing a connection factory. It should be generalized only with documented input/precedence rules and tests. The related PR remains open, so the property-map extension must not be described as already merged into upstream `main`. + +=== Pipe-derived Integrations and Kamelet properties: partially implementable + +A Pipe is converted into an Integration by `pkg/controller/pipe/integration.go`, which calls the binding converters for its source and sink. When a Pipe endpoint references a Kamelet, `pkg/util/bindings/kamelet.go` reads the endpoint's declared properties, stores them as Integration application properties using keys of the form `camel.kamelet.<kamelet-name>.<id>.<property>`, and emits a `kamelet:<name>/<id>` endpoint URI. + +This creates two useful facts for discovery: the generated Integration contains the explicitly configured Kamelet invocation properties, but its endpoint URI identifies the Kamelet rather than necessarily exposing the underlying consumer component URI. Consequently, the current URI-only mapper cannot infer an inner AMQP or other component endpoint solely from a `kamelet:` URI. Review Comment: Good point. We could only support Kamelets from Apache catalog, but it would be asymmetric towards any user provided Kamelet. Given this limitation we should probably remove this requirement from the design and the implementation. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
