[ 
https://issues.apache.org/jira/browse/CAMEL-25648?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Luigi De Masi updated CAMEL-25648:
----------------------------------
    Description: 
h2. Motivation

Follow up CAMEL-25618 and [PR 
#27791|https://github.com/apache/camel/pull/27791] with an optional integration 
for production semantic monitoring through Micrometer and Prometheus.

The semantic audit/monitoring foundation supplies typed events and current 
measurements through a bounded runtime cache. It does not yet export semantic 
measurements to Micrometer. Durable time-series history, querying, analysis and 
correlation belong to the external monitoring system. Operators should be able 
to monitor evaluation rates, execution failures, latency and returned-result 
distributions across application instances and restarts, using the monitoring 
backend independently of the Camel TUI.

h2. Delivery plan and current status

[PR #29806|https://github.com/apache/camel/pull/29806] implements the shared 
exporter in {{camel-semantic-micrometer}} and is under review. The module split 
follows the [agreed packaging 
approach|https://github.com/apache/camel/pull/29806#issuecomment-6851050690]: 
the bridge depends on both Camel components, while Semantic continues 
publishing backend-independent events. Adding the artifact does not 
automatically register a collector; applications explicitly register 
{{MicrometerSemanticMetrics}} with their existing registry.

Local validation of commit {{cd66663cda8f}} passed: 428 Semantic tests, 12 
bridge tests, 1,131 catalog tests, a standalone Maven consumer with only the 
bridge dependency, the full 723-module build with {{mvnd clean install 
-DskipTests}}, and Antora documentation validation. The Semantic module's 
classpath regression confirms evaluation works without Micrometer, and 
packaged-JAR inspection confirms that exporter classes live only in the bridge. 
These are local results, not a claim that hosted CI or runtime/native 
acceptance is complete.

Remaining runtime work is separate from PR #29806:
* Camel Main registration and the management metrics endpoint example.
* Spring Boot starter/Actuator integration and example.
* Camel Quarkus Semantic extension/integration, including JVM and native 
validation.
* Equivalent runnable examples exercised against a real Prometheus server in 
all three runtimes.

This issue remains open after the shared-exporter PR merges until the runtime 
work and the acceptance criteria below are satisfied. The staged delivery does 
not remove the original runtime requirements.

h2. Scope

h3. Shared semantic metrics integration

* Provide the optional, runtime-independent {{camel-semantic-micrometer}} 
integration module with normal dependencies on {{camel-semantic}} and 
{{camel-micrometer}}. Applications add the bridge artifact to obtain both 
components transitively. Keep all Micrometer-specific exporter code and 
dependencies out of {{camel-semantic}}; the general {{camel-micrometer}} 
component does not depend on Semantic. Reuse the application/runtime 
{{MeterRegistry}} and its lifecycle. A Prometheus registry implementation 
remains an application/runtime choice.
* Use the typed semantic event/measurement foundation from CAMEL-25618. Give 
each measurement one ingestion path, independent of audit retention, input 
capture, JMX and UI polling. Avoid duplicate counts from request lifecycle 
events, batch envelopes or concurrent monitoring consumers.
* Record completed assessment members, execution outcomes, result availability 
and evaluation duration. Keep provider attempts/retries separate from semantic 
completions. Record duration observations in timers/histograms so percentiles 
can be queried correctly; do not reconstruct percentiles from snapshot means.
* Preserve Boolean, choice, score and classification semantics in result 
counts/distributions. Failures or missing results are not false answers; 
probability entries are not additional completed answers; classification may 
select multiple labels. Export expert-supplied measurements only when their 
units and aggregation semantics permit a documented mapping.
* Support aggregation by expert or definition using consistent, bounded 
dimensions. Direct expert calls remain distinguishable from definition 
evaluations. Bound series/cardinality and expose collection exclusions or loss. 
Do not use prompts, input, free-form interpretation, exception messages, 
correlation IDs, exchange IDs, breadcrumb IDs or invocation IDs as metric 
labels. Keep per-execution identifiers in audit/log/trace records. Standard 
trace/span exemplars may provide sampled links where runtime tracing and the 
registry support them, without unique metric labels or Camel-owned example 
audit-event ID lists.
* Keep event callbacks short. Scraping, historical queries and backend I/O must 
not block evaluation callbacks. Monitoring failures must not change the 
assessment result.

h3. Runtime integration

Use the same metric names, units, labels and counting semantics in all three 
runtimes, with each runtime supplying its normal registry and HTTP exposure:

||Runtime||Existing integration to reuse||
|Camel Main|{{camel-micrometer-prometheus}} and the management metrics 
endpoint.|
|Camel Spring Boot|{{camel-micrometer-starter}}, Spring Boot Actuator and 
{{micrometer-registry-prometheus}}.|
|Camel Quarkus|{{camel-quarkus-micrometer}} and 
{{quarkus-micrometer-registry-prometheus}}.|

Provide configuration/examples and integration coverage for all three, 
coordinating starter/extension changes in their respective repositories. 
Validate the new integration on the JVM and in a supported Quarkus native 
configuration; existing Micrometer native support alone is not proof of 
semantic integration support.

h3. Responsibility boundary

Micrometer records measurements in the application registry. Prometheus scrapes 
them and owns time-series storage and retention. Grafana or another monitoring 
application owns historical queries, time selection, comparisons, anomaly 
detection, alerting, dashboards and cross-signal navigation.

This issue adds instrumentation and export only: no SemanticMonitoringStore 
implementation, snapshot-document persistence, Prometheus query client in 
Camel, historical TUI integration, baseline/comparison model or example 
audit-event IDs. Camel's local Monitoring view remains a current-value 
inspector and works without Prometheus.

Provide example PromQL queries for operators to use in their monitoring 
backend, including rates, result distributions, latency histograms, counter 
resets and multiple instances. Do not turn those examples into a Camel query 
service. Audit investigation can filter a matching [since, until) interval, 
definition/operation and typed result. Audit capture and retention may cover 
only a subset of the monitored population; per-evaluation correlation labels 
are unnecessary.

h2. Acceptance criteria

# Adding {{camel-semantic-micrometer}} brings {{camel-semantic}} and 
{{camel-micrometer}} transitively; using {{camel-semantic}} alone requires no 
Micrometer dependency and continues to work without it.
# A runnable example in each runtime exposes equivalent semantic metrics to a 
real Prometheus scrape, with documented setup and queries for evaluation rate, 
failure rate, duration and result distributions.
# Tests cover audit-disabled operation, direct and definition evaluations, 
batches, failures, restart/reset behavior, duplicate-count prevention and 
bounded cardinality.
# Histogram-based latency queries use actual duration observations. Result 
metrics retain their assessment meaning and never imply route actions, accuracy 
loss or security incidents.
# Example backend queries cover expert/definition grouping, rates, counter 
resets and multi-instance aggregation. Historical queries and presentation 
remain in the monitoring backend; Camel adds no history-query service or 
dashboard engine.
# Documentation explains registry ownership, scrape cadence, labels, units, 
retention, multi-instance aggregation and the distinction between backend 
metric history, transient current-value reads and independently retained audit 
records. Document exemplar availability/limits without promising one link per 
evaluation.
# No new inference, synthetic demonstration measurements or raw-input retention 
is required for monitoring.

h2. References

* [Shared exporter PR #29806|https://github.com/apache/camel/pull/29806]
* CAMEL-25618
* [Camel Main Micrometer 
Prometheus|https://camel.apache.org/components/next/others/micrometer-prometheus.html]
* [Camel Spring Boot 
Micrometer|https://camel.apache.org/camel-spring-boot/next/starters/micrometer.html]
* [Camel Quarkus 
Micrometer|https://camel.apache.org/camel-quarkus/next/reference/extensions/micrometer.html]

----
_AI-generated by Codex on behalf of 
[luigidemasi|https://github.com/luigidemasi]._

  was:
h2. Motivation

Follow up CAMEL-25618 and [PR 
#27791|https://github.com/apache/camel/pull/27791] with an optional integration 
for production semantic monitoring through Micrometer and Prometheus.

The semantic audit/monitoring foundation supplies typed events and current 
measurements through a bounded runtime cache. It does not yet export semantic 
measurements to Micrometer. Durable time-series history, querying, analysis and 
correlation belong to the external monitoring system. Operators should be able 
to monitor evaluation rates, execution failures, latency and returned-result 
distributions across application instances and restarts, using the monitoring 
backend independently of the Camel TUI.

h2. Scope

h3. Shared semantic metrics integration

* Add an optional runtime-independent integration using Micrometer's 
{{MeterRegistry}}. Reuse the application/runtime registry and its lifecycle; 
keep Micrometer and Prometheus optional dependencies for semantic users.
* Use the typed semantic event/measurement foundation from CAMEL-25618. Give 
each measurement one ingestion path, independent of audit retention, input 
capture, JMX and UI polling. Avoid duplicate counts from request lifecycle 
events, batch envelopes or concurrent monitoring consumers.
* Record completed assessment members, execution outcomes, result availability 
and evaluation duration. Keep provider attempts/retries separate from semantic 
completions. Record duration observations in timers/histograms so percentiles 
can be queried correctly; do not reconstruct percentiles from snapshot means.
* Preserve Boolean, choice, score and classification semantics in result 
counts/distributions. Failures or missing results are not false answers; 
probability entries are not additional completed answers; classification may 
select multiple labels. Export expert-supplied measurements only when their 
units and aggregation semantics permit a documented mapping.
* Support aggregation by expert or definition using consistent, bounded 
dimensions. Direct expert calls remain distinguishable from definition 
evaluations. Bound series/cardinality and expose collection exclusions or loss. 
Do not use prompts, input, free-form interpretation, exception messages, 
correlation IDs, exchange IDs, breadcrumb IDs or invocation IDs as metric 
labels. Keep per-execution identifiers in audit/log/trace records. Standard 
trace/span exemplars may provide sampled links where runtime tracing and the 
registry support them, without unique metric labels or Camel-owned example 
audit-event ID lists.
* Keep event callbacks short. Scraping, historical queries and backend I/O must 
not block evaluation callbacks. Monitoring failures must not change the 
assessment result.

h3. Runtime integration

Use the same metric names, units, labels and counting semantics in all three 
runtimes, with each runtime supplying its normal registry and HTTP exposure:

||Runtime||Existing integration to reuse||
|Camel Main|{{camel-micrometer-prometheus}} and the management metrics 
endpoint.|
|Camel Spring Boot|{{camel-micrometer-starter}}, Spring Boot Actuator and 
{{micrometer-registry-prometheus}}.|
|Camel Quarkus|{{camel-quarkus-micrometer}} and 
{{quarkus-micrometer-registry-prometheus}}.|

Provide configuration/examples and integration coverage for all three, 
coordinating starter/extension changes in their respective repositories. 
Validate the new integration on the JVM and in a supported Quarkus native 
configuration; existing Micrometer native support alone is not proof of 
semantic integration support.

h3. Responsibility boundary

Micrometer records measurements in the application registry. Prometheus scrapes 
them and owns time-series storage and retention. Grafana or another monitoring 
application owns historical queries, time selection, comparisons, anomaly 
detection, alerting, dashboards and cross-signal navigation.

This issue adds instrumentation and export only: no SemanticMonitoringStore 
implementation, snapshot-document persistence, Prometheus query client in 
Camel, historical TUI integration, baseline/comparison model or example 
audit-event IDs. Camel's local Monitoring view remains a current-value 
inspector and works without Prometheus.

Provide example PromQL queries for operators to use in their monitoring 
backend, including rates, result distributions, latency histograms, counter 
resets and multiple instances. Do not turn those examples into a Camel query 
service. Audit investigation can filter a matching [since, until) interval, 
definition/operation and typed result. Audit capture and retention may cover 
only a subset of the monitored population; per-evaluation correlation labels 
are unnecessary.

h2. Acceptance criteria

# A runnable example in each runtime exposes equivalent semantic metrics to a 
real Prometheus scrape, with documented setup and queries for evaluation rate, 
failure rate, duration and result distributions.
# Tests cover audit-disabled operation, direct and definition evaluations, 
batches, failures, restart/reset behavior, duplicate-count prevention and 
bounded cardinality.
# Histogram-based latency queries use actual duration observations. Result 
metrics retain their assessment meaning and never imply route actions, accuracy 
loss or security incidents.
# Example backend queries cover expert/definition grouping, rates, counter 
resets and multi-instance aggregation. Historical queries and presentation 
remain in the monitoring backend; Camel adds no history-query service or 
dashboard engine.
# Documentation explains registry ownership, scrape cadence, labels, units, 
retention, multi-instance aggregation and the distinction between backend 
metric history, transient current-value reads and independently retained audit 
records. Document exemplar availability/limits without promising one link per 
evaluation.
# No new inference, synthetic demonstration measurements or raw-input retention 
is required for monitoring.

h2. References

* CAMEL-25618
* [Camel Main Micrometer 
Prometheus|https://camel.apache.org/components/next/others/micrometer-prometheus.html]
* [Camel Spring Boot 
Micrometer|https://camel.apache.org/camel-spring-boot/next/starters/micrometer.html]
* [Camel Quarkus 
Micrometer|https://camel.apache.org/camel-quarkus/next/reference/extensions/micrometer.html]

----
_AI-generated by Codex on behalf of 
[luigidemasi|https://github.com/luigidemasi]._

        Summary: camel-semantic-micrometer - Micrometer instrumentation and 
Prometheus export  (was: camel-semantic - Micrometer instrumentation and 
Prometheus export)

> camel-semantic-micrometer - Micrometer instrumentation and Prometheus export
> ----------------------------------------------------------------------------
>
>                 Key: CAMEL-25648
>                 URL: https://issues.apache.org/jira/browse/CAMEL-25648
>             Project: Camel
>          Issue Type: Improvement
>          Components: camel-ai, camel-jbang, camel-micrometer
>            Reporter: Luigi De Masi
>            Assignee: Luigi De Masi
>            Priority: Major
>
> h2. Motivation
> Follow up CAMEL-25618 and [PR 
> #27791|https://github.com/apache/camel/pull/27791] with an optional 
> integration for production semantic monitoring through Micrometer and 
> Prometheus.
> The semantic audit/monitoring foundation supplies typed events and current 
> measurements through a bounded runtime cache. It does not yet export semantic 
> measurements to Micrometer. Durable time-series history, querying, analysis 
> and correlation belong to the external monitoring system. Operators should be 
> able to monitor evaluation rates, execution failures, latency and 
> returned-result distributions across application instances and restarts, 
> using the monitoring backend independently of the Camel TUI.
> h2. Delivery plan and current status
> [PR #29806|https://github.com/apache/camel/pull/29806] implements the shared 
> exporter in {{camel-semantic-micrometer}} and is under review. The module 
> split follows the [agreed packaging 
> approach|https://github.com/apache/camel/pull/29806#issuecomment-6851050690]: 
> the bridge depends on both Camel components, while Semantic continues 
> publishing backend-independent events. Adding the artifact does not 
> automatically register a collector; applications explicitly register 
> {{MicrometerSemanticMetrics}} with their existing registry.
> Local validation of commit {{cd66663cda8f}} passed: 428 Semantic tests, 12 
> bridge tests, 1,131 catalog tests, a standalone Maven consumer with only the 
> bridge dependency, the full 723-module build with {{mvnd clean install 
> -DskipTests}}, and Antora documentation validation. The Semantic module's 
> classpath regression confirms evaluation works without Micrometer, and 
> packaged-JAR inspection confirms that exporter classes live only in the 
> bridge. These are local results, not a claim that hosted CI or runtime/native 
> acceptance is complete.
> Remaining runtime work is separate from PR #29806:
> * Camel Main registration and the management metrics endpoint example.
> * Spring Boot starter/Actuator integration and example.
> * Camel Quarkus Semantic extension/integration, including JVM and native 
> validation.
> * Equivalent runnable examples exercised against a real Prometheus server in 
> all three runtimes.
> This issue remains open after the shared-exporter PR merges until the runtime 
> work and the acceptance criteria below are satisfied. The staged delivery 
> does not remove the original runtime requirements.
> h2. Scope
> h3. Shared semantic metrics integration
> * Provide the optional, runtime-independent {{camel-semantic-micrometer}} 
> integration module with normal dependencies on {{camel-semantic}} and 
> {{camel-micrometer}}. Applications add the bridge artifact to obtain both 
> components transitively. Keep all Micrometer-specific exporter code and 
> dependencies out of {{camel-semantic}}; the general {{camel-micrometer}} 
> component does not depend on Semantic. Reuse the application/runtime 
> {{MeterRegistry}} and its lifecycle. A Prometheus registry implementation 
> remains an application/runtime choice.
> * Use the typed semantic event/measurement foundation from CAMEL-25618. Give 
> each measurement one ingestion path, independent of audit retention, input 
> capture, JMX and UI polling. Avoid duplicate counts from request lifecycle 
> events, batch envelopes or concurrent monitoring consumers.
> * Record completed assessment members, execution outcomes, result 
> availability and evaluation duration. Keep provider attempts/retries separate 
> from semantic completions. Record duration observations in timers/histograms 
> so percentiles can be queried correctly; do not reconstruct percentiles from 
> snapshot means.
> * Preserve Boolean, choice, score and classification semantics in result 
> counts/distributions. Failures or missing results are not false answers; 
> probability entries are not additional completed answers; classification may 
> select multiple labels. Export expert-supplied measurements only when their 
> units and aggregation semantics permit a documented mapping.
> * Support aggregation by expert or definition using consistent, bounded 
> dimensions. Direct expert calls remain distinguishable from definition 
> evaluations. Bound series/cardinality and expose collection exclusions or 
> loss. Do not use prompts, input, free-form interpretation, exception 
> messages, correlation IDs, exchange IDs, breadcrumb IDs or invocation IDs as 
> metric labels. Keep per-execution identifiers in audit/log/trace records. 
> Standard trace/span exemplars may provide sampled links where runtime tracing 
> and the registry support them, without unique metric labels or Camel-owned 
> example audit-event ID lists.
> * Keep event callbacks short. Scraping, historical queries and backend I/O 
> must not block evaluation callbacks. Monitoring failures must not change the 
> assessment result.
> h3. Runtime integration
> Use the same metric names, units, labels and counting semantics in all three 
> runtimes, with each runtime supplying its normal registry and HTTP exposure:
> ||Runtime||Existing integration to reuse||
> |Camel Main|{{camel-micrometer-prometheus}} and the management metrics 
> endpoint.|
> |Camel Spring Boot|{{camel-micrometer-starter}}, Spring Boot Actuator and 
> {{micrometer-registry-prometheus}}.|
> |Camel Quarkus|{{camel-quarkus-micrometer}} and 
> {{quarkus-micrometer-registry-prometheus}}.|
> Provide configuration/examples and integration coverage for all three, 
> coordinating starter/extension changes in their respective repositories. 
> Validate the new integration on the JVM and in a supported Quarkus native 
> configuration; existing Micrometer native support alone is not proof of 
> semantic integration support.
> h3. Responsibility boundary
> Micrometer records measurements in the application registry. Prometheus 
> scrapes them and owns time-series storage and retention. Grafana or another 
> monitoring application owns historical queries, time selection, comparisons, 
> anomaly detection, alerting, dashboards and cross-signal navigation.
> This issue adds instrumentation and export only: no SemanticMonitoringStore 
> implementation, snapshot-document persistence, Prometheus query client in 
> Camel, historical TUI integration, baseline/comparison model or example 
> audit-event IDs. Camel's local Monitoring view remains a current-value 
> inspector and works without Prometheus.
> Provide example PromQL queries for operators to use in their monitoring 
> backend, including rates, result distributions, latency histograms, counter 
> resets and multiple instances. Do not turn those examples into a Camel query 
> service. Audit investigation can filter a matching [since, until) interval, 
> definition/operation and typed result. Audit capture and retention may cover 
> only a subset of the monitored population; per-evaluation correlation labels 
> are unnecessary.
> h2. Acceptance criteria
> # Adding {{camel-semantic-micrometer}} brings {{camel-semantic}} and 
> {{camel-micrometer}} transitively; using {{camel-semantic}} alone requires no 
> Micrometer dependency and continues to work without it.
> # A runnable example in each runtime exposes equivalent semantic metrics to a 
> real Prometheus scrape, with documented setup and queries for evaluation 
> rate, failure rate, duration and result distributions.
> # Tests cover audit-disabled operation, direct and definition evaluations, 
> batches, failures, restart/reset behavior, duplicate-count prevention and 
> bounded cardinality.
> # Histogram-based latency queries use actual duration observations. Result 
> metrics retain their assessment meaning and never imply route actions, 
> accuracy loss or security incidents.
> # Example backend queries cover expert/definition grouping, rates, counter 
> resets and multi-instance aggregation. Historical queries and presentation 
> remain in the monitoring backend; Camel adds no history-query service or 
> dashboard engine.
> # Documentation explains registry ownership, scrape cadence, labels, units, 
> retention, multi-instance aggregation and the distinction between backend 
> metric history, transient current-value reads and independently retained 
> audit records. Document exemplar availability/limits without promising one 
> link per evaluation.
> # No new inference, synthetic demonstration measurements or raw-input 
> retention is required for monitoring.
> h2. References
> * [Shared exporter PR #29806|https://github.com/apache/camel/pull/29806]
> * CAMEL-25618
> * [Camel Main Micrometer 
> Prometheus|https://camel.apache.org/components/next/others/micrometer-prometheus.html]
> * [Camel Spring Boot 
> Micrometer|https://camel.apache.org/camel-spring-boot/next/starters/micrometer.html]
> * [Camel Quarkus 
> Micrometer|https://camel.apache.org/camel-quarkus/next/reference/extensions/micrometer.html]
> ----
> _AI-generated by Codex on behalf of 
> [luigidemasi|https://github.com/luigidemasi]._



--
This message was sent by Atlassian Jira
(v10.3.26#10030026)

Reply via email to