This is an automated email from the ASF dual-hosted git repository.
Croway pushed a commit to branch main
in repository https://gitbox.apache.org/repos/asf/camel-spring-boot-examples.git
The following commit(s) were added to refs/heads/main by this push:
new b26889dc Rework genai-observability example: camel infra stack, two
models, GenAI dashboard
b26889dc is described below
commit b26889dce976b34cff05441e3d5f3fc3ebcdd544
Author: croway <[email protected]>
AuthorDate: Mon Aug 31 10:45:26 2026 +0200
Rework genai-observability example: camel infra stack, two models, GenAI
dashboard
- Replace docker-compose with 'camel infra run observability' (Prometheus
scrapes
/observe/metrics on the 9876 management port, VictoriaTraces OTLP on
10428)
- Fix startup on Spring Boot 4: drop the incompatible langchain4j Ollama
starter
(langchain4j/langchain4j#6236) and build the ChatModel beans from plain
langchain4j-ollama; add the missing camel-yaml-dsl-starter
- Export traces via Micrometer Tracing with the OpenTelemetry bridge, so
route
spans and gen_ai.* observation spans reach VictoriaTraces (CAMEL-24570)
- Use two small models on two routes so all GenAI metrics/spans carry
distinct
gen_ai.request.model series
- Add a Perses GenAI dashboard (perses-genai-dashboard.json) and document
the
panels with screenshots in the README
Co-Authored-By: Claude Fable 5 <[email protected]>
---
genai-observability/README.adoc | 106 +++-
genai-observability/docker-compose.yml | 20 -
genai-observability/docs/genai-dashboard-calls.png | Bin 0 -> 237089 bytes
.../docs/genai-dashboard-tokens.png | Bin 0 -> 94387 bytes
genai-observability/perses-genai-dashboard.json | 566 +++++++++++++++++++++
genai-observability/pom.xml | 25 +-
genai-observability/prometheus.yml | 16 -
.../example/genai/ChatModelConfiguration.java | 61 +++
.../src/main/resources/application.properties | 13 +-
.../main/resources/camel/genai-route.camel.yaml | 26 +-
10 files changed, 773 insertions(+), 60 deletions(-)
diff --git a/genai-observability/README.adoc b/genai-observability/README.adoc
index 777f7342..f216e92b 100644
--- a/genai-observability/README.adoc
+++ b/genai-observability/README.adoc
@@ -4,7 +4,7 @@
This example demonstrates *GenAI observability* in Apache Camel 4.23+ with
Spring Boot:
OpenTelemetry `gen_ai.*` span attributes and Micrometer metrics for
`langchain4j-chat`
-LLM calls, visualized with Prometheus and VictoriaTraces.
+LLM calls, visualized with Prometheus, VictoriaTraces and Perses.
It pairs with the community blog post *GenAI Observability with Spring Boot
and the Camel Observability Stack*
(`docs/blog-drafts/genai-observability-02-spring-boot-obs-stack.adoc` in the
@@ -14,12 +14,29 @@ The example uses `camel-ai-observability-starter` for
Spring Boot configuration
GenAI observability toggle (`camel.aiobservability.enabled`), together with
`camel-ai-observability`
for span and metric emission when a tracing or metrics backend is present.
+Two routes call two different (small) Ollama models, so every GenAI metric and
span carries a
+distinct `gen_ai.request.model` and dashboards show per-model series side by
side.
+
+The `camel-observability-services-starter` dependency configures an
opinionated observability
+setup: all Actuator endpoints move to a dedicated management port `9876` under
the `/observe`
+base path, with the Prometheus endpoint mapped to `/observe/metrics`. Both the
standard
+Spring Boot / Camel metrics and the `gen_ai.*` metrics are exposed there,
since they share
+the same Micrometer registry.
+
+Traces use the Spring Boot idiomatic setup: Micrometer Tracing with the
OpenTelemetry bridge
+(`spring-boot-micrometer-tracing-opentelemetry`,
`micrometer-tracing-bridge-otel` and
+`opentelemetry-exporter-otlp`). Spring Boot auto-configures the OpenTelemetry
SDK, the OTLP
+span exporter (`management.opentelemetry.tracing.export.otlp.endpoint`) and a
tracing handler
+on the Actuator `ObservationRegistry`. Camel route spans (via
`camel-opentelemetry2`) and the
+`gen_ai.*` client spans (recorded as Micrometer Observations) end up in the
same trace.
+
=== Prerequisites
* Java 17+
* Maven 3.9+
-* Docker (for observability stack)
-* https://ollama.com/[Ollama] with `llama3.2` pulled
+* https://camel.apache.org/manual/camel-jbang.html[Camel CLI (JBang)] 4.22+
+* Docker (used by the Camel CLI to start the observability stack)
+* https://ollama.com/[Ollama] with `llama3.2:1b` and `qwen3:0.6b` pulled
== Build
@@ -34,17 +51,21 @@ Terminal 1 — Ollama:
[source,shell]
----
-ollama pull llama3.2
+ollama pull llama3.2:1b
+ollama pull qwen3:0.6b
ollama serve
----
-Terminal 2 — observability stack (Prometheus + VictoriaTraces + Perses):
+Terminal 2 — observability stack (Prometheus + VictoriaTraces + VictoriaLogs +
Perses):
[source,shell]
----
-docker compose up -d
+camel infra run observability
----
+The stack starts on fixed ports and the bundled Prometheus is already
configured to scrape
+the application metrics at `host.docker.internal:9876/observe/metrics`.
+
Terminal 3 — Spring Boot application:
[source,shell]
@@ -52,16 +73,83 @@ Terminal 3 — Spring Boot application:
mvn spring-boot:run
----
+To use different models than the defaults:
+
+[source,shell]
+----
+mvn spring-boot:run
-Dspring-boot.run.arguments="--langchain4j.ollama.chat-model-1.model-name=llama3.2
--langchain4j.ollama.chat-model-2.model-name=qwen3:1.7b"
+----
+
== Verify GenAI observability
+Metrics are exposed on the management port `9876` (not the application port
`8080`):
+
[source,shell]
----
-curl -s http://localhost:8080/actuator/prometheus | grep gen_ai
+curl -s http://localhost:9876/observe/metrics | grep gen_ai
----
* Prometheus: `http://localhost:9090`
-* VictoriaTraces: `http://localhost:9428/select/vmui`
-* Perses: `http://localhost:8088`
+* VictoriaTraces: `http://localhost:10428/select/vmui`
+* Perses: `http://localhost:3000`
+
+== Perses dashboards
+
+The observability stack provisions Perses with a general Camel dashboard
(uptime, exchanges,
+routes, JVM):
+
+* Camel overview: `http://localhost:3000/projects/camel/dashboards/overview`
+
+A GenAI dashboard (LLM calls, per-model latency, error ratio, token usage) can
be created
+through the Perses REST API using the dashboard definition in
`perses-genai-dashboard.json`:
+
+[source,shell]
+----
+curl -X POST http://localhost:3000/api/v1/projects \
+ -H 'Content-Type: application/json' \
+ -d '{"kind":"Project","metadata":{"name":"camel_genai"},"spec":{}}'
+
+curl -X POST http://localhost:3000/api/v1/projects/camel_genai/dashboards \
+ -H 'Content-Type: application/json' \
+ --data @perses-genai-dashboard.json
+----
+
+* GenAI overview:
`http://localhost:3000/projects/camel_genai/dashboards/overview`
+
+The dashboard itself contains a "How this dashboard was created" section with
the same
+instructions. Note that Perses state lives in the container: after restarting
the
+observability stack, re-run the two commands above to recreate the GenAI
dashboard.
+
+=== Reading the GenAI dashboard
+
+image::docs/genai-dashboard-calls.png[GenAI Summary and LLM Calls sections]
+
+* *GenAI Summary* — running totals across all models: LLM calls, errors,
in-flight calls,
+ and input/output token counters. With calls that take longer than the timer
period,
+ "In-flight Calls" sits permanently at 1: the route serializes requests, and
a new one
+ starts as soon as the previous completes.
+* *Call rate* — calls per second, one series per `gen_ai.request.model`. A
fast model
+ settles at the timer frequency; a slow model's rate is capped by its own
latency.
+* *Error ratio* — failed calls (tagged `error!="none"`) over total, per model.
Flat 0%
+ lines mean every call succeeded.
+* *Mean / Max LLM latency* — per-model duration of the LLM call itself (the
+ `gen_ai.client.operation` timer, not the whole route).
+
+image::docs/genai-dashboard-tokens.png[Token Usage section]
+
+* *Token throughput* and *Avg tokens per call* — from the
`gen_ai.client.token.usage`
+ counter, split by model and `gen_ai.token.type` (input/output).
+
+The screenshots were taken with `llama3.2` and `qwen3.5:0.8b` (via the
model-name
+overrides shown above); the default models behave similarly, since the `qwen3`
family
+also reasons before answering. The two models make the point of GenAI
observability
+visible on identical prompts:
+`llama3.2` answers in about a second with a few dozen output tokens, while
+`qwen3.5:0.8b` - a *thinking* model - emits roughly 2K output tokens per call
(mostly
+reasoning tokens before the one-sentence answer), which drives both its
~30-45s latency
+and its dominant share of token throughput. Same workload, an order of
magnitude more
+token spend - exactly the kind of cost/latency trade-off these metrics are
meant to
+surface.
== Help and contributions
diff --git a/genai-observability/docker-compose.yml
b/genai-observability/docker-compose.yml
deleted file mode 100644
index 71034773..00000000
--- a/genai-observability/docker-compose.yml
+++ /dev/null
@@ -1,20 +0,0 @@
-# Observability stack for Camel GenAI observability blog (Blog 2)
-# Images aligned with camel-test-infra-observability
-
-services:
- prometheus:
- image: quay.io/prometheus/prometheus:v3.13.2
- ports:
- - "9090:9090"
- volumes:
- - ./prometheus.yml:/etc/prometheus/prometheus.yml:ro
-
- victoriatraces:
- image: mirror.gcr.io/victoriametrics/victoria-traces:v0.10.0
- ports:
- - "9428:9428"
-
- perses:
- image: mirror.gcr.io/persesdev/perses:v0.54.0
- ports:
- - "8088:8080"
diff --git a/genai-observability/docs/genai-dashboard-calls.png
b/genai-observability/docs/genai-dashboard-calls.png
new file mode 100644
index 00000000..98569acb
Binary files /dev/null and b/genai-observability/docs/genai-dashboard-calls.png
differ
diff --git a/genai-observability/docs/genai-dashboard-tokens.png
b/genai-observability/docs/genai-dashboard-tokens.png
new file mode 100644
index 00000000..89bc1cf6
Binary files /dev/null and
b/genai-observability/docs/genai-dashboard-tokens.png differ
diff --git a/genai-observability/perses-genai-dashboard.json
b/genai-observability/perses-genai-dashboard.json
new file mode 100644
index 00000000..79b1441f
--- /dev/null
+++ b/genai-observability/perses-genai-dashboard.json
@@ -0,0 +1,566 @@
+{
+ "kind": "Dashboard",
+ "metadata": {
+ "name": "overview"
+ },
+ "spec": {
+ "display": {
+ "name": "GenAI Overview"
+ },
+ "duration": "30m",
+ "panels": {
+ "llmCalls": {
+ "kind": "Panel",
+ "spec": {
+ "display": {
+ "name": "LLM Calls"
+ },
+ "plugin": {
+ "kind": "StatChart",
+ "spec": {
+ "calculation": "last-number",
+ "format": {
+ "unit": "decimal",
+ "shortValues": true
+ },
+ "sparkline": {}
+ }
+ },
+ "queries": [
+ {
+ "kind": "TimeSeriesQuery",
+ "spec": {
+ "plugin": {
+ "kind": "PrometheusTimeSeriesQuery",
+ "spec": {
+ "query": "sum(gen_ai_client_operation_seconds_count)"
+ }
+ }
+ }
+ }
+ ]
+ }
+ },
+ "llmErrors": {
+ "kind": "Panel",
+ "spec": {
+ "display": {
+ "name": "LLM Errors"
+ },
+ "plugin": {
+ "kind": "StatChart",
+ "spec": {
+ "calculation": "last-number",
+ "format": {
+ "unit": "decimal",
+ "shortValues": true
+ },
+ "sparkline": {}
+ }
+ },
+ "queries": [
+ {
+ "kind": "TimeSeriesQuery",
+ "spec": {
+ "plugin": {
+ "kind": "PrometheusTimeSeriesQuery",
+ "spec": {
+ "query":
"sum(gen_ai_client_operation_seconds_count{error!=\"none\"}) or vector(0)"
+ }
+ }
+ }
+ }
+ ]
+ }
+ },
+ "inflightCalls": {
+ "kind": "Panel",
+ "spec": {
+ "display": {
+ "name": "In-flight Calls"
+ },
+ "plugin": {
+ "kind": "StatChart",
+ "spec": {
+ "calculation": "last-number",
+ "format": {
+ "unit": "decimal",
+ "shortValues": true
+ },
+ "sparkline": {}
+ }
+ },
+ "queries": [
+ {
+ "kind": "TimeSeriesQuery",
+ "spec": {
+ "plugin": {
+ "kind": "PrometheusTimeSeriesQuery",
+ "spec": {
+ "query":
"sum(gen_ai_client_operation_active_seconds_count)"
+ }
+ }
+ }
+ }
+ ]
+ }
+ },
+ "inputTokens": {
+ "kind": "Panel",
+ "spec": {
+ "display": {
+ "name": "Input Tokens"
+ },
+ "plugin": {
+ "kind": "StatChart",
+ "spec": {
+ "calculation": "last-number",
+ "format": {
+ "unit": "decimal",
+ "shortValues": true
+ },
+ "sparkline": {}
+ }
+ },
+ "queries": [
+ {
+ "kind": "TimeSeriesQuery",
+ "spec": {
+ "plugin": {
+ "kind": "PrometheusTimeSeriesQuery",
+ "spec": {
+ "query":
"sum(gen_ai_client_token_usage_total{gen_ai_token_type=\"input\"})"
+ }
+ }
+ }
+ }
+ ]
+ }
+ },
+ "outputTokens": {
+ "kind": "Panel",
+ "spec": {
+ "display": {
+ "name": "Output Tokens"
+ },
+ "plugin": {
+ "kind": "StatChart",
+ "spec": {
+ "calculation": "last-number",
+ "format": {
+ "unit": "decimal",
+ "shortValues": true
+ },
+ "sparkline": {}
+ }
+ },
+ "queries": [
+ {
+ "kind": "TimeSeriesQuery",
+ "spec": {
+ "plugin": {
+ "kind": "PrometheusTimeSeriesQuery",
+ "spec": {
+ "query":
"sum(gen_ai_client_token_usage_total{gen_ai_token_type=\"output\"})"
+ }
+ }
+ }
+ }
+ ]
+ }
+ },
+ "callRate": {
+ "kind": "Panel",
+ "spec": {
+ "display": {
+ "name": "Call rate (calls/s)"
+ },
+ "plugin": {
+ "kind": "TimeSeriesChart",
+ "spec": {
+ "legend": {
+ "position": "bottom"
+ },
+ "yAxis": {
+ "format": {
+ "unit": "decimal",
+ "decimalPlaces": 2
+ }
+ }
+ }
+ },
+ "queries": [
+ {
+ "kind": "TimeSeriesQuery",
+ "spec": {
+ "plugin": {
+ "kind": "PrometheusTimeSeriesQuery",
+ "spec": {
+ "query": "sum by (gen_ai_system, gen_ai_request_model)
(rate(gen_ai_client_operation_seconds_count[1m]))",
+ "seriesNameFormat": "{{gen_ai_system}}
{{gen_ai_request_model}}"
+ }
+ }
+ }
+ }
+ ]
+ }
+ },
+ "errorRatio": {
+ "kind": "Panel",
+ "spec": {
+ "display": {
+ "name": "Error ratio"
+ },
+ "plugin": {
+ "kind": "TimeSeriesChart",
+ "spec": {
+ "legend": {
+ "position": "bottom"
+ },
+ "yAxis": {
+ "format": {
+ "unit": "percent-decimal",
+ "decimalPlaces": 2
+ }
+ }
+ }
+ },
+ "queries": [
+ {
+ "kind": "TimeSeriesQuery",
+ "spec": {
+ "plugin": {
+ "kind": "PrometheusTimeSeriesQuery",
+ "spec": {
+ "query": "sum by (gen_ai_request_model)
((rate(gen_ai_client_operation_seconds_count{error!=\"none\"}[5m]) or
rate(gen_ai_client_operation_seconds_count[5m]) * 0)) /
on(gen_ai_request_model) group_left sum by (gen_ai_request_model)
(rate(gen_ai_client_operation_seconds_count[5m]) > 0)",
+ "seriesNameFormat": "{{gen_ai_request_model}}"
+ }
+ }
+ }
+ }
+ ]
+ }
+ },
+ "latencyMean": {
+ "kind": "Panel",
+ "spec": {
+ "display": {
+ "name": "Mean LLM latency"
+ },
+ "plugin": {
+ "kind": "TimeSeriesChart",
+ "spec": {
+ "legend": {
+ "position": "bottom"
+ },
+ "yAxis": {
+ "format": {
+ "unit": "seconds",
+ "decimalPlaces": 2
+ }
+ }
+ }
+ },
+ "queries": [
+ {
+ "kind": "TimeSeriesQuery",
+ "spec": {
+ "plugin": {
+ "kind": "PrometheusTimeSeriesQuery",
+ "spec": {
+ "query": "sum by (gen_ai_system, gen_ai_request_model)
(rate(gen_ai_client_operation_seconds_sum[5m])) / sum by (gen_ai_system,
gen_ai_request_model) (rate(gen_ai_client_operation_seconds_count[5m]))",
+ "seriesNameFormat": "{{gen_ai_system}}
{{gen_ai_request_model}}"
+ }
+ }
+ }
+ }
+ ]
+ }
+ },
+ "latencyMax": {
+ "kind": "Panel",
+ "spec": {
+ "display": {
+ "name": "Max LLM latency"
+ },
+ "plugin": {
+ "kind": "TimeSeriesChart",
+ "spec": {
+ "legend": {
+ "position": "bottom"
+ },
+ "yAxis": {
+ "format": {
+ "unit": "seconds",
+ "decimalPlaces": 2
+ }
+ }
+ }
+ },
+ "queries": [
+ {
+ "kind": "TimeSeriesQuery",
+ "spec": {
+ "plugin": {
+ "kind": "PrometheusTimeSeriesQuery",
+ "spec": {
+ "query": "gen_ai_client_operation_seconds_max",
+ "seriesNameFormat": "{{gen_ai_system}}
{{gen_ai_request_model}}"
+ }
+ }
+ }
+ }
+ ]
+ }
+ },
+ "tokenThroughput": {
+ "kind": "Panel",
+ "spec": {
+ "display": {
+ "name": "Token throughput (tokens/s)"
+ },
+ "plugin": {
+ "kind": "TimeSeriesChart",
+ "spec": {
+ "legend": {
+ "position": "bottom"
+ },
+ "yAxis": {
+ "format": {
+ "unit": "decimal",
+ "decimalPlaces": 2
+ }
+ }
+ }
+ },
+ "queries": [
+ {
+ "kind": "TimeSeriesQuery",
+ "spec": {
+ "plugin": {
+ "kind": "PrometheusTimeSeriesQuery",
+ "spec": {
+ "query": "sum by (gen_ai_request_model, gen_ai_token_type)
(rate(gen_ai_client_token_usage_total[1m]))",
+ "seriesNameFormat": "{{gen_ai_request_model}}
{{gen_ai_token_type}}"
+ }
+ }
+ }
+ }
+ ]
+ }
+ },
+ "tokensPerCall": {
+ "kind": "Panel",
+ "spec": {
+ "display": {
+ "name": "Avg tokens per call"
+ },
+ "plugin": {
+ "kind": "TimeSeriesChart",
+ "spec": {
+ "legend": {
+ "position": "bottom"
+ },
+ "yAxis": {
+ "format": {
+ "unit": "decimal",
+ "decimalPlaces": 2
+ }
+ }
+ }
+ },
+ "queries": [
+ {
+ "kind": "TimeSeriesQuery",
+ "spec": {
+ "plugin": {
+ "kind": "PrometheusTimeSeriesQuery",
+ "spec": {
+ "query": "sum by (gen_ai_request_model, gen_ai_token_type)
(rate(gen_ai_client_token_usage_total[5m])) / on(gen_ai_request_model)
group_left sum by (gen_ai_request_model)
(rate(gen_ai_client_operation_seconds_count[5m]))",
+ "seriesNameFormat": "{{gen_ai_request_model}}
{{gen_ai_token_type}}"
+ }
+ }
+ }
+ }
+ ]
+ }
+ },
+ "howTo": {
+ "kind": "Panel",
+ "spec": {
+ "display": {
+ "name": "How this dashboard was created"
+ },
+ "plugin": {
+ "kind": "Markdown",
+ "spec": {
+ "text": "## How this dashboard was created\n\nMetrics come from
**camel-ai-observability**: the Spring Boot app exposes `gen_ai.*` meters
on\n`http://localhost:9876/observe/metrics` and the stack's Prometheus scrapes
that endpoint.\n\nThe dashboard is plain JSON managed through the Perses REST
API. To recreate it (e.g. after\nrestarting `camel infra run observability`),
from the `genai-observability` example directory:\n\n```\ncurl -X POST
http://localhost:3000/api/v1/proje [...]
+ }
+ }
+ }
+ }
+ },
+ "layouts": [
+ {
+ "kind": "Grid",
+ "spec": {
+ "display": {
+ "title": "GenAI Summary",
+ "collapse": {
+ "open": true
+ }
+ },
+ "items": [
+ {
+ "x": 0,
+ "y": 0,
+ "width": 4,
+ "height": 4,
+ "content": {
+ "$ref": "#/spec/panels/llmCalls"
+ }
+ },
+ {
+ "x": 4,
+ "y": 0,
+ "width": 4,
+ "height": 4,
+ "content": {
+ "$ref": "#/spec/panels/llmErrors"
+ }
+ },
+ {
+ "x": 8,
+ "y": 0,
+ "width": 4,
+ "height": 4,
+ "content": {
+ "$ref": "#/spec/panels/inflightCalls"
+ }
+ },
+ {
+ "x": 12,
+ "y": 0,
+ "width": 4,
+ "height": 4,
+ "content": {
+ "$ref": "#/spec/panels/inputTokens"
+ }
+ },
+ {
+ "x": 16,
+ "y": 0,
+ "width": 4,
+ "height": 4,
+ "content": {
+ "$ref": "#/spec/panels/outputTokens"
+ }
+ }
+ ]
+ }
+ },
+ {
+ "kind": "Grid",
+ "spec": {
+ "display": {
+ "title": "LLM Calls",
+ "collapse": {
+ "open": true
+ }
+ },
+ "items": [
+ {
+ "x": 0,
+ "y": 0,
+ "width": 12,
+ "height": 8,
+ "content": {
+ "$ref": "#/spec/panels/callRate"
+ }
+ },
+ {
+ "x": 12,
+ "y": 0,
+ "width": 12,
+ "height": 8,
+ "content": {
+ "$ref": "#/spec/panels/errorRatio"
+ }
+ },
+ {
+ "x": 0,
+ "y": 8,
+ "width": 12,
+ "height": 8,
+ "content": {
+ "$ref": "#/spec/panels/latencyMean"
+ }
+ },
+ {
+ "x": 12,
+ "y": 8,
+ "width": 12,
+ "height": 8,
+ "content": {
+ "$ref": "#/spec/panels/latencyMax"
+ }
+ }
+ ]
+ }
+ },
+ {
+ "kind": "Grid",
+ "spec": {
+ "display": {
+ "title": "Token Usage",
+ "collapse": {
+ "open": true
+ }
+ },
+ "items": [
+ {
+ "x": 0,
+ "y": 0,
+ "width": 12,
+ "height": 8,
+ "content": {
+ "$ref": "#/spec/panels/tokenThroughput"
+ }
+ },
+ {
+ "x": 12,
+ "y": 0,
+ "width": 12,
+ "height": 8,
+ "content": {
+ "$ref": "#/spec/panels/tokensPerCall"
+ }
+ }
+ ]
+ }
+ },
+ {
+ "kind": "Grid",
+ "spec": {
+ "display": {
+ "title": "About",
+ "collapse": {
+ "open": false
+ }
+ },
+ "items": [
+ {
+ "x": 0,
+ "y": 0,
+ "width": 24,
+ "height": 12,
+ "content": {
+ "$ref": "#/spec/panels/howTo"
+ }
+ }
+ ]
+ }
+ }
+ ]
+ }
+}
\ No newline at end of file
diff --git a/genai-observability/pom.xml b/genai-observability/pom.xml
index 1462f6b0..e1abf9d6 100644
--- a/genai-observability/pom.xml
+++ b/genai-observability/pom.xml
@@ -34,7 +34,7 @@
<properties>
<category>AI</category>
- <langchain4j-beta-version>1.19.0-beta29</langchain4j-beta-version>
+ <langchain4j-version>1.19.0</langchain4j-version>
<project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
<project.reporting.outputEncoding>UTF-8</project.reporting.outputEncoding>
</properties>
@@ -79,6 +79,10 @@
<groupId>org.apache.camel.springboot</groupId>
<artifactId>camel-langchain4j-chat-starter</artifactId>
</dependency>
+ <dependency>
+ <groupId>org.apache.camel.springboot</groupId>
+ <artifactId>camel-yaml-dsl-starter</artifactId>
+ </dependency>
<dependency>
<groupId>org.apache.camel</groupId>
<artifactId>camel-ai-observability</artifactId>
@@ -86,8 +90,23 @@
</dependency>
<dependency>
<groupId>dev.langchain4j</groupId>
- <artifactId>langchain4j-ollama-spring-boot-starter</artifactId>
- <version>${langchain4j-beta-version}</version>
+ <artifactId>langchain4j-ollama</artifactId>
+ <version>${langchain4j-version}</version>
+ </dependency>
+ <!-- Micrometer Tracing with the OpenTelemetry bridge: Spring Boot
auto-configures the
+ OpenTelemetry SDK, an OTLP span exporter and a tracing handler on
the
+ ObservationRegistry, so both Camel route spans and gen_ai.* spans
are exported -->
+ <dependency>
+ <groupId>org.springframework.boot</groupId>
+
<artifactId>spring-boot-micrometer-tracing-opentelemetry</artifactId>
+ </dependency>
+ <dependency>
+ <groupId>io.micrometer</groupId>
+ <artifactId>micrometer-tracing-bridge-otel</artifactId>
+ </dependency>
+ <dependency>
+ <groupId>io.opentelemetry</groupId>
+ <artifactId>opentelemetry-exporter-otlp</artifactId>
</dependency>
<dependency>
<groupId>org.springframework.boot</groupId>
diff --git a/genai-observability/prometheus.yml
b/genai-observability/prometheus.yml
deleted file mode 100644
index a0a9125a..00000000
--- a/genai-observability/prometheus.yml
+++ /dev/null
@@ -1,16 +0,0 @@
-#
-# Licensed to the Apache Software Foundation (ASF) under one or more
-# contributor license agreements. See the NOTICE file distributed with
-# this work for additional information regarding copyright ownership.
-# The ASF licenses this file to You under the Apache License, Version 2.0
-#
-
-global:
- scrape_interval: 15s
-
-scrape_configs:
- - job_name: camel-spring-boot
- metrics_path: /actuator/prometheus
- static_configs:
- - targets:
- - host.docker.internal:8080
diff --git
a/genai-observability/src/main/java/org/apache/camel/example/genai/ChatModelConfiguration.java
b/genai-observability/src/main/java/org/apache/camel/example/genai/ChatModelConfiguration.java
new file mode 100644
index 00000000..9ede0b8a
--- /dev/null
+++
b/genai-observability/src/main/java/org/apache/camel/example/genai/ChatModelConfiguration.java
@@ -0,0 +1,61 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.camel.example.genai;
+
+import java.time.Duration;
+
+import dev.langchain4j.model.chat.ChatModel;
+import dev.langchain4j.model.ollama.OllamaChatModel;
+import org.springframework.beans.factory.annotation.Value;
+import org.springframework.context.annotation.Bean;
+import org.springframework.context.annotation.Configuration;
+
+/**
+ * Two chat models against the same Ollama instance, so the GenAI dashboards
show per-model series
+ * (gen_ai.request.model is a tag/attribute on every metric and span).
+ */
+@Configuration
+public class ChatModelConfiguration {
+
+ @Value("${langchain4j.ollama.chat-model.base-url}")
+ private String baseUrl;
+
+ @Value("${langchain4j.ollama.chat-model.temperature}")
+ private Double temperature;
+
+ @Value("${langchain4j.ollama.chat-model.timeout}")
+ private Duration timeout;
+
+ @Bean
+ ChatModel
chatModel1(@Value("${langchain4j.ollama.chat-model-1.model-name}") String
modelName) {
+ return buildModel(modelName);
+ }
+
+ @Bean
+ ChatModel
chatModel2(@Value("${langchain4j.ollama.chat-model-2.model-name}") String
modelName) {
+ return buildModel(modelName);
+ }
+
+ private ChatModel buildModel(String modelName) {
+ return OllamaChatModel.builder()
+ .baseUrl(baseUrl)
+ .modelName(modelName)
+ .temperature(temperature)
+ .timeout(timeout)
+ .build();
+ }
+}
diff --git a/genai-observability/src/main/resources/application.properties
b/genai-observability/src/main/resources/application.properties
index 7571a31b..d4e9ef86 100644
--- a/genai-observability/src/main/resources/application.properties
+++ b/genai-observability/src/main/resources/application.properties
@@ -19,11 +19,13 @@
spring.application.name=genai-observability
server.port=8080
-# LangChain4j Ollama (auto-configures chatLanguageModel bean)
+# LangChain4j Ollama (used by ChatModelConfiguration to build the
chatModel1/chatModel2 beans).
+# Two small models so the GenAI dashboards show per-model series.
langchain4j.ollama.chat-model.base-url=http://localhost:11434
-langchain4j.ollama.chat-model.model-name=llama3.2
langchain4j.ollama.chat-model.temperature=0.2
langchain4j.ollama.chat-model.timeout=PT120S
+langchain4j.ollama.chat-model-1.model-name=llama3.2:1b
+langchain4j.ollama.chat-model-2.model-name=qwen3:0.6b
# Camel YAML routes
camel.main.routes-include-pattern=camel/*
@@ -37,7 +39,6 @@
management.endpoints.web.exposure.include=health,prometheus,info
management.endpoint.prometheus.access=read_only
management.prometheus.metrics.export.enabled=true
-# OTLP export to VictoriaTraces (docker compose stack)
-camel.opentelemetry2.export-target=jaeger
-otel.exporter.otlp.endpoint=http://localhost:9428/insert/opentelemetry/v1/traces
-otel.exporter.otlp.protocol=http/protobuf
+# OTLP/HTTP trace export to VictoriaTraces (camel infra run observability)
+management.opentelemetry.tracing.export.otlp.endpoint=http://localhost:10428/insert/opentelemetry/v1/traces
+management.tracing.sampling.probability=1.0
diff --git
a/genai-observability/src/main/resources/camel/genai-route.camel.yaml
b/genai-observability/src/main/resources/camel/genai-route.camel.yaml
index af7a4a8a..affe70ff 100644
--- a/genai-observability/src/main/resources/camel/genai-route.camel.yaml
+++ b/genai-observability/src/main/resources/camel/genai-route.camel.yaml
@@ -6,19 +6,33 @@
#
- route:
- id: genai-chat
+ id: genai-chat-1
from:
- uri: timer:genai
+ uri: timer:genai1
parameters:
period: "15000"
steps:
- setBody:
constant: "In one sentence, what is Apache Camel integration?"
- to:
- uri: langchain4j-chat:demo
+ uri: langchain4j-chat:model1
parameters:
- chatModel: "#chatLanguageModel"
+ chatModel: "#chatModel1"
- log:
- message: "LLM reply: ${body}"
+ message: "[${header.CamelLangChain4jChatResponseModel}] LLM reply:
${body}"
+
+- route:
+ id: genai-chat-2
+ from:
+ uri: timer:genai2
+ parameters:
+ period: "20000"
+ steps:
+ - setBody:
+ constant: "In one sentence, what is an Enterprise Integration
Pattern?"
+ - to:
+ uri: langchain4j-chat:model2
+ parameters:
+ chatModel: "#chatModel2"
- log:
- message: "Models: req=${header.CamelLangChain4jChatRequestModel}
resp=${header.CamelLangChain4jChatResponseModel}"
+ message: "[${header.CamelLangChain4jChatResponseModel}] LLM reply:
${body}"
