This is an automated email from the ASF dual-hosted git repository. Croway pushed a commit to branch genai-observability-camel-infra in repository https://gitbox.apache.org/repos/asf/camel-spring-boot-examples.git
commit 913f72b4046316d34daca04157dd594ac47913a3 Author: croway <[email protected]> AuthorDate: Mon Aug 31 10:45:26 2026 +0200 Rework genai-observability example: camel infra stack, two models, GenAI dashboard - Replace docker-compose with 'camel infra run observability' (Prometheus scrapes /observe/metrics on the 9876 management port, VictoriaTraces OTLP on 10428) - Fix startup on Spring Boot 4: drop the incompatible langchain4j Ollama starter (langchain4j/langchain4j#6236) and build the ChatModel beans from plain langchain4j-ollama; add the missing camel-yaml-dsl-starter - Export traces via Micrometer Tracing with the OpenTelemetry bridge, so route spans and gen_ai.* observation spans reach VictoriaTraces (CAMEL-24570) - Use two small models on two routes so all GenAI metrics/spans carry distinct gen_ai.request.model series - Add a Perses GenAI dashboard (perses-genai-dashboard.json) and document the panels with screenshots in the README Co-Authored-By: Claude Fable 5 <[email protected]> --- genai-observability/README.adoc | 106 +++- genai-observability/docker-compose.yml | 20 - genai-observability/docs/genai-dashboard-calls.png | Bin 0 -> 237089 bytes .../docs/genai-dashboard-tokens.png | Bin 0 -> 94387 bytes genai-observability/perses-genai-dashboard.json | 566 +++++++++++++++++++++ genai-observability/pom.xml | 25 +- genai-observability/prometheus.yml | 16 - .../example/genai/ChatModelConfiguration.java | 61 +++ .../src/main/resources/application.properties | 13 +- .../main/resources/camel/genai-route.camel.yaml | 26 +- 10 files changed, 773 insertions(+), 60 deletions(-) diff --git a/genai-observability/README.adoc b/genai-observability/README.adoc index 777f7342..f216e92b 100644 --- a/genai-observability/README.adoc +++ b/genai-observability/README.adoc @@ -4,7 +4,7 @@ This example demonstrates *GenAI observability* in Apache Camel 4.23+ with Spring Boot: OpenTelemetry `gen_ai.*` span attributes and Micrometer metrics for `langchain4j-chat` -LLM calls, visualized with Prometheus and VictoriaTraces. +LLM calls, visualized with Prometheus, VictoriaTraces and Perses. It pairs with the community blog post *GenAI Observability with Spring Boot and the Camel Observability Stack* (`docs/blog-drafts/genai-observability-02-spring-boot-obs-stack.adoc` in the @@ -14,12 +14,29 @@ The example uses `camel-ai-observability-starter` for Spring Boot configuration GenAI observability toggle (`camel.aiobservability.enabled`), together with `camel-ai-observability` for span and metric emission when a tracing or metrics backend is present. +Two routes call two different (small) Ollama models, so every GenAI metric and span carries a +distinct `gen_ai.request.model` and dashboards show per-model series side by side. + +The `camel-observability-services-starter` dependency configures an opinionated observability +setup: all Actuator endpoints move to a dedicated management port `9876` under the `/observe` +base path, with the Prometheus endpoint mapped to `/observe/metrics`. Both the standard +Spring Boot / Camel metrics and the `gen_ai.*` metrics are exposed there, since they share +the same Micrometer registry. + +Traces use the Spring Boot idiomatic setup: Micrometer Tracing with the OpenTelemetry bridge +(`spring-boot-micrometer-tracing-opentelemetry`, `micrometer-tracing-bridge-otel` and +`opentelemetry-exporter-otlp`). Spring Boot auto-configures the OpenTelemetry SDK, the OTLP +span exporter (`management.opentelemetry.tracing.export.otlp.endpoint`) and a tracing handler +on the Actuator `ObservationRegistry`. Camel route spans (via `camel-opentelemetry2`) and the +`gen_ai.*` client spans (recorded as Micrometer Observations) end up in the same trace. + === Prerequisites * Java 17+ * Maven 3.9+ -* Docker (for observability stack) -* https://ollama.com/[Ollama] with `llama3.2` pulled +* https://camel.apache.org/manual/camel-jbang.html[Camel CLI (JBang)] 4.22+ +* Docker (used by the Camel CLI to start the observability stack) +* https://ollama.com/[Ollama] with `llama3.2:1b` and `qwen3:0.6b` pulled == Build @@ -34,17 +51,21 @@ Terminal 1 — Ollama: [source,shell] ---- -ollama pull llama3.2 +ollama pull llama3.2:1b +ollama pull qwen3:0.6b ollama serve ---- -Terminal 2 — observability stack (Prometheus + VictoriaTraces + Perses): +Terminal 2 — observability stack (Prometheus + VictoriaTraces + VictoriaLogs + Perses): [source,shell] ---- -docker compose up -d +camel infra run observability ---- +The stack starts on fixed ports and the bundled Prometheus is already configured to scrape +the application metrics at `host.docker.internal:9876/observe/metrics`. + Terminal 3 — Spring Boot application: [source,shell] @@ -52,16 +73,83 @@ Terminal 3 — Spring Boot application: mvn spring-boot:run ---- +To use different models than the defaults: + +[source,shell] +---- +mvn spring-boot:run -Dspring-boot.run.arguments="--langchain4j.ollama.chat-model-1.model-name=llama3.2 --langchain4j.ollama.chat-model-2.model-name=qwen3:1.7b" +---- + == Verify GenAI observability +Metrics are exposed on the management port `9876` (not the application port `8080`): + [source,shell] ---- -curl -s http://localhost:8080/actuator/prometheus | grep gen_ai +curl -s http://localhost:9876/observe/metrics | grep gen_ai ---- * Prometheus: `http://localhost:9090` -* VictoriaTraces: `http://localhost:9428/select/vmui` -* Perses: `http://localhost:8088` +* VictoriaTraces: `http://localhost:10428/select/vmui` +* Perses: `http://localhost:3000` + +== Perses dashboards + +The observability stack provisions Perses with a general Camel dashboard (uptime, exchanges, +routes, JVM): + +* Camel overview: `http://localhost:3000/projects/camel/dashboards/overview` + +A GenAI dashboard (LLM calls, per-model latency, error ratio, token usage) can be created +through the Perses REST API using the dashboard definition in `perses-genai-dashboard.json`: + +[source,shell] +---- +curl -X POST http://localhost:3000/api/v1/projects \ + -H 'Content-Type: application/json' \ + -d '{"kind":"Project","metadata":{"name":"camel_genai"},"spec":{}}' + +curl -X POST http://localhost:3000/api/v1/projects/camel_genai/dashboards \ + -H 'Content-Type: application/json' \ + --data @perses-genai-dashboard.json +---- + +* GenAI overview: `http://localhost:3000/projects/camel_genai/dashboards/overview` + +The dashboard itself contains a "How this dashboard was created" section with the same +instructions. Note that Perses state lives in the container: after restarting the +observability stack, re-run the two commands above to recreate the GenAI dashboard. + +=== Reading the GenAI dashboard + +image::docs/genai-dashboard-calls.png[GenAI Summary and LLM Calls sections] + +* *GenAI Summary* — running totals across all models: LLM calls, errors, in-flight calls, + and input/output token counters. With calls that take longer than the timer period, + "In-flight Calls" sits permanently at 1: the route serializes requests, and a new one + starts as soon as the previous completes. +* *Call rate* — calls per second, one series per `gen_ai.request.model`. A fast model + settles at the timer frequency; a slow model's rate is capped by its own latency. +* *Error ratio* — failed calls (tagged `error!="none"`) over total, per model. Flat 0% + lines mean every call succeeded. +* *Mean / Max LLM latency* — per-model duration of the LLM call itself (the + `gen_ai.client.operation` timer, not the whole route). + +image::docs/genai-dashboard-tokens.png[Token Usage section] + +* *Token throughput* and *Avg tokens per call* — from the `gen_ai.client.token.usage` + counter, split by model and `gen_ai.token.type` (input/output). + +The screenshots were taken with `llama3.2` and `qwen3.5:0.8b` (via the model-name +overrides shown above); the default models behave similarly, since the `qwen3` family +also reasons before answering. The two models make the point of GenAI observability +visible on identical prompts: +`llama3.2` answers in about a second with a few dozen output tokens, while +`qwen3.5:0.8b` - a *thinking* model - emits roughly 2K output tokens per call (mostly +reasoning tokens before the one-sentence answer), which drives both its ~30-45s latency +and its dominant share of token throughput. Same workload, an order of magnitude more +token spend - exactly the kind of cost/latency trade-off these metrics are meant to +surface. == Help and contributions diff --git a/genai-observability/docker-compose.yml b/genai-observability/docker-compose.yml deleted file mode 100644 index 71034773..00000000 --- a/genai-observability/docker-compose.yml +++ /dev/null @@ -1,20 +0,0 @@ -# Observability stack for Camel GenAI observability blog (Blog 2) -# Images aligned with camel-test-infra-observability - -services: - prometheus: - image: quay.io/prometheus/prometheus:v3.13.2 - ports: - - "9090:9090" - volumes: - - ./prometheus.yml:/etc/prometheus/prometheus.yml:ro - - victoriatraces: - image: mirror.gcr.io/victoriametrics/victoria-traces:v0.10.0 - ports: - - "9428:9428" - - perses: - image: mirror.gcr.io/persesdev/perses:v0.54.0 - ports: - - "8088:8080" diff --git a/genai-observability/docs/genai-dashboard-calls.png b/genai-observability/docs/genai-dashboard-calls.png new file mode 100644 index 00000000..98569acb Binary files /dev/null and b/genai-observability/docs/genai-dashboard-calls.png differ diff --git a/genai-observability/docs/genai-dashboard-tokens.png b/genai-observability/docs/genai-dashboard-tokens.png new file mode 100644 index 00000000..89bc1cf6 Binary files /dev/null and b/genai-observability/docs/genai-dashboard-tokens.png differ diff --git a/genai-observability/perses-genai-dashboard.json b/genai-observability/perses-genai-dashboard.json new file mode 100644 index 00000000..79b1441f --- /dev/null +++ b/genai-observability/perses-genai-dashboard.json @@ -0,0 +1,566 @@ +{ + "kind": "Dashboard", + "metadata": { + "name": "overview" + }, + "spec": { + "display": { + "name": "GenAI Overview" + }, + "duration": "30m", + "panels": { + "llmCalls": { + "kind": "Panel", + "spec": { + "display": { + "name": "LLM Calls" + }, + "plugin": { + "kind": "StatChart", + "spec": { + "calculation": "last-number", + "format": { + "unit": "decimal", + "shortValues": true + }, + "sparkline": {} + } + }, + "queries": [ + { + "kind": "TimeSeriesQuery", + "spec": { + "plugin": { + "kind": "PrometheusTimeSeriesQuery", + "spec": { + "query": "sum(gen_ai_client_operation_seconds_count)" + } + } + } + } + ] + } + }, + "llmErrors": { + "kind": "Panel", + "spec": { + "display": { + "name": "LLM Errors" + }, + "plugin": { + "kind": "StatChart", + "spec": { + "calculation": "last-number", + "format": { + "unit": "decimal", + "shortValues": true + }, + "sparkline": {} + } + }, + "queries": [ + { + "kind": "TimeSeriesQuery", + "spec": { + "plugin": { + "kind": "PrometheusTimeSeriesQuery", + "spec": { + "query": "sum(gen_ai_client_operation_seconds_count{error!=\"none\"}) or vector(0)" + } + } + } + } + ] + } + }, + "inflightCalls": { + "kind": "Panel", + "spec": { + "display": { + "name": "In-flight Calls" + }, + "plugin": { + "kind": "StatChart", + "spec": { + "calculation": "last-number", + "format": { + "unit": "decimal", + "shortValues": true + }, + "sparkline": {} + } + }, + "queries": [ + { + "kind": "TimeSeriesQuery", + "spec": { + "plugin": { + "kind": "PrometheusTimeSeriesQuery", + "spec": { + "query": "sum(gen_ai_client_operation_active_seconds_count)" + } + } + } + } + ] + } + }, + "inputTokens": { + "kind": "Panel", + "spec": { + "display": { + "name": "Input Tokens" + }, + "plugin": { + "kind": "StatChart", + "spec": { + "calculation": "last-number", + "format": { + "unit": "decimal", + "shortValues": true + }, + "sparkline": {} + } + }, + "queries": [ + { + "kind": "TimeSeriesQuery", + "spec": { + "plugin": { + "kind": "PrometheusTimeSeriesQuery", + "spec": { + "query": "sum(gen_ai_client_token_usage_total{gen_ai_token_type=\"input\"})" + } + } + } + } + ] + } + }, + "outputTokens": { + "kind": "Panel", + "spec": { + "display": { + "name": "Output Tokens" + }, + "plugin": { + "kind": "StatChart", + "spec": { + "calculation": "last-number", + "format": { + "unit": "decimal", + "shortValues": true + }, + "sparkline": {} + } + }, + "queries": [ + { + "kind": "TimeSeriesQuery", + "spec": { + "plugin": { + "kind": "PrometheusTimeSeriesQuery", + "spec": { + "query": "sum(gen_ai_client_token_usage_total{gen_ai_token_type=\"output\"})" + } + } + } + } + ] + } + }, + "callRate": { + "kind": "Panel", + "spec": { + "display": { + "name": "Call rate (calls/s)" + }, + "plugin": { + "kind": "TimeSeriesChart", + "spec": { + "legend": { + "position": "bottom" + }, + "yAxis": { + "format": { + "unit": "decimal", + "decimalPlaces": 2 + } + } + } + }, + "queries": [ + { + "kind": "TimeSeriesQuery", + "spec": { + "plugin": { + "kind": "PrometheusTimeSeriesQuery", + "spec": { + "query": "sum by (gen_ai_system, gen_ai_request_model) (rate(gen_ai_client_operation_seconds_count[1m]))", + "seriesNameFormat": "{{gen_ai_system}} {{gen_ai_request_model}}" + } + } + } + } + ] + } + }, + "errorRatio": { + "kind": "Panel", + "spec": { + "display": { + "name": "Error ratio" + }, + "plugin": { + "kind": "TimeSeriesChart", + "spec": { + "legend": { + "position": "bottom" + }, + "yAxis": { + "format": { + "unit": "percent-decimal", + "decimalPlaces": 2 + } + } + } + }, + "queries": [ + { + "kind": "TimeSeriesQuery", + "spec": { + "plugin": { + "kind": "PrometheusTimeSeriesQuery", + "spec": { + "query": "sum by (gen_ai_request_model) ((rate(gen_ai_client_operation_seconds_count{error!=\"none\"}[5m]) or rate(gen_ai_client_operation_seconds_count[5m]) * 0)) / on(gen_ai_request_model) group_left sum by (gen_ai_request_model) (rate(gen_ai_client_operation_seconds_count[5m]) > 0)", + "seriesNameFormat": "{{gen_ai_request_model}}" + } + } + } + } + ] + } + }, + "latencyMean": { + "kind": "Panel", + "spec": { + "display": { + "name": "Mean LLM latency" + }, + "plugin": { + "kind": "TimeSeriesChart", + "spec": { + "legend": { + "position": "bottom" + }, + "yAxis": { + "format": { + "unit": "seconds", + "decimalPlaces": 2 + } + } + } + }, + "queries": [ + { + "kind": "TimeSeriesQuery", + "spec": { + "plugin": { + "kind": "PrometheusTimeSeriesQuery", + "spec": { + "query": "sum by (gen_ai_system, gen_ai_request_model) (rate(gen_ai_client_operation_seconds_sum[5m])) / sum by (gen_ai_system, gen_ai_request_model) (rate(gen_ai_client_operation_seconds_count[5m]))", + "seriesNameFormat": "{{gen_ai_system}} {{gen_ai_request_model}}" + } + } + } + } + ] + } + }, + "latencyMax": { + "kind": "Panel", + "spec": { + "display": { + "name": "Max LLM latency" + }, + "plugin": { + "kind": "TimeSeriesChart", + "spec": { + "legend": { + "position": "bottom" + }, + "yAxis": { + "format": { + "unit": "seconds", + "decimalPlaces": 2 + } + } + } + }, + "queries": [ + { + "kind": "TimeSeriesQuery", + "spec": { + "plugin": { + "kind": "PrometheusTimeSeriesQuery", + "spec": { + "query": "gen_ai_client_operation_seconds_max", + "seriesNameFormat": "{{gen_ai_system}} {{gen_ai_request_model}}" + } + } + } + } + ] + } + }, + "tokenThroughput": { + "kind": "Panel", + "spec": { + "display": { + "name": "Token throughput (tokens/s)" + }, + "plugin": { + "kind": "TimeSeriesChart", + "spec": { + "legend": { + "position": "bottom" + }, + "yAxis": { + "format": { + "unit": "decimal", + "decimalPlaces": 2 + } + } + } + }, + "queries": [ + { + "kind": "TimeSeriesQuery", + "spec": { + "plugin": { + "kind": "PrometheusTimeSeriesQuery", + "spec": { + "query": "sum by (gen_ai_request_model, gen_ai_token_type) (rate(gen_ai_client_token_usage_total[1m]))", + "seriesNameFormat": "{{gen_ai_request_model}} {{gen_ai_token_type}}" + } + } + } + } + ] + } + }, + "tokensPerCall": { + "kind": "Panel", + "spec": { + "display": { + "name": "Avg tokens per call" + }, + "plugin": { + "kind": "TimeSeriesChart", + "spec": { + "legend": { + "position": "bottom" + }, + "yAxis": { + "format": { + "unit": "decimal", + "decimalPlaces": 2 + } + } + } + }, + "queries": [ + { + "kind": "TimeSeriesQuery", + "spec": { + "plugin": { + "kind": "PrometheusTimeSeriesQuery", + "spec": { + "query": "sum by (gen_ai_request_model, gen_ai_token_type) (rate(gen_ai_client_token_usage_total[5m])) / on(gen_ai_request_model) group_left sum by (gen_ai_request_model) (rate(gen_ai_client_operation_seconds_count[5m]))", + "seriesNameFormat": "{{gen_ai_request_model}} {{gen_ai_token_type}}" + } + } + } + } + ] + } + }, + "howTo": { + "kind": "Panel", + "spec": { + "display": { + "name": "How this dashboard was created" + }, + "plugin": { + "kind": "Markdown", + "spec": { + "text": "## How this dashboard was created\n\nMetrics come from **camel-ai-observability**: the Spring Boot app exposes `gen_ai.*` meters on\n`http://localhost:9876/observe/metrics` and the stack's Prometheus scrapes that endpoint.\n\nThe dashboard is plain JSON managed through the Perses REST API. To recreate it (e.g. after\nrestarting `camel infra run observability`), from the `genai-observability` example directory:\n\n```\ncurl -X POST http://localhost:3000/api/v1/proje [...] + } + } + } + } + }, + "layouts": [ + { + "kind": "Grid", + "spec": { + "display": { + "title": "GenAI Summary", + "collapse": { + "open": true + } + }, + "items": [ + { + "x": 0, + "y": 0, + "width": 4, + "height": 4, + "content": { + "$ref": "#/spec/panels/llmCalls" + } + }, + { + "x": 4, + "y": 0, + "width": 4, + "height": 4, + "content": { + "$ref": "#/spec/panels/llmErrors" + } + }, + { + "x": 8, + "y": 0, + "width": 4, + "height": 4, + "content": { + "$ref": "#/spec/panels/inflightCalls" + } + }, + { + "x": 12, + "y": 0, + "width": 4, + "height": 4, + "content": { + "$ref": "#/spec/panels/inputTokens" + } + }, + { + "x": 16, + "y": 0, + "width": 4, + "height": 4, + "content": { + "$ref": "#/spec/panels/outputTokens" + } + } + ] + } + }, + { + "kind": "Grid", + "spec": { + "display": { + "title": "LLM Calls", + "collapse": { + "open": true + } + }, + "items": [ + { + "x": 0, + "y": 0, + "width": 12, + "height": 8, + "content": { + "$ref": "#/spec/panels/callRate" + } + }, + { + "x": 12, + "y": 0, + "width": 12, + "height": 8, + "content": { + "$ref": "#/spec/panels/errorRatio" + } + }, + { + "x": 0, + "y": 8, + "width": 12, + "height": 8, + "content": { + "$ref": "#/spec/panels/latencyMean" + } + }, + { + "x": 12, + "y": 8, + "width": 12, + "height": 8, + "content": { + "$ref": "#/spec/panels/latencyMax" + } + } + ] + } + }, + { + "kind": "Grid", + "spec": { + "display": { + "title": "Token Usage", + "collapse": { + "open": true + } + }, + "items": [ + { + "x": 0, + "y": 0, + "width": 12, + "height": 8, + "content": { + "$ref": "#/spec/panels/tokenThroughput" + } + }, + { + "x": 12, + "y": 0, + "width": 12, + "height": 8, + "content": { + "$ref": "#/spec/panels/tokensPerCall" + } + } + ] + } + }, + { + "kind": "Grid", + "spec": { + "display": { + "title": "About", + "collapse": { + "open": false + } + }, + "items": [ + { + "x": 0, + "y": 0, + "width": 24, + "height": 12, + "content": { + "$ref": "#/spec/panels/howTo" + } + } + ] + } + } + ] + } +} \ No newline at end of file diff --git a/genai-observability/pom.xml b/genai-observability/pom.xml index 1462f6b0..e1abf9d6 100644 --- a/genai-observability/pom.xml +++ b/genai-observability/pom.xml @@ -34,7 +34,7 @@ <properties> <category>AI</category> - <langchain4j-beta-version>1.19.0-beta29</langchain4j-beta-version> + <langchain4j-version>1.19.0</langchain4j-version> <project.build.sourceEncoding>UTF-8</project.build.sourceEncoding> <project.reporting.outputEncoding>UTF-8</project.reporting.outputEncoding> </properties> @@ -79,6 +79,10 @@ <groupId>org.apache.camel.springboot</groupId> <artifactId>camel-langchain4j-chat-starter</artifactId> </dependency> + <dependency> + <groupId>org.apache.camel.springboot</groupId> + <artifactId>camel-yaml-dsl-starter</artifactId> + </dependency> <dependency> <groupId>org.apache.camel</groupId> <artifactId>camel-ai-observability</artifactId> @@ -86,8 +90,23 @@ </dependency> <dependency> <groupId>dev.langchain4j</groupId> - <artifactId>langchain4j-ollama-spring-boot-starter</artifactId> - <version>${langchain4j-beta-version}</version> + <artifactId>langchain4j-ollama</artifactId> + <version>${langchain4j-version}</version> + </dependency> + <!-- Micrometer Tracing with the OpenTelemetry bridge: Spring Boot auto-configures the + OpenTelemetry SDK, an OTLP span exporter and a tracing handler on the + ObservationRegistry, so both Camel route spans and gen_ai.* spans are exported --> + <dependency> + <groupId>org.springframework.boot</groupId> + <artifactId>spring-boot-micrometer-tracing-opentelemetry</artifactId> + </dependency> + <dependency> + <groupId>io.micrometer</groupId> + <artifactId>micrometer-tracing-bridge-otel</artifactId> + </dependency> + <dependency> + <groupId>io.opentelemetry</groupId> + <artifactId>opentelemetry-exporter-otlp</artifactId> </dependency> <dependency> <groupId>org.springframework.boot</groupId> diff --git a/genai-observability/prometheus.yml b/genai-observability/prometheus.yml deleted file mode 100644 index a0a9125a..00000000 --- a/genai-observability/prometheus.yml +++ /dev/null @@ -1,16 +0,0 @@ -# -# Licensed to the Apache Software Foundation (ASF) under one or more -# contributor license agreements. See the NOTICE file distributed with -# this work for additional information regarding copyright ownership. -# The ASF licenses this file to You under the Apache License, Version 2.0 -# - -global: - scrape_interval: 15s - -scrape_configs: - - job_name: camel-spring-boot - metrics_path: /actuator/prometheus - static_configs: - - targets: - - host.docker.internal:8080 diff --git a/genai-observability/src/main/java/org/apache/camel/example/genai/ChatModelConfiguration.java b/genai-observability/src/main/java/org/apache/camel/example/genai/ChatModelConfiguration.java new file mode 100644 index 00000000..9ede0b8a --- /dev/null +++ b/genai-observability/src/main/java/org/apache/camel/example/genai/ChatModelConfiguration.java @@ -0,0 +1,61 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one or more + * contributor license agreements. See the NOTICE file distributed with + * this work for additional information regarding copyright ownership. + * The ASF licenses this file to You under the Apache License, Version 2.0 + * (the "License"); you may not use this file except in compliance with + * the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.apache.camel.example.genai; + +import java.time.Duration; + +import dev.langchain4j.model.chat.ChatModel; +import dev.langchain4j.model.ollama.OllamaChatModel; +import org.springframework.beans.factory.annotation.Value; +import org.springframework.context.annotation.Bean; +import org.springframework.context.annotation.Configuration; + +/** + * Two chat models against the same Ollama instance, so the GenAI dashboards show per-model series + * (gen_ai.request.model is a tag/attribute on every metric and span). + */ +@Configuration +public class ChatModelConfiguration { + + @Value("${langchain4j.ollama.chat-model.base-url}") + private String baseUrl; + + @Value("${langchain4j.ollama.chat-model.temperature}") + private Double temperature; + + @Value("${langchain4j.ollama.chat-model.timeout}") + private Duration timeout; + + @Bean + ChatModel chatModel1(@Value("${langchain4j.ollama.chat-model-1.model-name}") String modelName) { + return buildModel(modelName); + } + + @Bean + ChatModel chatModel2(@Value("${langchain4j.ollama.chat-model-2.model-name}") String modelName) { + return buildModel(modelName); + } + + private ChatModel buildModel(String modelName) { + return OllamaChatModel.builder() + .baseUrl(baseUrl) + .modelName(modelName) + .temperature(temperature) + .timeout(timeout) + .build(); + } +} diff --git a/genai-observability/src/main/resources/application.properties b/genai-observability/src/main/resources/application.properties index 7571a31b..d4e9ef86 100644 --- a/genai-observability/src/main/resources/application.properties +++ b/genai-observability/src/main/resources/application.properties @@ -19,11 +19,13 @@ spring.application.name=genai-observability server.port=8080 -# LangChain4j Ollama (auto-configures chatLanguageModel bean) +# LangChain4j Ollama (used by ChatModelConfiguration to build the chatModel1/chatModel2 beans). +# Two small models so the GenAI dashboards show per-model series. langchain4j.ollama.chat-model.base-url=http://localhost:11434 -langchain4j.ollama.chat-model.model-name=llama3.2 langchain4j.ollama.chat-model.temperature=0.2 langchain4j.ollama.chat-model.timeout=PT120S +langchain4j.ollama.chat-model-1.model-name=llama3.2:1b +langchain4j.ollama.chat-model-2.model-name=qwen3:0.6b # Camel YAML routes camel.main.routes-include-pattern=camel/* @@ -37,7 +39,6 @@ management.endpoints.web.exposure.include=health,prometheus,info management.endpoint.prometheus.access=read_only management.prometheus.metrics.export.enabled=true -# OTLP export to VictoriaTraces (docker compose stack) -camel.opentelemetry2.export-target=jaeger -otel.exporter.otlp.endpoint=http://localhost:9428/insert/opentelemetry/v1/traces -otel.exporter.otlp.protocol=http/protobuf +# OTLP/HTTP trace export to VictoriaTraces (camel infra run observability) +management.opentelemetry.tracing.export.otlp.endpoint=http://localhost:10428/insert/opentelemetry/v1/traces +management.tracing.sampling.probability=1.0 diff --git a/genai-observability/src/main/resources/camel/genai-route.camel.yaml b/genai-observability/src/main/resources/camel/genai-route.camel.yaml index af7a4a8a..affe70ff 100644 --- a/genai-observability/src/main/resources/camel/genai-route.camel.yaml +++ b/genai-observability/src/main/resources/camel/genai-route.camel.yaml @@ -6,19 +6,33 @@ # - route: - id: genai-chat + id: genai-chat-1 from: - uri: timer:genai + uri: timer:genai1 parameters: period: "15000" steps: - setBody: constant: "In one sentence, what is Apache Camel integration?" - to: - uri: langchain4j-chat:demo + uri: langchain4j-chat:model1 parameters: - chatModel: "#chatLanguageModel" + chatModel: "#chatModel1" - log: - message: "LLM reply: ${body}" + message: "[${header.CamelLangChain4jChatResponseModel}] LLM reply: ${body}" + +- route: + id: genai-chat-2 + from: + uri: timer:genai2 + parameters: + period: "20000" + steps: + - setBody: + constant: "In one sentence, what is an Enterprise Integration Pattern?" + - to: + uri: langchain4j-chat:model2 + parameters: + chatModel: "#chatModel2" - log: - message: "Models: req=${header.CamelLangChain4jChatRequestModel} resp=${header.CamelLangChain4jChatResponseModel}" + message: "[${header.CamelLangChain4jChatResponseModel}] LLM reply: ${body}"
