This is an automated email from the ASF dual-hosted git repository.
davsclaus pushed a commit to branch main
in repository https://gitbox.apache.org/repos/asf/camel.git
The following commit(s) were added to refs/heads/main by this push:
new 757d4edca435 CAMEL-27142: camel-openai - record the detailed and full
GenAI data for the responses operation (#30326)
757d4edca435 is described below
commit 757d4edca4355b42198d0baf5fea107c3ab8e3c9
Author: Guillaume Nodet <[email protected]>
AuthorDate: Fri Nov 20 08:32:49 2026 +0100
CAMEL-27142: camel-openai - record the detailed and full GenAI data for the
responses operation (#30326)
Mirror the observability enhancements from CAMEL-27131 (OpenAIProducer) to
OpenAIResponsesProducer:
- processObservedToolLoop(): wraps the tool loop in an invoke_agent span at
level detailed or higher, named after the route id; model calls and tool
calls
started inside nest under it, giving tracing backends one agent run per
turn.
- observedCall(): records conversation id, request parameters (temperature,
top_p, max_tokens), tracedByFramework flag, cached input tokens, input
messages
(level full), system instructions (level full), and tool definitions
(level full)
on each per-model-call chat span.
- OpenAIResponsesGenAiMessages: converts ResponseCreateParams input items
and
Response output items to GenAiMessages/GenAiToolDefinitions for the full
level.
- OpenAIResponsesObservabilityLevelTest: verifies that the tool loop
produces
one invoke_agent span with two nested chat spans and an execute_tool
span, with
input messages and tool definitions recorded at level full.
Follow-ups in the PR:
- fix compilation errors: pass conversationId explicitly, fix the
cachedTokens accessor
- toolDefinitions parameters, an assistant message before a function call,
image placeholders, and the tracedByFramework check of the responses service
- regenerate the catalog copy of ai-observability.adoc
- the detailed-level test uses the route endpoint and passes
temperature/maxTokens via headers
Co-authored-by: Claude Sonnet 4.6 <[email protected]>
---
.../camel/catalog/docs/ai-observability.adoc | 4 +-
.../src/main/docs/ai-observability.adoc | 4 +-
.../openai/OpenAIResponsesGenAiMessages.java | 173 +++++++++++++++
.../component/openai/OpenAIResponsesProducer.java | 103 +++++++--
.../OpenAIResponsesObservabilityLevelTest.java | 242 +++++++++++++++++++++
5 files changed, 501 insertions(+), 25 deletions(-)
diff --git
a/catalog/camel-catalog/src/generated/resources/org/apache/camel/catalog/docs/ai-observability.adoc
b/catalog/camel-catalog/src/generated/resources/org/apache/camel/catalog/docs/ai-observability.adoc
index 82d5cec29d35..1d926f5b1718 100644
---
a/catalog/camel-catalog/src/generated/resources/org/apache/camel/catalog/docs/ai-observability.adoc
+++
b/catalog/camel-catalog/src/generated/resources/org/apache/camel/catalog/docs/ai-observability.adoc
@@ -115,7 +115,9 @@ the agent run and the Camel route tools are still recorded
by Camel. The framewo
that is current when the endpoint is called, which is the route span: they
appear next to the agent run, not under it,
because Camel makes its spans current only while a processor runs.
* `openai`: the `openai-java` instrumentation of OpenTelemetry (in the
OpenTelemetry Java agent) wraps the OpenAI client
-used by `camel-openai` and traces its chat completions and embeddings. When
the client is wrapped, those calls are left
+used by `camel-openai` and traces its chat completions and embeddings. The
`responses()` operation is not yet
+instrumented by the OpenTelemetry Java agent (as of 2.33.x), so Camel always
records its own span for Responses API
+calls. When the client is wrapped, chat completion and embeddings calls are
left
to it, except at level `full`: the instrumentation records the messages as log
events, not as span attributes, so Camel
keeps its own span and logs a warning. Start the JVM with
`-Dotel.instrumentation.openai-java.enabled=false` to keep
only the Camel spans.
diff --git
a/components/camel-ai/camel-ai-observability/src/main/docs/ai-observability.adoc
b/components/camel-ai/camel-ai-observability/src/main/docs/ai-observability.adoc
index 82d5cec29d35..1d926f5b1718 100644
---
a/components/camel-ai/camel-ai-observability/src/main/docs/ai-observability.adoc
+++
b/components/camel-ai/camel-ai-observability/src/main/docs/ai-observability.adoc
@@ -115,7 +115,9 @@ the agent run and the Camel route tools are still recorded
by Camel. The framewo
that is current when the endpoint is called, which is the route span: they
appear next to the agent run, not under it,
because Camel makes its spans current only while a processor runs.
* `openai`: the `openai-java` instrumentation of OpenTelemetry (in the
OpenTelemetry Java agent) wraps the OpenAI client
-used by `camel-openai` and traces its chat completions and embeddings. When
the client is wrapped, those calls are left
+used by `camel-openai` and traces its chat completions and embeddings. The
`responses()` operation is not yet
+instrumented by the OpenTelemetry Java agent (as of 2.33.x), so Camel always
records its own span for Responses API
+calls. When the client is wrapped, chat completion and embeddings calls are
left
to it, except at level `full`: the instrumentation records the messages as log
events, not as span attributes, so Camel
keeps its own span and logs a warning. Start the JVM with
`-Dotel.instrumentation.openai-java.enabled=false` to keep
only the Camel spans.
diff --git
a/components/camel-ai/camel-openai/src/main/java/org/apache/camel/component/openai/OpenAIResponsesGenAiMessages.java
b/components/camel-ai/camel-openai/src/main/java/org/apache/camel/component/openai/OpenAIResponsesGenAiMessages.java
new file mode 100644
index 000000000000..1da5fcd8afd7
--- /dev/null
+++
b/components/camel-ai/camel-openai/src/main/java/org/apache/camel/component/openai/OpenAIResponsesGenAiMessages.java
@@ -0,0 +1,173 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.camel.component.openai;
+
+import java.util.List;
+import java.util.Locale;
+import java.util.Map;
+import java.util.Objects;
+import java.util.stream.Collectors;
+
+import com.openai.core.ObjectMappers;
+import com.openai.models.responses.EasyInputMessage;
+import com.openai.models.responses.Response;
+import com.openai.models.responses.ResponseCreateParams;
+import com.openai.models.responses.ResponseFunctionToolCall;
+import com.openai.models.responses.ResponseInputContent;
+import com.openai.models.responses.ResponseInputItem;
+import com.openai.models.responses.ResponseOutputItem;
+import com.openai.models.responses.ResponseOutputMessage;
+import com.openai.models.responses.Tool;
+import org.apache.camel.component.ai.observability.GenAiMessages;
+import org.apache.camel.component.ai.observability.GenAiToolDefinitions;
+
+/**
+ * Converts Responses API requests and responses into the messages of the
GenAI semantic conventions, for observability
+ * level {@code full}.
+ */
+final class OpenAIResponsesGenAiMessages {
+
+ private OpenAIResponsesGenAiMessages() {
+ }
+
+ static GenAiMessages input(ResponseCreateParams params) {
+ GenAiMessages messages = new GenAiMessages();
+ params.input().ifPresent(input -> {
+ if (input.isText()) {
+ messages.text(GenAiMessages.USER, input.asText());
+ } else if (input.isResponse()) {
+ addInputItems(messages, input.asResponse());
+ }
+ });
+ return messages;
+ }
+
+ static GenAiMessages output(Response response, String finishReason) {
+ GenAiMessages messages = new GenAiMessages();
+ for (ResponseOutputItem item : response.output()) {
+ if (item.isMessage()) {
+ String text = item.asMessage().content().stream()
+ .filter(ResponseOutputMessage.Content::isOutputText)
+ .map(c -> c.asOutputText().text())
+ .collect(Collectors.joining("\n"));
+ messages.text(GenAiMessages.ASSISTANT, text.isEmpty() ? null :
text);
+ } else if (item.isFunctionCall()) {
+ ResponseFunctionToolCall call = item.asFunctionCall();
+ messages.toolCall(call.callId(), call.name(),
call.arguments());
+ }
+ }
+ return messages.finishReason(finishReason);
+ }
+
+ @SuppressWarnings("unchecked")
+ static GenAiToolDefinitions toolDefinitions(ResponseCreateParams params) {
+ GenAiToolDefinitions definitions = new GenAiToolDefinitions();
+ for (Tool tool : params.tools().orElse(List.of())) {
+ tool.function().ifPresent(function -> definitions.function(
+ function.name(),
+ function.description().orElse(null),
+ function.parameters()
+ .map(p ->
ObjectMappers.jsonMapper().convertValue(p, Map.class))
+ .orElse(null)));
+ }
+ return definitions;
+ }
+
+ private static void addInputItems(GenAiMessages messages,
List<ResponseInputItem> items) {
+ boolean lastIsAssistant = false;
+ for (ResponseInputItem item : items) {
+ if (item.isEasyInputMessage()) {
+ EasyInputMessage msg = item.asEasyInputMessage();
+ String role = roleOf(msg.role().asString());
+ lastIsAssistant = GenAiMessages.ASSISTANT.equals(role);
+ addEasyInputMessage(messages, role, msg.content());
+ } else if (item.isMessage()) {
+ // ResponseInputItem.Message - image/PDF prompt items from
OpenAIResponsesInputBuilder
+ ResponseInputItem.Message msg = item.asMessage();
+ String role = roleOf(msg.role().asString());
+ lastIsAssistant = GenAiMessages.ASSISTANT.equals(role);
+ String text = msg.content().stream()
+ .filter(ResponseInputContent::isInputText)
+ .map(c -> c.asInputText().text())
+ .collect(Collectors.joining("\n"));
+ if (text.isEmpty()) {
+ // binary content (image/PDF) - record placeholder
+ messages.text(role, null);
+ } else {
+ messages.text(role, text);
+ }
+ } else if (item.isResponseOutputMessage()) {
+ // assistant preamble from
OpenAIResponsesSupport.toInputItems()
+ ResponseOutputMessage msg = item.asResponseOutputMessage();
+ String text = msg.content().stream()
+ .filter(ResponseOutputMessage.Content::isOutputText)
+ .map(c -> c.asOutputText().text())
+ .collect(Collectors.joining("\n"));
+ lastIsAssistant = true;
+ messages.text(GenAiMessages.ASSISTANT, text.isEmpty() ? null :
text);
+ } else if (item.isFunctionCallOutput()) {
+ lastIsAssistant = false;
+ ResponseInputItem.FunctionCallOutput output =
item.asFunctionCallOutput();
+ String callId = output.callId().orElse(null);
+ String content = output.output().string().orElse(null);
+ messages.toolResult(callId, content);
+ } else if (item.isFunctionCall()) {
+ // Ensure tool_call lands in an assistant message per semconv
+ if (!lastIsAssistant) {
+ messages.text(GenAiMessages.ASSISTANT, null);
+ }
+ lastIsAssistant = true;
+ ResponseFunctionToolCall call = item.asFunctionCall();
+ messages.toolCall(call.callId(), call.name(),
call.arguments());
+ }
+ }
+ }
+
+ private static void addEasyInputMessage(GenAiMessages messages, String
role, EasyInputMessage.Content content) {
+ if (content.isTextInput()) {
+ messages.text(role, content.asTextInput());
+ } else if (content.isResponseInputMessageContentList()) {
+ String text = content.asResponseInputMessageContentList().stream()
+ .map(c -> {
+ if (c.isInputText()) {
+ return c.asInputText().text();
+ }
+ if (c.isInputImage()) {
+ return "[image]";
+ }
+ if (c.isInputFile()) {
+ return "[file]";
+ }
+ return null;
+ })
+ .filter(Objects::nonNull)
+ .collect(Collectors.joining("\n"));
+ messages.text(role, text.isEmpty() ? null : text);
+ }
+ }
+
+ private static String roleOf(String apiRole) {
+ if (apiRole == null) {
+ return GenAiMessages.USER;
+ }
+ return switch (apiRole.toLowerCase(Locale.ROOT)) {
+ case "assistant" -> GenAiMessages.ASSISTANT;
+ case "system", "developer" -> GenAiMessages.SYSTEM;
+ default -> GenAiMessages.USER;
+ };
+ }
+}
diff --git
a/components/camel-ai/camel-openai/src/main/java/org/apache/camel/component/openai/OpenAIResponsesProducer.java
b/components/camel-ai/camel-openai/src/main/java/org/apache/camel/component/openai/OpenAIResponsesProducer.java
index 84583603f964..0cb74e146ba4 100644
---
a/components/camel-ai/camel-openai/src/main/java/org/apache/camel/component/openai/OpenAIResponsesProducer.java
+++
b/components/camel-ai/camel-openai/src/main/java/org/apache/camel/component/openai/OpenAIResponsesProducer.java
@@ -32,13 +32,16 @@ import org.apache.camel.AsyncCallback;
import org.apache.camel.Exchange;
import org.apache.camel.Message;
import org.apache.camel.component.ai.observability.GenAiErrorSupport;
+import org.apache.camel.component.ai.observability.GenAiMessages;
import org.apache.camel.component.ai.observability.GenAiObservability;
+import org.apache.camel.component.ai.observability.GenAiObservabilityLevel;
import org.apache.camel.component.ai.observability.GenAiObservation;
import org.apache.camel.component.ai.observability.GenAiObservationContext;
import org.apache.camel.component.ai.observability.GenAiOperationName;
import org.apache.camel.component.ai.observability.GenAiUsage;
import org.apache.camel.component.ai.tool.AiToolSearchSession;
import org.apache.camel.support.DefaultAsyncProducer;
+import org.apache.camel.support.ExchangeHelper;
import org.apache.camel.support.ResourceHelper;
import org.apache.camel.support.service.ServiceHelper;
import org.apache.camel.util.ObjectHelper;
@@ -169,7 +172,8 @@ public class OpenAIResponsesProducer extends
DefaultAsyncProducer {
}
String conversationId = resolveParameter(in,
OpenAIConstants.CONVERSATION_ID, config.getConversationId(),
String.class);
- if (ObjectHelper.isNotEmpty(conversationId)) {
+ conversationId = ObjectHelper.isNotEmpty(conversationId) ?
conversationId : null;
+ if (conversationId != null) {
paramsBuilder.conversation(conversationId);
}
if (config.isBackground()) {
@@ -226,10 +230,15 @@ public class OpenAIResponsesProducer extends
DefaultAsyncProducer {
"background cannot be combined with automatic tool
execution, the tool loop needs each answer. "
+ "Set
autoToolExecution=false to handle the function calls in the route");
}
- processToolLoop(exchange, config, params, inputSpec.items(),
model,
- ObjectHelper.isNotEmpty(conversationId) ?
conversationId : null);
+ if (GenAiObservability.isLevel(exchange.getContext(),
GenAiObservabilityLevel.DETAILED)) {
+ processObservedToolLoop(exchange, config, params,
inputSpec.items(), model, conversationId);
+ } else {
+ processToolLoop(exchange, config, params,
inputSpec.items(), model,
+ conversationId);
+ }
} else {
- Response response = createResponse(exchange, model, params);
+ Response response = createResponse(exchange, model, params,
+ conversationId);
List<ResponseFunctionToolCall> functionCalls =
OpenAIResponsesSupport.extractFunctionCalls(response);
// with autoToolExecution=false the route handles the function
calls requested by the model
Object body = functionCalls.isEmpty() ?
OpenAIResponsesSupport.extractAssistantText(response) : functionCalls;
@@ -247,6 +256,40 @@ public class OpenAIResponsesProducer extends
DefaultAsyncProducer {
}
}
+ /**
+ * Wraps the tool loop in an {@code invoke_agent} span at observability
level {@code detailed} or higher. The model
+ * calls and tool calls started inside nest under it, giving tracing
backends one agent run per user turn.
+ */
+ private void processObservedToolLoop(
+ Exchange exchange, OpenAIConfiguration config,
ResponseCreateParams params, List<ResponseInputItem> input,
+ String model, String conversationId)
+ throws Exception {
+ String routeId = ExchangeHelper.getAtRouteId(exchange);
+ GenAiObservation agentRun = GenAiObservability.start(exchange,
GenAiObservationContext.builder()
+ .operationName(GenAiOperationName.INVOKE_AGENT)
+ .agentName(routeId != null ? routeId : "openai")
+ .system("openai")
+ .requestModel(model)
+ .conversationId(conversationId)
+ .componentScheme("openai")
+ .build());
+ agentRun.recordInputMessages(() ->
OpenAIResponsesGenAiMessages.input(params));
+ try {
+ processToolLoop(exchange, config, params, input, model,
+ conversationId);
+ // the model calls carry the token usage; repeating it on the
agent run would double-count
+ agentRun.recordSuccess(GenAiUsage.of((Long) null, null,
+
exchange.getMessage().getHeader(OpenAIConstants.FINISH_REASON, String.class),
null));
+ agentRun.recordOutputMessages(() -> new
GenAiMessages().text(GenAiMessages.ASSISTANT,
+ exchange.getMessage().getBody(String.class)));
+ } catch (Exception e) {
+ agentRun.recordError(e);
+ throw e;
+ } finally {
+ agentRun.close();
+ }
+ }
+
/**
* Runs the tool loop: the function calls requested by the model are
executed on the MCP servers and route tools of
* the endpoint, and their results are sent back with the conversation so
far until the model answers.
@@ -303,7 +346,7 @@ public class OpenAIResponsesProducer extends
DefaultAsyncProducer {
}
ResponseCreateParams params = paramsBuilder.build();
- Response response = createResponse(exchange, model, params);
+ Response response = createResponse(exchange, model, params,
conversationId);
tokenTracker.addUsage(response);
tokenTracker.setHeaders(exchange.getMessage());
long iterationPromptTokens =
tokenTracker.promptTokensSince(tokensBefore);
@@ -392,22 +435,50 @@ public class OpenAIResponsesProducer extends
DefaultAsyncProducer {
}
}
- private Response createResponse(Exchange exchange, String model,
ResponseCreateParams params) throws Exception {
- return observedCall(exchange, model, () ->
getEndpoint().getClient().responses().create(params));
+ private Response createResponse(Exchange exchange, String model,
ResponseCreateParams params, String conversationId)
+ throws Exception {
+ return observedCall(exchange, model, params, conversationId,
+ () -> getEndpoint().getClient().responses().create(params));
}
- private Response observedCall(Exchange exchange, String model,
ThrowingSupplier<Response, Exception> call)
+ private Response observedCall(
+ Exchange exchange, String model, ResponseCreateParams params,
String conversationId,
+ ThrowingSupplier<Response, Exception> call)
throws Exception {
GenAiObservationContext observationContext =
GenAiObservationContext.builder()
.operationName(GenAiOperationName.CHAT)
.system("openai")
.requestModel(model)
.componentScheme("openai")
+ .conversationId(conversationId)
+ .requestParameter("temperature",
params.temperature().orElse(null))
+ .requestParameter("top_p", params.topP().orElse(null))
+ .requestParameter("max_tokens",
params.maxOutputTokens().orElse(null))
.build();
GenAiObservation observation = GenAiObservability.start(exchange,
observationContext);
+ observation.recordInputMessages(() ->
OpenAIResponsesGenAiMessages.input(params));
+ observation.recordSystemInstructions(() ->
params.instructions().orElse(null));
+ observation.recordToolDefinitions(() ->
OpenAIResponsesGenAiMessages.toolDefinitions(params));
try {
Response response = call.get();
- recordResponseSuccess(observation, response);
+ String finishReason =
OpenAIResponsesSupport.extractFinishStatus(response)
+ .map(OpenAIResponsesProducer::mapFinishReason)
+ .orElse(null);
+ response.usage().ifPresentOrElse(
+ usage -> {
+ observation.recordSuccess(GenAiUsage.of(
+ usage.inputTokens(),
+ usage.outputTokens(),
+ finishReason,
+
OpenAIResponsesSupport.modelName(response.model())));
+ observation.recordCachedInputTokens(
+ usage._inputTokensDetails().asKnown()
+ .flatMap(d ->
d._cachedTokens().asKnown()).orElse(null));
+ },
+ () -> observation.recordSuccess(
+ GenAiUsage.of((Long) null, null, finishReason,
+
OpenAIResponsesSupport.modelName(response.model()))));
+ observation.recordOutputMessages(() ->
OpenAIResponsesGenAiMessages.output(response, finishReason));
return response;
} catch (Exception e) {
GenAiErrorSupport.apply(exchange, e);
@@ -418,20 +489,6 @@ public class OpenAIResponsesProducer extends
DefaultAsyncProducer {
}
}
- private static void recordResponseSuccess(GenAiObservation observation,
Response response) {
- String finishReason =
OpenAIResponsesSupport.extractFinishStatus(response)
- .map(OpenAIResponsesProducer::mapFinishReason)
- .orElse(null);
- response.usage().ifPresentOrElse(
- usage -> observation.recordSuccess(GenAiUsage.of(
- usage.inputTokens(),
- usage.outputTokens(),
- finishReason,
- OpenAIResponsesSupport.modelName(response.model()))),
- () -> observation.recordSuccess(
- GenAiUsage.of((Long) null, null, finishReason,
OpenAIResponsesSupport.modelName(response.model()))));
- }
-
private void finishExchange(Exchange exchange, OpenAIConfiguration config,
Response response, Object body)
throws Exception {
if (config.isStoreFullResponse()) {
diff --git
a/components/camel-ai/camel-openai/src/test/java/org/apache/camel/component/openai/OpenAIResponsesObservabilityLevelTest.java
b/components/camel-ai/camel-openai/src/test/java/org/apache/camel/component/openai/OpenAIResponsesObservabilityLevelTest.java
new file mode 100644
index 000000000000..2ec30c0558f9
--- /dev/null
+++
b/components/camel-ai/camel-openai/src/test/java/org/apache/camel/component/openai/OpenAIResponsesObservabilityLevelTest.java
@@ -0,0 +1,242 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.camel.component.openai;
+
+import java.util.ArrayList;
+import java.util.HashMap;
+import java.util.List;
+import java.util.Map;
+import java.util.Set;
+
+import com.openai.models.chat.completions.ChatCompletionFunctionTool;
+import io.modelcontextprotocol.client.McpSyncClient;
+import io.modelcontextprotocol.spec.McpSchema;
+import org.apache.camel.CamelContext;
+import org.apache.camel.CamelContextAware;
+import org.apache.camel.builder.RouteBuilder;
+import org.apache.camel.component.ai.observability.GenAiAttributes;
+import
org.apache.camel.component.ai.observability.GenAiObservabilityProperties;
+import org.apache.camel.telemetry.Span;
+import org.apache.camel.telemetry.SpanContextPropagationExtractor;
+import org.apache.camel.telemetry.SpanContextPropagationInjector;
+import org.apache.camel.telemetry.SpanLifecycleManager;
+import org.apache.camel.telemetry.Tracer;
+import org.apache.camel.test.infra.openai.mock.OpenAIMock;
+import org.apache.camel.test.junit6.CamelTestSupport;
+import org.junit.jupiter.api.Test;
+import org.junit.jupiter.api.extension.RegisterExtension;
+
+import static org.assertj.core.api.Assertions.assertThat;
+import static org.mockito.ArgumentMatchers.any;
+import static org.mockito.Mockito.mock;
+import static org.mockito.Mockito.when;
+
+/**
+ * At observability level full, the Responses API tool loop is wrapped in one
invoke_agent span with one chat span per
+ * model call and an execute_tool span per MCP tool call.
+ */
+class OpenAIResponsesObservabilityLevelTest extends CamelTestSupport {
+
+ private static final String ENDPOINT =
"openai:responses?model=gpt-4o-mini&apiKey=dummy&autoToolExecution=true";
+
+ @RegisterExtension
+ static OpenAIMock openAIMock = new OpenAIMock().builder()
+ .when("weather please")
+ .invokeTool("get_weather")
+ .replyWith("It is sunny")
+ .end()
+ .build();
+
+ private final List<RecordingSpan> closedSpans = new ArrayList<>();
+
+ @Override
+ protected CamelContext createCamelContext() throws Exception {
+ CamelContext context = super.createCamelContext();
+ Tracer tracer = new Tracer() {
+ @Override
+ protected void initTracer() {
+ setSpanLifecycleManager(new RecordingLifecycleManager());
+ }
+ };
+ CamelContextAware.trySetCamelContext(tracer, context);
+ tracer.init(context);
+
context.getPropertiesComponent().addOverrideProperty(GenAiObservabilityProperties.LEVEL,
"full");
+ return context;
+ }
+
+ @Override
+ protected RouteBuilder createRouteBuilder() {
+ return new RouteBuilder() {
+ @Override
+ public void configure() {
+ from("direct:responses").routeId("weather-responses")
+ .to(ENDPOINT + "&baseUrl=" + openAIMock.getBaseUrl() +
"/v1");
+ }
+ };
+ }
+
+ @Test
+ void shouldRecordTheResponsesToolLoopAsAnAgentRun() {
+ injectWeatherTool(ENDPOINT + "&baseUrl=" + openAIMock.getBaseUrl() +
"/v1");
+
+ assertThat(template.requestBody("direct:responses", "weather please",
String.class)).isEqualTo("It is sunny");
+
+ RecordingSpan run = span("invoke_agent weather-responses");
+ List<RecordingSpan> calls = closedSpans.stream().filter(s ->
s.name().equals("chat gpt-4o-mini")).toList();
+ RecordingSpan tool = span("execute_tool get_weather");
+
+ // invoke_agent: agent name set, no token totals (model calls carry
them)
+ assertThat(run.tags()).containsEntry(GenAiAttributes.AGENT_NAME,
"weather-responses")
+ .doesNotContainKey(GenAiAttributes.INPUT_TOKENS);
+ // input messages recorded on the agent run at level full
+ assertThat((String)
run.tags().get(GenAiAttributes.INPUT_MESSAGES)).contains("weather please");
+ // output messages recorded on the agent run
+ assertThat((String)
run.tags().get(GenAiAttributes.OUTPUT_MESSAGES)).contains("It is sunny");
+
+ // two model calls: first requests the tool, second delivers the final
answer
+ assertThat(calls).hasSize(2).allSatisfy(call ->
assertThat(call.parent()).isSameAs(run));
+ // tool definitions recorded on the first call at level full
+ assertThat((String)
calls.get(0).tags().get(GenAiAttributes.TOOL_DEFINITIONS))
+ .contains("\"type\":\"function\"", "\"name\":\"get_weather\"");
+ // system instructions recorded when present (none in this test, but
output messages on model calls)
+ assertThat((String)
calls.get(1).tags().get(GenAiAttributes.OUTPUT_MESSAGES)).contains("It is
sunny");
+
+ // second call's input messages include tool result
+ assertThat((String)
calls.get(1).tags().get(GenAiAttributes.INPUT_MESSAGES)).contains("tool");
+
+ // execute_tool span under the agent run
+ assertThat(tool.parent()).isSameAs(run);
+ assertThat(tool.tags()).containsEntry(GenAiAttributes.TOOL_NAME,
"get_weather")
+ .containsEntry(GenAiAttributes.TOOL_CALL_RESULT, "sunny");
+ }
+
+ @Test
+ void shouldRecordRequestDetailsAtDetailedLevel() {
+ closedSpans.clear();
+
context.getPropertiesComponent().addOverrideProperty(GenAiObservabilityProperties.LEVEL,
"detailed");
+
+ String endpointUri = ENDPOINT + "&baseUrl=" + openAIMock.getBaseUrl()
+ "/v1";
+ injectWeatherTool(endpointUri);
+
+ template.request("direct:responses", e -> {
+ e.getIn().setBody("weather please");
+ e.getIn().setHeader(OpenAIConstants.CONVERSATION_ID, "conv-test");
+ e.getIn().setHeader(OpenAIConstants.TEMPERATURE, 0.7);
+ e.getIn().setHeader(OpenAIConstants.MAX_TOKENS, 200);
+ });
+
+ RecordingSpan run = span("invoke_agent weather-responses");
+ List<RecordingSpan> chatSpans = closedSpans.stream().filter(s ->
s.name().equals("chat gpt-4o-mini")).toList();
+
+ // conversation_id on the invoke_agent span
+ assertThat(run.tags()).containsEntry(GenAiAttributes.CONVERSATION_ID,
"conv-test");
+
+ // request params on the chat spans
+ assertThat(chatSpans).isNotEmpty();
+ assertThat(chatSpans.get(0).tags())
+ .containsEntry(GenAiAttributes.REQUEST_PREFIX + "temperature",
0.7)
+ .containsEntry(GenAiAttributes.REQUEST_PREFIX + "max_tokens",
200L)
+ .containsEntry(GenAiAttributes.CONVERSATION_ID, "conv-test");
+
+ // INPUT_MESSAGES is NOT set at detailed level
+
assertThat(chatSpans.get(0).tags()).doesNotContainKey(GenAiAttributes.INPUT_MESSAGES);
+
assertThat(run.tags()).doesNotContainKey(GenAiAttributes.INPUT_MESSAGES);
+ }
+
+ private RecordingSpan span(String name) {
+ List<RecordingSpan> spans = closedSpans.stream().filter(s ->
s.name().equals(name)).toList();
+ assertThat(spans).as("spans named %s", name).hasSize(1);
+ return spans.get(0);
+ }
+
+ private void injectWeatherTool(String endpointUri) {
+ OpenAIEndpoint endpoint = context.getEndpoint(endpointUri,
OpenAIEndpoint.class);
+ McpToolState current = endpoint.getMcpToolState();
+ McpSyncClient client = mock(McpSyncClient.class);
+
when(client.callTool(any(McpSchema.CallToolRequest.class))).thenReturn(McpSchema.CallToolResult.builder()
+ .content(List.of(new McpSchema.TextContent(null, "sunny",
null)))
+ .isError(false)
+ .build());
+ List<ChatCompletionFunctionTool> tools = new
ArrayList<>(current.tools());
+ tools.add(AiToolSpecToOpenAI.function("get_weather", "weather
forecast", "{\"type\":\"object\"}"));
+ endpoint.setMcpToolState(new McpToolState(
+ tools, Map.of("get_weather", client), Map.of("get_weather",
"stub"), Set.of(), current.routeTools()));
+ }
+
+ private final class RecordingLifecycleManager implements
SpanLifecycleManager {
+
+ @Override
+ public Span create(String spanName, String spanKind, Span parent,
SpanContextPropagationExtractor extractor) {
+ return new RecordingSpan(spanName, parent, new HashMap<>());
+ }
+
+ @Override
+ public void activate(Span span) {
+ // noop
+ }
+
+ @Override
+ public void deactivate(Span span) {
+ // noop
+ }
+
+ @Override
+ public void close(Span span) {
+ synchronized (closedSpans) {
+ closedSpans.add((RecordingSpan) span);
+ }
+ }
+
+ @Override
+ public void inject(Span span, SpanContextPropagationInjector injector,
boolean includeTracing) {
+ // noop
+ }
+ }
+
+ private record RecordingSpan(String name, Span parent, Map<String, Object>
tags) implements Span {
+
+ @Override
+ public void log(Map<String, String> fields) {
+ // noop
+ }
+
+ @Override
+ public void setTag(String key, String value) {
+ tags.put(key, value);
+ }
+
+ @Override
+ public void setTag(String key, long value) {
+ tags.put(key, value);
+ }
+
+ @Override
+ public void setTag(String key, double value) {
+ tags.put(key, value);
+ }
+
+ @Override
+ public void setComponent(String component) {
+ tags.put("component", component);
+ }
+
+ @Override
+ public void setError(boolean isError) {
+ tags.put("error", isError);
+ }
+ }
+}