This is an automated email from the ASF dual-hosted git repository.

davsclaus pushed a commit to branch main
in repository https://gitbox.apache.org/repos/asf/camel.git


The following commit(s) were added to refs/heads/main by this push:
     new 757d4edca435 CAMEL-27142: camel-openai - record the detailed and full 
GenAI data for the responses operation (#30326)
757d4edca435 is described below

commit 757d4edca4355b42198d0baf5fea107c3ab8e3c9
Author: Guillaume Nodet <[email protected]>
AuthorDate: Fri Nov 20 08:32:49 2026 +0100

    CAMEL-27142: camel-openai - record the detailed and full GenAI data for the 
responses operation (#30326)
    
    Mirror the observability enhancements from CAMEL-27131 (OpenAIProducer) to
    OpenAIResponsesProducer:
    
    - processObservedToolLoop(): wraps the tool loop in an invoke_agent span at
      level detailed or higher, named after the route id; model calls and tool 
calls
      started inside nest under it, giving tracing backends one agent run per 
turn.
    - observedCall(): records conversation id, request parameters (temperature,
      top_p, max_tokens), tracedByFramework flag, cached input tokens, input 
messages
      (level full), system instructions (level full), and tool definitions 
(level full)
      on each per-model-call chat span.
    - OpenAIResponsesGenAiMessages: converts ResponseCreateParams input items 
and
      Response output items to GenAiMessages/GenAiToolDefinitions for the full 
level.
    - OpenAIResponsesObservabilityLevelTest: verifies that the tool loop 
produces
      one invoke_agent span with two nested chat spans and an execute_tool 
span, with
      input messages and tool definitions recorded at level full.
    
    Follow-ups in the PR:
    - fix compilation errors: pass conversationId explicitly, fix the 
cachedTokens accessor
    - toolDefinitions parameters, an assistant message before a function call, 
image placeholders, and the tracedByFramework check of the responses service
    - regenerate the catalog copy of ai-observability.adoc
    - the detailed-level test uses the route endpoint and passes 
temperature/maxTokens via headers
    
    Co-authored-by: Claude Sonnet 4.6 <[email protected]>
---
 .../camel/catalog/docs/ai-observability.adoc       |   4 +-
 .../src/main/docs/ai-observability.adoc            |   4 +-
 .../openai/OpenAIResponsesGenAiMessages.java       | 173 +++++++++++++++
 .../component/openai/OpenAIResponsesProducer.java  | 103 +++++++--
 .../OpenAIResponsesObservabilityLevelTest.java     | 242 +++++++++++++++++++++
 5 files changed, 501 insertions(+), 25 deletions(-)

diff --git 
a/catalog/camel-catalog/src/generated/resources/org/apache/camel/catalog/docs/ai-observability.adoc
 
b/catalog/camel-catalog/src/generated/resources/org/apache/camel/catalog/docs/ai-observability.adoc
index 82d5cec29d35..1d926f5b1718 100644
--- 
a/catalog/camel-catalog/src/generated/resources/org/apache/camel/catalog/docs/ai-observability.adoc
+++ 
b/catalog/camel-catalog/src/generated/resources/org/apache/camel/catalog/docs/ai-observability.adoc
@@ -115,7 +115,9 @@ the agent run and the Camel route tools are still recorded 
by Camel. The framewo
 that is current when the endpoint is called, which is the route span: they 
appear next to the agent run, not under it,
 because Camel makes its spans current only while a processor runs.
 * `openai`: the `openai-java` instrumentation of OpenTelemetry (in the 
OpenTelemetry Java agent) wraps the OpenAI client
-used by `camel-openai` and traces its chat completions and embeddings. When 
the client is wrapped, those calls are left
+used by `camel-openai` and traces its chat completions and embeddings. The 
`responses()` operation is not yet
+instrumented by the OpenTelemetry Java agent (as of 2.33.x), so Camel always 
records its own span for Responses API
+calls. When the client is wrapped, chat completion and embeddings calls are 
left
 to it, except at level `full`: the instrumentation records the messages as log 
events, not as span attributes, so Camel
 keeps its own span and logs a warning. Start the JVM with 
`-Dotel.instrumentation.openai-java.enabled=false` to keep
 only the Camel spans.
diff --git 
a/components/camel-ai/camel-ai-observability/src/main/docs/ai-observability.adoc
 
b/components/camel-ai/camel-ai-observability/src/main/docs/ai-observability.adoc
index 82d5cec29d35..1d926f5b1718 100644
--- 
a/components/camel-ai/camel-ai-observability/src/main/docs/ai-observability.adoc
+++ 
b/components/camel-ai/camel-ai-observability/src/main/docs/ai-observability.adoc
@@ -115,7 +115,9 @@ the agent run and the Camel route tools are still recorded 
by Camel. The framewo
 that is current when the endpoint is called, which is the route span: they 
appear next to the agent run, not under it,
 because Camel makes its spans current only while a processor runs.
 * `openai`: the `openai-java` instrumentation of OpenTelemetry (in the 
OpenTelemetry Java agent) wraps the OpenAI client
-used by `camel-openai` and traces its chat completions and embeddings. When 
the client is wrapped, those calls are left
+used by `camel-openai` and traces its chat completions and embeddings. The 
`responses()` operation is not yet
+instrumented by the OpenTelemetry Java agent (as of 2.33.x), so Camel always 
records its own span for Responses API
+calls. When the client is wrapped, chat completion and embeddings calls are 
left
 to it, except at level `full`: the instrumentation records the messages as log 
events, not as span attributes, so Camel
 keeps its own span and logs a warning. Start the JVM with 
`-Dotel.instrumentation.openai-java.enabled=false` to keep
 only the Camel spans.
diff --git 
a/components/camel-ai/camel-openai/src/main/java/org/apache/camel/component/openai/OpenAIResponsesGenAiMessages.java
 
b/components/camel-ai/camel-openai/src/main/java/org/apache/camel/component/openai/OpenAIResponsesGenAiMessages.java
new file mode 100644
index 000000000000..1da5fcd8afd7
--- /dev/null
+++ 
b/components/camel-ai/camel-openai/src/main/java/org/apache/camel/component/openai/OpenAIResponsesGenAiMessages.java
@@ -0,0 +1,173 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements.  See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License.  You may obtain a copy of the License at
+ *
+ *      http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.camel.component.openai;
+
+import java.util.List;
+import java.util.Locale;
+import java.util.Map;
+import java.util.Objects;
+import java.util.stream.Collectors;
+
+import com.openai.core.ObjectMappers;
+import com.openai.models.responses.EasyInputMessage;
+import com.openai.models.responses.Response;
+import com.openai.models.responses.ResponseCreateParams;
+import com.openai.models.responses.ResponseFunctionToolCall;
+import com.openai.models.responses.ResponseInputContent;
+import com.openai.models.responses.ResponseInputItem;
+import com.openai.models.responses.ResponseOutputItem;
+import com.openai.models.responses.ResponseOutputMessage;
+import com.openai.models.responses.Tool;
+import org.apache.camel.component.ai.observability.GenAiMessages;
+import org.apache.camel.component.ai.observability.GenAiToolDefinitions;
+
+/**
+ * Converts Responses API requests and responses into the messages of the 
GenAI semantic conventions, for observability
+ * level {@code full}.
+ */
+final class OpenAIResponsesGenAiMessages {
+
+    private OpenAIResponsesGenAiMessages() {
+    }
+
+    static GenAiMessages input(ResponseCreateParams params) {
+        GenAiMessages messages = new GenAiMessages();
+        params.input().ifPresent(input -> {
+            if (input.isText()) {
+                messages.text(GenAiMessages.USER, input.asText());
+            } else if (input.isResponse()) {
+                addInputItems(messages, input.asResponse());
+            }
+        });
+        return messages;
+    }
+
+    static GenAiMessages output(Response response, String finishReason) {
+        GenAiMessages messages = new GenAiMessages();
+        for (ResponseOutputItem item : response.output()) {
+            if (item.isMessage()) {
+                String text = item.asMessage().content().stream()
+                        .filter(ResponseOutputMessage.Content::isOutputText)
+                        .map(c -> c.asOutputText().text())
+                        .collect(Collectors.joining("\n"));
+                messages.text(GenAiMessages.ASSISTANT, text.isEmpty() ? null : 
text);
+            } else if (item.isFunctionCall()) {
+                ResponseFunctionToolCall call = item.asFunctionCall();
+                messages.toolCall(call.callId(), call.name(), 
call.arguments());
+            }
+        }
+        return messages.finishReason(finishReason);
+    }
+
+    @SuppressWarnings("unchecked")
+    static GenAiToolDefinitions toolDefinitions(ResponseCreateParams params) {
+        GenAiToolDefinitions definitions = new GenAiToolDefinitions();
+        for (Tool tool : params.tools().orElse(List.of())) {
+            tool.function().ifPresent(function -> definitions.function(
+                    function.name(),
+                    function.description().orElse(null),
+                    function.parameters()
+                            .map(p -> 
ObjectMappers.jsonMapper().convertValue(p, Map.class))
+                            .orElse(null)));
+        }
+        return definitions;
+    }
+
+    private static void addInputItems(GenAiMessages messages, 
List<ResponseInputItem> items) {
+        boolean lastIsAssistant = false;
+        for (ResponseInputItem item : items) {
+            if (item.isEasyInputMessage()) {
+                EasyInputMessage msg = item.asEasyInputMessage();
+                String role = roleOf(msg.role().asString());
+                lastIsAssistant = GenAiMessages.ASSISTANT.equals(role);
+                addEasyInputMessage(messages, role, msg.content());
+            } else if (item.isMessage()) {
+                // ResponseInputItem.Message - image/PDF prompt items from 
OpenAIResponsesInputBuilder
+                ResponseInputItem.Message msg = item.asMessage();
+                String role = roleOf(msg.role().asString());
+                lastIsAssistant = GenAiMessages.ASSISTANT.equals(role);
+                String text = msg.content().stream()
+                        .filter(ResponseInputContent::isInputText)
+                        .map(c -> c.asInputText().text())
+                        .collect(Collectors.joining("\n"));
+                if (text.isEmpty()) {
+                    // binary content (image/PDF) - record placeholder
+                    messages.text(role, null);
+                } else {
+                    messages.text(role, text);
+                }
+            } else if (item.isResponseOutputMessage()) {
+                // assistant preamble from 
OpenAIResponsesSupport.toInputItems()
+                ResponseOutputMessage msg = item.asResponseOutputMessage();
+                String text = msg.content().stream()
+                        .filter(ResponseOutputMessage.Content::isOutputText)
+                        .map(c -> c.asOutputText().text())
+                        .collect(Collectors.joining("\n"));
+                lastIsAssistant = true;
+                messages.text(GenAiMessages.ASSISTANT, text.isEmpty() ? null : 
text);
+            } else if (item.isFunctionCallOutput()) {
+                lastIsAssistant = false;
+                ResponseInputItem.FunctionCallOutput output = 
item.asFunctionCallOutput();
+                String callId = output.callId().orElse(null);
+                String content = output.output().string().orElse(null);
+                messages.toolResult(callId, content);
+            } else if (item.isFunctionCall()) {
+                // Ensure tool_call lands in an assistant message per semconv
+                if (!lastIsAssistant) {
+                    messages.text(GenAiMessages.ASSISTANT, null);
+                }
+                lastIsAssistant = true;
+                ResponseFunctionToolCall call = item.asFunctionCall();
+                messages.toolCall(call.callId(), call.name(), 
call.arguments());
+            }
+        }
+    }
+
+    private static void addEasyInputMessage(GenAiMessages messages, String 
role, EasyInputMessage.Content content) {
+        if (content.isTextInput()) {
+            messages.text(role, content.asTextInput());
+        } else if (content.isResponseInputMessageContentList()) {
+            String text = content.asResponseInputMessageContentList().stream()
+                    .map(c -> {
+                        if (c.isInputText()) {
+                            return c.asInputText().text();
+                        }
+                        if (c.isInputImage()) {
+                            return "[image]";
+                        }
+                        if (c.isInputFile()) {
+                            return "[file]";
+                        }
+                        return null;
+                    })
+                    .filter(Objects::nonNull)
+                    .collect(Collectors.joining("\n"));
+            messages.text(role, text.isEmpty() ? null : text);
+        }
+    }
+
+    private static String roleOf(String apiRole) {
+        if (apiRole == null) {
+            return GenAiMessages.USER;
+        }
+        return switch (apiRole.toLowerCase(Locale.ROOT)) {
+            case "assistant" -> GenAiMessages.ASSISTANT;
+            case "system", "developer" -> GenAiMessages.SYSTEM;
+            default -> GenAiMessages.USER;
+        };
+    }
+}
diff --git 
a/components/camel-ai/camel-openai/src/main/java/org/apache/camel/component/openai/OpenAIResponsesProducer.java
 
b/components/camel-ai/camel-openai/src/main/java/org/apache/camel/component/openai/OpenAIResponsesProducer.java
index 84583603f964..0cb74e146ba4 100644
--- 
a/components/camel-ai/camel-openai/src/main/java/org/apache/camel/component/openai/OpenAIResponsesProducer.java
+++ 
b/components/camel-ai/camel-openai/src/main/java/org/apache/camel/component/openai/OpenAIResponsesProducer.java
@@ -32,13 +32,16 @@ import org.apache.camel.AsyncCallback;
 import org.apache.camel.Exchange;
 import org.apache.camel.Message;
 import org.apache.camel.component.ai.observability.GenAiErrorSupport;
+import org.apache.camel.component.ai.observability.GenAiMessages;
 import org.apache.camel.component.ai.observability.GenAiObservability;
+import org.apache.camel.component.ai.observability.GenAiObservabilityLevel;
 import org.apache.camel.component.ai.observability.GenAiObservation;
 import org.apache.camel.component.ai.observability.GenAiObservationContext;
 import org.apache.camel.component.ai.observability.GenAiOperationName;
 import org.apache.camel.component.ai.observability.GenAiUsage;
 import org.apache.camel.component.ai.tool.AiToolSearchSession;
 import org.apache.camel.support.DefaultAsyncProducer;
+import org.apache.camel.support.ExchangeHelper;
 import org.apache.camel.support.ResourceHelper;
 import org.apache.camel.support.service.ServiceHelper;
 import org.apache.camel.util.ObjectHelper;
@@ -169,7 +172,8 @@ public class OpenAIResponsesProducer extends 
DefaultAsyncProducer {
         }
         String conversationId = resolveParameter(in, 
OpenAIConstants.CONVERSATION_ID, config.getConversationId(),
                 String.class);
-        if (ObjectHelper.isNotEmpty(conversationId)) {
+        conversationId = ObjectHelper.isNotEmpty(conversationId) ? 
conversationId : null;
+        if (conversationId != null) {
             paramsBuilder.conversation(conversationId);
         }
         if (config.isBackground()) {
@@ -226,10 +230,15 @@ public class OpenAIResponsesProducer extends 
DefaultAsyncProducer {
                             "background cannot be combined with automatic tool 
execution, the tool loop needs each answer. "
                                                        + "Set 
autoToolExecution=false to handle the function calls in the route");
                 }
-                processToolLoop(exchange, config, params, inputSpec.items(), 
model,
-                        ObjectHelper.isNotEmpty(conversationId) ? 
conversationId : null);
+                if (GenAiObservability.isLevel(exchange.getContext(), 
GenAiObservabilityLevel.DETAILED)) {
+                    processObservedToolLoop(exchange, config, params, 
inputSpec.items(), model, conversationId);
+                } else {
+                    processToolLoop(exchange, config, params, 
inputSpec.items(), model,
+                            conversationId);
+                }
             } else {
-                Response response = createResponse(exchange, model, params);
+                Response response = createResponse(exchange, model, params,
+                        conversationId);
                 List<ResponseFunctionToolCall> functionCalls = 
OpenAIResponsesSupport.extractFunctionCalls(response);
                 // with autoToolExecution=false the route handles the function 
calls requested by the model
                 Object body = functionCalls.isEmpty() ? 
OpenAIResponsesSupport.extractAssistantText(response) : functionCalls;
@@ -247,6 +256,40 @@ public class OpenAIResponsesProducer extends 
DefaultAsyncProducer {
         }
     }
 
+    /**
+     * Wraps the tool loop in an {@code invoke_agent} span at observability 
level {@code detailed} or higher. The model
+     * calls and tool calls started inside nest under it, giving tracing 
backends one agent run per user turn.
+     */
+    private void processObservedToolLoop(
+            Exchange exchange, OpenAIConfiguration config, 
ResponseCreateParams params, List<ResponseInputItem> input,
+            String model, String conversationId)
+            throws Exception {
+        String routeId = ExchangeHelper.getAtRouteId(exchange);
+        GenAiObservation agentRun = GenAiObservability.start(exchange, 
GenAiObservationContext.builder()
+                .operationName(GenAiOperationName.INVOKE_AGENT)
+                .agentName(routeId != null ? routeId : "openai")
+                .system("openai")
+                .requestModel(model)
+                .conversationId(conversationId)
+                .componentScheme("openai")
+                .build());
+        agentRun.recordInputMessages(() -> 
OpenAIResponsesGenAiMessages.input(params));
+        try {
+            processToolLoop(exchange, config, params, input, model,
+                    conversationId);
+            // the model calls carry the token usage; repeating it on the 
agent run would double-count
+            agentRun.recordSuccess(GenAiUsage.of((Long) null, null,
+                    
exchange.getMessage().getHeader(OpenAIConstants.FINISH_REASON, String.class), 
null));
+            agentRun.recordOutputMessages(() -> new 
GenAiMessages().text(GenAiMessages.ASSISTANT,
+                    exchange.getMessage().getBody(String.class)));
+        } catch (Exception e) {
+            agentRun.recordError(e);
+            throw e;
+        } finally {
+            agentRun.close();
+        }
+    }
+
     /**
      * Runs the tool loop: the function calls requested by the model are 
executed on the MCP servers and route tools of
      * the endpoint, and their results are sent back with the conversation so 
far until the model answers.
@@ -303,7 +346,7 @@ public class OpenAIResponsesProducer extends 
DefaultAsyncProducer {
                 }
 
                 ResponseCreateParams params = paramsBuilder.build();
-                Response response = createResponse(exchange, model, params);
+                Response response = createResponse(exchange, model, params, 
conversationId);
                 tokenTracker.addUsage(response);
                 tokenTracker.setHeaders(exchange.getMessage());
                 long iterationPromptTokens = 
tokenTracker.promptTokensSince(tokensBefore);
@@ -392,22 +435,50 @@ public class OpenAIResponsesProducer extends 
DefaultAsyncProducer {
         }
     }
 
-    private Response createResponse(Exchange exchange, String model, 
ResponseCreateParams params) throws Exception {
-        return observedCall(exchange, model, () -> 
getEndpoint().getClient().responses().create(params));
+    private Response createResponse(Exchange exchange, String model, 
ResponseCreateParams params, String conversationId)
+            throws Exception {
+        return observedCall(exchange, model, params, conversationId,
+                () -> getEndpoint().getClient().responses().create(params));
     }
 
-    private Response observedCall(Exchange exchange, String model, 
ThrowingSupplier<Response, Exception> call)
+    private Response observedCall(
+            Exchange exchange, String model, ResponseCreateParams params, 
String conversationId,
+            ThrowingSupplier<Response, Exception> call)
             throws Exception {
         GenAiObservationContext observationContext = 
GenAiObservationContext.builder()
                 .operationName(GenAiOperationName.CHAT)
                 .system("openai")
                 .requestModel(model)
                 .componentScheme("openai")
+                .conversationId(conversationId)
+                .requestParameter("temperature", 
params.temperature().orElse(null))
+                .requestParameter("top_p", params.topP().orElse(null))
+                .requestParameter("max_tokens", 
params.maxOutputTokens().orElse(null))
                 .build();
         GenAiObservation observation = GenAiObservability.start(exchange, 
observationContext);
+        observation.recordInputMessages(() -> 
OpenAIResponsesGenAiMessages.input(params));
+        observation.recordSystemInstructions(() -> 
params.instructions().orElse(null));
+        observation.recordToolDefinitions(() -> 
OpenAIResponsesGenAiMessages.toolDefinitions(params));
         try {
             Response response = call.get();
-            recordResponseSuccess(observation, response);
+            String finishReason = 
OpenAIResponsesSupport.extractFinishStatus(response)
+                    .map(OpenAIResponsesProducer::mapFinishReason)
+                    .orElse(null);
+            response.usage().ifPresentOrElse(
+                    usage -> {
+                        observation.recordSuccess(GenAiUsage.of(
+                                usage.inputTokens(),
+                                usage.outputTokens(),
+                                finishReason,
+                                
OpenAIResponsesSupport.modelName(response.model())));
+                        observation.recordCachedInputTokens(
+                                usage._inputTokensDetails().asKnown()
+                                        .flatMap(d -> 
d._cachedTokens().asKnown()).orElse(null));
+                    },
+                    () -> observation.recordSuccess(
+                            GenAiUsage.of((Long) null, null, finishReason,
+                                    
OpenAIResponsesSupport.modelName(response.model()))));
+            observation.recordOutputMessages(() -> 
OpenAIResponsesGenAiMessages.output(response, finishReason));
             return response;
         } catch (Exception e) {
             GenAiErrorSupport.apply(exchange, e);
@@ -418,20 +489,6 @@ public class OpenAIResponsesProducer extends 
DefaultAsyncProducer {
         }
     }
 
-    private static void recordResponseSuccess(GenAiObservation observation, 
Response response) {
-        String finishReason = 
OpenAIResponsesSupport.extractFinishStatus(response)
-                .map(OpenAIResponsesProducer::mapFinishReason)
-                .orElse(null);
-        response.usage().ifPresentOrElse(
-                usage -> observation.recordSuccess(GenAiUsage.of(
-                        usage.inputTokens(),
-                        usage.outputTokens(),
-                        finishReason,
-                        OpenAIResponsesSupport.modelName(response.model()))),
-                () -> observation.recordSuccess(
-                        GenAiUsage.of((Long) null, null, finishReason, 
OpenAIResponsesSupport.modelName(response.model()))));
-    }
-
     private void finishExchange(Exchange exchange, OpenAIConfiguration config, 
Response response, Object body)
             throws Exception {
         if (config.isStoreFullResponse()) {
diff --git 
a/components/camel-ai/camel-openai/src/test/java/org/apache/camel/component/openai/OpenAIResponsesObservabilityLevelTest.java
 
b/components/camel-ai/camel-openai/src/test/java/org/apache/camel/component/openai/OpenAIResponsesObservabilityLevelTest.java
new file mode 100644
index 000000000000..2ec30c0558f9
--- /dev/null
+++ 
b/components/camel-ai/camel-openai/src/test/java/org/apache/camel/component/openai/OpenAIResponsesObservabilityLevelTest.java
@@ -0,0 +1,242 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements.  See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License.  You may obtain a copy of the License at
+ *
+ *      http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.camel.component.openai;
+
+import java.util.ArrayList;
+import java.util.HashMap;
+import java.util.List;
+import java.util.Map;
+import java.util.Set;
+
+import com.openai.models.chat.completions.ChatCompletionFunctionTool;
+import io.modelcontextprotocol.client.McpSyncClient;
+import io.modelcontextprotocol.spec.McpSchema;
+import org.apache.camel.CamelContext;
+import org.apache.camel.CamelContextAware;
+import org.apache.camel.builder.RouteBuilder;
+import org.apache.camel.component.ai.observability.GenAiAttributes;
+import 
org.apache.camel.component.ai.observability.GenAiObservabilityProperties;
+import org.apache.camel.telemetry.Span;
+import org.apache.camel.telemetry.SpanContextPropagationExtractor;
+import org.apache.camel.telemetry.SpanContextPropagationInjector;
+import org.apache.camel.telemetry.SpanLifecycleManager;
+import org.apache.camel.telemetry.Tracer;
+import org.apache.camel.test.infra.openai.mock.OpenAIMock;
+import org.apache.camel.test.junit6.CamelTestSupport;
+import org.junit.jupiter.api.Test;
+import org.junit.jupiter.api.extension.RegisterExtension;
+
+import static org.assertj.core.api.Assertions.assertThat;
+import static org.mockito.ArgumentMatchers.any;
+import static org.mockito.Mockito.mock;
+import static org.mockito.Mockito.when;
+
+/**
+ * At observability level full, the Responses API tool loop is wrapped in one 
invoke_agent span with one chat span per
+ * model call and an execute_tool span per MCP tool call.
+ */
+class OpenAIResponsesObservabilityLevelTest extends CamelTestSupport {
+
+    private static final String ENDPOINT = 
"openai:responses?model=gpt-4o-mini&apiKey=dummy&autoToolExecution=true";
+
+    @RegisterExtension
+    static OpenAIMock openAIMock = new OpenAIMock().builder()
+            .when("weather please")
+            .invokeTool("get_weather")
+            .replyWith("It is sunny")
+            .end()
+            .build();
+
+    private final List<RecordingSpan> closedSpans = new ArrayList<>();
+
+    @Override
+    protected CamelContext createCamelContext() throws Exception {
+        CamelContext context = super.createCamelContext();
+        Tracer tracer = new Tracer() {
+            @Override
+            protected void initTracer() {
+                setSpanLifecycleManager(new RecordingLifecycleManager());
+            }
+        };
+        CamelContextAware.trySetCamelContext(tracer, context);
+        tracer.init(context);
+        
context.getPropertiesComponent().addOverrideProperty(GenAiObservabilityProperties.LEVEL,
 "full");
+        return context;
+    }
+
+    @Override
+    protected RouteBuilder createRouteBuilder() {
+        return new RouteBuilder() {
+            @Override
+            public void configure() {
+                from("direct:responses").routeId("weather-responses")
+                        .to(ENDPOINT + "&baseUrl=" + openAIMock.getBaseUrl() + 
"/v1");
+            }
+        };
+    }
+
+    @Test
+    void shouldRecordTheResponsesToolLoopAsAnAgentRun() {
+        injectWeatherTool(ENDPOINT + "&baseUrl=" + openAIMock.getBaseUrl() + 
"/v1");
+
+        assertThat(template.requestBody("direct:responses", "weather please", 
String.class)).isEqualTo("It is sunny");
+
+        RecordingSpan run = span("invoke_agent weather-responses");
+        List<RecordingSpan> calls = closedSpans.stream().filter(s -> 
s.name().equals("chat gpt-4o-mini")).toList();
+        RecordingSpan tool = span("execute_tool get_weather");
+
+        // invoke_agent: agent name set, no token totals (model calls carry 
them)
+        assertThat(run.tags()).containsEntry(GenAiAttributes.AGENT_NAME, 
"weather-responses")
+                .doesNotContainKey(GenAiAttributes.INPUT_TOKENS);
+        // input messages recorded on the agent run at level full
+        assertThat((String) 
run.tags().get(GenAiAttributes.INPUT_MESSAGES)).contains("weather please");
+        // output messages recorded on the agent run
+        assertThat((String) 
run.tags().get(GenAiAttributes.OUTPUT_MESSAGES)).contains("It is sunny");
+
+        // two model calls: first requests the tool, second delivers the final 
answer
+        assertThat(calls).hasSize(2).allSatisfy(call -> 
assertThat(call.parent()).isSameAs(run));
+        // tool definitions recorded on the first call at level full
+        assertThat((String) 
calls.get(0).tags().get(GenAiAttributes.TOOL_DEFINITIONS))
+                .contains("\"type\":\"function\"", "\"name\":\"get_weather\"");
+        // system instructions recorded when present (none in this test, but 
output messages on model calls)
+        assertThat((String) 
calls.get(1).tags().get(GenAiAttributes.OUTPUT_MESSAGES)).contains("It is 
sunny");
+
+        // second call's input messages include tool result
+        assertThat((String) 
calls.get(1).tags().get(GenAiAttributes.INPUT_MESSAGES)).contains("tool");
+
+        // execute_tool span under the agent run
+        assertThat(tool.parent()).isSameAs(run);
+        assertThat(tool.tags()).containsEntry(GenAiAttributes.TOOL_NAME, 
"get_weather")
+                .containsEntry(GenAiAttributes.TOOL_CALL_RESULT, "sunny");
+    }
+
+    @Test
+    void shouldRecordRequestDetailsAtDetailedLevel() {
+        closedSpans.clear();
+        
context.getPropertiesComponent().addOverrideProperty(GenAiObservabilityProperties.LEVEL,
 "detailed");
+
+        String endpointUri = ENDPOINT + "&baseUrl=" + openAIMock.getBaseUrl() 
+ "/v1";
+        injectWeatherTool(endpointUri);
+
+        template.request("direct:responses", e -> {
+            e.getIn().setBody("weather please");
+            e.getIn().setHeader(OpenAIConstants.CONVERSATION_ID, "conv-test");
+            e.getIn().setHeader(OpenAIConstants.TEMPERATURE, 0.7);
+            e.getIn().setHeader(OpenAIConstants.MAX_TOKENS, 200);
+        });
+
+        RecordingSpan run = span("invoke_agent weather-responses");
+        List<RecordingSpan> chatSpans = closedSpans.stream().filter(s -> 
s.name().equals("chat gpt-4o-mini")).toList();
+
+        // conversation_id on the invoke_agent span
+        assertThat(run.tags()).containsEntry(GenAiAttributes.CONVERSATION_ID, 
"conv-test");
+
+        // request params on the chat spans
+        assertThat(chatSpans).isNotEmpty();
+        assertThat(chatSpans.get(0).tags())
+                .containsEntry(GenAiAttributes.REQUEST_PREFIX + "temperature", 
0.7)
+                .containsEntry(GenAiAttributes.REQUEST_PREFIX + "max_tokens", 
200L)
+                .containsEntry(GenAiAttributes.CONVERSATION_ID, "conv-test");
+
+        // INPUT_MESSAGES is NOT set at detailed level
+        
assertThat(chatSpans.get(0).tags()).doesNotContainKey(GenAiAttributes.INPUT_MESSAGES);
+        
assertThat(run.tags()).doesNotContainKey(GenAiAttributes.INPUT_MESSAGES);
+    }
+
+    private RecordingSpan span(String name) {
+        List<RecordingSpan> spans = closedSpans.stream().filter(s -> 
s.name().equals(name)).toList();
+        assertThat(spans).as("spans named %s", name).hasSize(1);
+        return spans.get(0);
+    }
+
+    private void injectWeatherTool(String endpointUri) {
+        OpenAIEndpoint endpoint = context.getEndpoint(endpointUri, 
OpenAIEndpoint.class);
+        McpToolState current = endpoint.getMcpToolState();
+        McpSyncClient client = mock(McpSyncClient.class);
+        
when(client.callTool(any(McpSchema.CallToolRequest.class))).thenReturn(McpSchema.CallToolResult.builder()
+                .content(List.of(new McpSchema.TextContent(null, "sunny", 
null)))
+                .isError(false)
+                .build());
+        List<ChatCompletionFunctionTool> tools = new 
ArrayList<>(current.tools());
+        tools.add(AiToolSpecToOpenAI.function("get_weather", "weather 
forecast", "{\"type\":\"object\"}"));
+        endpoint.setMcpToolState(new McpToolState(
+                tools, Map.of("get_weather", client), Map.of("get_weather", 
"stub"), Set.of(), current.routeTools()));
+    }
+
+    private final class RecordingLifecycleManager implements 
SpanLifecycleManager {
+
+        @Override
+        public Span create(String spanName, String spanKind, Span parent, 
SpanContextPropagationExtractor extractor) {
+            return new RecordingSpan(spanName, parent, new HashMap<>());
+        }
+
+        @Override
+        public void activate(Span span) {
+            // noop
+        }
+
+        @Override
+        public void deactivate(Span span) {
+            // noop
+        }
+
+        @Override
+        public void close(Span span) {
+            synchronized (closedSpans) {
+                closedSpans.add((RecordingSpan) span);
+            }
+        }
+
+        @Override
+        public void inject(Span span, SpanContextPropagationInjector injector, 
boolean includeTracing) {
+            // noop
+        }
+    }
+
+    private record RecordingSpan(String name, Span parent, Map<String, Object> 
tags) implements Span {
+
+        @Override
+        public void log(Map<String, String> fields) {
+            // noop
+        }
+
+        @Override
+        public void setTag(String key, String value) {
+            tags.put(key, value);
+        }
+
+        @Override
+        public void setTag(String key, long value) {
+            tags.put(key, value);
+        }
+
+        @Override
+        public void setTag(String key, double value) {
+            tags.put(key, value);
+        }
+
+        @Override
+        public void setComponent(String component) {
+            tags.put("component", component);
+        }
+
+        @Override
+        public void setError(boolean isError) {
+            tags.put("error", isError);
+        }
+    }
+}

Reply via email to