This is an automated email from the ASF dual-hosted git repository.
wenjin272 pushed a commit to branch main
in repository https://gitbox.apache.org/repos/asf/flink-agents.git
The following commit(s) were added to refs/heads/main by this push:
new f94242bfa [integration][java] Send user images and documents through
the OpenAI Responses API (#1275)
f94242bfa is described below
commit f94242bfa3a7518041fba9d5ed4e1a047b9d50e9
Author: Edson <[email protected]>
AuthorDate: Wed Oct 28 01:55:24 2026 -0400
[integration][java] Send user images and documents through the OpenAI
Responses API (#1275)
Generated-by: Claude Code 2.1.259 (Claude Opus 5.5)
---
.../calling-llms/chat-models/providers.md | 20 ++-
docs/content/docs/development/calling-llms/chat.md | 4 +-
.../openai/OpenAIResponsesModelConnection.java | 3 -
.../chatmodels/openai/OpenAIResponsesUtils.java | 103 ++++++++++--
.../openai/OpenAIResponsesModelConnectionTest.java | 19 ---
.../openai/OpenAIResponsesMultimodalTest.java | 179 +++++++++++++++++++++
6 files changed, 283 insertions(+), 45 deletions(-)
diff --git
a/docs/content/docs/development/calling-llms/chat-models/providers.md
b/docs/content/docs/development/calling-llms/chat-models/providers.md
index 805982c12..fb12cb573 100644
--- a/docs/content/docs/development/calling-llms/chat-models/providers.md
+++ b/docs/content/docs/development/calling-llms/chat-models/providers.md
@@ -51,7 +51,7 @@ model, deployment, and server must also support the requested
input and tool cal
|---|---|---|
| [Alibaba Cloud Model Studio
(DashScope)](#alibaba-cloud-model-studio-dashscope) | Python | No media input;
media blocks are rejected. |
| [OpenAI Chat Completions](#multimodal-input), [Azure OpenAI](#azure-openai),
[vLLM](#vllm) | Java, Python | User messages: images by URL or Base64; WAV/MP3
audio and documents by Base64. See the conversion and model restrictions below.
|
-| [OpenAI Responses](#responses-api) | Java | No media input; media blocks are
rejected. |
+| [OpenAI Responses](#responses-api) | Java | User messages: images by URL or
Base64; documents by URL or Base64. No audio or video. |
| [Anthropic](#anthropic) | Java, Python | User messages: JPEG/PNG/GIF/WebP
images and PDF documents by URL or Base64; plain-text documents by Base64. |
| [Ollama](#ollama) | Java, Python | User messages: Base64 images, sent
separately from text; requires a vision model. |
| [Gemini](#gemini) | Java | User messages: images, audio, video and documents
by URL or Base64. |
@@ -961,10 +961,9 @@ See [Provider Capabilities](#provider-capabilities) for
other integrations.
The OpenAI Responses integration currently has a Java implementation only. To
use OpenAI Responses API from Python agents, see [Cross-Language Resources]({{<
ref "docs/development/create-agent/resources#cross-language-resources" >}}).
{{< /hint >}}
-The Flink Agents Responses integration currently rejects media blocks before
-sending a request. Its capabilities differ from the Chat Completions
integration
-above, even when both connect to OpenAI. This is an integration limit, not a
claim
-that the upstream Responses API lacks multimodal support.
+The Flink Agents Responses integration sends images and documents from user
+messages, but not audio or video; see **Multimodal input** below. Its
capabilities
+differ from the Chat Completions integration above, even when both connect to
OpenAI.
##### OpenAIResponsesModelConnection Parameters
@@ -1040,6 +1039,17 @@ public class MyAgent extends Agent {
{{< /tabs >}}
+**Multimodal input.** The media blocks of a user message are sent as Responses
input content, in block order, next to its text blocks:
+
+| Block | Sent as |
+|---|---|
+| `TextBlock` | an `input_text` part |
+| `ImageBlock` | an `input_image` part: the URL, or a `data:` URI for Base64
data, with `detail: auto` |
+| `DocumentBlock` | an `input_file` part: Base64 data as `file_data` (a
`data:` URI) with the block's `name` as `filename` (`document` when unset), or
a URL as `file_url` |
+| `AudioBlock`, `VideoBlock` | not supported |
+
+A user message with only text blocks is sent as before. Audio, video, and
media in assistant messages or tool results raise
`UnsupportedContentBlockException` before the request is sent. Which image and
file types a model accepts is up to OpenAI.
+
#### Available Models
Visit the [OpenAI Models
documentation](https://platform.openai.com/docs/models) for the complete and
up-to-date list of available chat models.
diff --git a/docs/content/docs/development/calling-llms/chat.md
b/docs/content/docs/development/calling-llms/chat.md
index e5659746f..19beaf489 100644
--- a/docs/content/docs/development/calling-llms/chat.md
+++ b/docs/content/docs/development/calling-llms/chat.md
@@ -294,8 +294,8 @@ for tool results.
Representable content does not imply support by the selected provider
integration
or model. Check [Provider Capabilities]({{< ref
"docs/development/calling-llms/chat-models/providers#provider-capabilities" >}})
before using media. In particular, OpenAI Chat Completions and OpenAI Responses
-have different support in Flink Agents: the Responses integration currently
-rejects media blocks. Media that the integration does not support raises
+have different support in Flink Agents: only Chat Completions sends audio, and
only
+Responses accepts documents by URL. Media that the integration does not
support raises
`UnsupportedContentBlockException` (Java) or `UnsupportedContentBlockError`
(Python)
before the request is sent. Even when the integration accepts the media, the
model service may reject it if the selected model or deployment does not
support it.
diff --git
a/integrations/chat-models/openai/src/main/java/org/apache/flink/agents/integrations/chatmodels/openai/OpenAIResponsesModelConnection.java
b/integrations/chat-models/openai/src/main/java/org/apache/flink/agents/integrations/chatmodels/openai/OpenAIResponsesModelConnection.java
index 3ef2d4232..a349e0987 100644
---
a/integrations/chat-models/openai/src/main/java/org/apache/flink/agents/integrations/chatmodels/openai/OpenAIResponsesModelConnection.java
+++
b/integrations/chat-models/openai/src/main/java/org/apache/flink/agents/integrations/chatmodels/openai/OpenAIResponsesModelConnection.java
@@ -28,7 +28,6 @@ import com.openai.models.responses.ResponseInputItem;
import org.apache.flink.agents.api.chat.messages.ChatMessage;
import org.apache.flink.agents.api.chat.messages.ChatResult;
import org.apache.flink.agents.api.chat.messages.TokenUsage;
-import
org.apache.flink.agents.api.chat.messages.UnsupportedContentBlockException;
import org.apache.flink.agents.api.chat.model.BaseChatModelConnection;
import org.apache.flink.agents.api.resource.ResourceContext;
import org.apache.flink.agents.api.resource.ResourceDescriptor;
@@ -121,8 +120,6 @@ public class OpenAIResponsesModelConnection extends
BaseChatModelConnection {
List<ChatMessage> messages,
List<org.apache.flink.agents.api.tools.Tool> tools,
Map<String, Object> modelParams) {
- // Media blocks are not sent yet; fail rather than drop them (#1059).
- UnsupportedContentBlockException.rejectMedia("OpenAI Responses",
messages);
ResponseCreateParams params = buildRequest(messages, tools,
modelParams);
Response response = client.responses().create(params);
ChatMessage result = OpenAIResponsesUtils.convertResponse(response);
diff --git
a/integrations/chat-models/openai/src/main/java/org/apache/flink/agents/integrations/chatmodels/openai/OpenAIResponsesUtils.java
b/integrations/chat-models/openai/src/main/java/org/apache/flink/agents/integrations/chatmodels/openai/OpenAIResponsesUtils.java
index 1a800cbf2..e6c16b05b 100644
---
a/integrations/chat-models/openai/src/main/java/org/apache/flink/agents/integrations/chatmodels/openai/OpenAIResponsesUtils.java
+++
b/integrations/chat-models/openai/src/main/java/org/apache/flink/agents/integrations/chatmodels/openai/OpenAIResponsesUtils.java
@@ -26,18 +26,26 @@ import com.openai.models.responses.EasyInputMessage;
import com.openai.models.responses.FunctionTool;
import com.openai.models.responses.Response;
import com.openai.models.responses.ResponseFunctionToolCall;
+import com.openai.models.responses.ResponseInputFile;
+import com.openai.models.responses.ResponseInputImage;
import com.openai.models.responses.ResponseInputItem;
import com.openai.models.responses.ResponseOutputItem;
import com.openai.models.responses.ResponseOutputMessage;
import com.openai.models.responses.ResponseReasoningItem;
import com.openai.models.responses.Tool;
+import org.apache.flink.agents.api.chat.messages.Base64Source;
import org.apache.flink.agents.api.chat.messages.ChatMessage;
import org.apache.flink.agents.api.chat.messages.ContentBlock;
+import org.apache.flink.agents.api.chat.messages.DocumentBlock;
+import org.apache.flink.agents.api.chat.messages.ImageBlock;
+import org.apache.flink.agents.api.chat.messages.MediaBlock;
import org.apache.flink.agents.api.chat.messages.MessageRole;
import org.apache.flink.agents.api.chat.messages.ReasoningBlock;
import org.apache.flink.agents.api.chat.messages.TextBlock;
import org.apache.flink.agents.api.chat.messages.ToolCallBlock;
import org.apache.flink.agents.api.chat.messages.ToolResultBlock;
+import
org.apache.flink.agents.api.chat.messages.UnsupportedContentBlockException;
+import org.apache.flink.agents.api.chat.messages.UrlSource;
import org.apache.flink.agents.api.tools.ToolMetadata;
import java.util.ArrayList;
@@ -65,19 +73,22 @@ final class OpenAIResponsesUtils {
return items;
}
+ /**
+ * Converts a message to Responses input items. Only user messages can
carry media: images
+ * become {@code input_image} and documents {@code input_file}, by URL or
Base64 data. Audio and
+ * video, and media in an assistant message or a tool result, throw {@link
+ * UnsupportedContentBlockException}.
+ */
static List<ResponseInputItem> convertSingleMessage(ChatMessage message) {
List<ResponseInputItem> items = new ArrayList<>();
- List<? extends ContentBlock> contentBlocks =
- message.getRole() == MessageRole.TOOL
- ? ((ToolResultBlock)
message.getBlocks().get(0)).getBlocks()
- : message.getBlocks();
- for (ContentBlock block : contentBlocks) {
- if (block instanceof
org.apache.flink.agents.api.chat.messages.MediaBlock) {
- throw new IllegalArgumentException(
- "OpenAI Responses does not support " + block.getType()
+ " content yet");
- }
- }
MessageRole role = message.getRole();
+ if (role != MessageRole.USER) {
+ requireTextOnly(
+ message,
+ role == MessageRole.TOOL
+ ? ((ToolResultBlock)
message.getBlocks().get(0)).getBlocks()
+ : message.getBlocks());
+ }
String content = message.getText();
switch (role) {
@@ -91,12 +102,7 @@ final class OpenAIResponsesUtils {
break;
case USER:
- items.add(
- ResponseInputItem.ofMessage(
- ResponseInputItem.Message.builder()
-
.role(ResponseInputItem.Message.Role.USER)
- .addInputTextContent(content)
- .build()));
+
items.add(ResponseInputItem.ofMessage(convertUserMessage(message, content)));
break;
case ASSISTANT:
@@ -125,6 +131,71 @@ final class OpenAIResponsesUtils {
return items;
}
+ private static ResponseInputItem.Message convertUserMessage(ChatMessage
message, String text) {
+ ResponseInputItem.Message.Builder builder =
+
ResponseInputItem.Message.builder().role(ResponseInputItem.Message.Role.USER);
+ if (message.getBlocks().stream().noneMatch(block -> block instanceof
MediaBlock)) {
+ // Unchanged for text-only messages: one input_text with the text
projection.
+ return builder.addInputTextContent(text).build();
+ }
+ for (ContentBlock block : message.getBlocks()) {
+ if (block instanceof TextBlock) {
+ builder.addInputTextContent(((TextBlock) block).getText());
+ } else if (block instanceof ImageBlock) {
+ builder.addContent(
+ ResponseInputImage.builder()
+ .imageUrl(urlOrDataUri((MediaBlock) block))
+ .detail(ResponseInputImage.Detail.AUTO)
+ .build());
+ } else if (block instanceof DocumentBlock) {
+ builder.addContent(toInputFile((DocumentBlock) block));
+ } else {
+ throw unsupported(block, "the Responses API takes images and
documents only");
+ }
+ }
+ return builder.build();
+ }
+
+ private static ResponseInputFile toInputFile(DocumentBlock document) {
+ ResponseInputFile.Builder file = ResponseInputFile.builder();
+ if (document.getSource() instanceof UrlSource) {
+ file.fileUrl(((UrlSource) document.getSource()).getUrl());
+ if (document.getName() != null) {
+ file.filename(document.getName());
+ }
+ } else {
+ file.fileData(urlOrDataUri(document))
+ .filename(document.getName() != null ? document.getName()
: "document");
+ }
+ return file.build();
+ }
+
+ private static String urlOrDataUri(MediaBlock block) {
+ if (block.getSource() instanceof UrlSource) {
+ return ((UrlSource) block.getSource()).getUrl();
+ }
+ return "data:"
+ + block.getMediaType()
+ + ";base64,"
+ + ((Base64Source) block.getSource()).getData();
+ }
+
+ private static void requireTextOnly(ChatMessage message, List<? extends
ContentBlock> blocks) {
+ for (ContentBlock block : blocks) {
+ if (block instanceof MediaBlock) {
+ throw unsupported(
+ block,
+ "only user messages can carry media, not "
+ + message.getRole().getValue()
+ + " messages");
+ }
+ }
+ }
+
+ private static UnsupportedContentBlockException unsupported(ContentBlock
block, String reason) {
+ return UnsupportedContentBlockException.forBlock("OpenAI Responses",
block, reason);
+ }
+
private static List<ResponseInputItem> convertAssistantMessage(ChatMessage
message) {
List<ResponseInputItem> items = new ArrayList<>();
for (ContentBlock block : message.getBlocks()) {
diff --git
a/integrations/chat-models/openai/src/test/java/org/apache/flink/agents/integrations/chatmodels/openai/OpenAIResponsesModelConnectionTest.java
b/integrations/chat-models/openai/src/test/java/org/apache/flink/agents/integrations/chatmodels/openai/OpenAIResponsesModelConnectionTest.java
index abe574270..2651d5b5d 100644
---
a/integrations/chat-models/openai/src/test/java/org/apache/flink/agents/integrations/chatmodels/openai/OpenAIResponsesModelConnectionTest.java
+++
b/integrations/chat-models/openai/src/test/java/org/apache/flink/agents/integrations/chatmodels/openai/OpenAIResponsesModelConnectionTest.java
@@ -30,8 +30,6 @@ import org.apache.flink.agents.api.chat.messages.ChatMessage;
import org.apache.flink.agents.api.chat.messages.ImageBlock;
import org.apache.flink.agents.api.chat.messages.MessageRole;
import org.apache.flink.agents.api.chat.messages.ReasoningBlock;
-import org.apache.flink.agents.api.chat.messages.TextBlock;
-import
org.apache.flink.agents.api.chat.messages.UnsupportedContentBlockException;
import org.apache.flink.agents.api.chat.model.BaseChatModelConnection;
import org.apache.flink.agents.api.resource.ResourceContext;
import org.apache.flink.agents.api.resource.ResourceDescriptor;
@@ -282,23 +280,6 @@ class OpenAIResponsesModelConnectionTest {
}
}
- @Test
- @DisplayName("Media blocks fail explicitly instead of being dropped")
- void testMediaBlocksFailExplicitly() {
- List<ChatMessage> messages =
- List.of(
- ChatMessage.user(
- List.of(
- TextBlock.of("Describe this"),
- ImageBlock.fromBase64("image/png",
"aGVsbG8="))));
-
- assertThatThrownBy(() -> connection().chat(messages, List.of(), new
HashMap<>()))
- .isInstanceOf(UnsupportedContentBlockException.class)
- .hasMessage(
- "OpenAI Responses cannot send an image block
(image/png, base64 source): this"
- + " integration sends text only.");
- }
-
@Test
void testReasoningAndToolCallRoundTripPreservesOrder() throws Exception {
ObjectMapper mapper = ObjectMappers.jsonMapper();
diff --git
a/integrations/chat-models/openai/src/test/java/org/apache/flink/agents/integrations/chatmodels/openai/OpenAIResponsesMultimodalTest.java
b/integrations/chat-models/openai/src/test/java/org/apache/flink/agents/integrations/chatmodels/openai/OpenAIResponsesMultimodalTest.java
new file mode 100644
index 000000000..cf8469b1a
--- /dev/null
+++
b/integrations/chat-models/openai/src/test/java/org/apache/flink/agents/integrations/chatmodels/openai/OpenAIResponsesMultimodalTest.java
@@ -0,0 +1,179 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.flink.agents.integrations.chatmodels.openai;
+
+import com.fasterxml.jackson.databind.JsonNode;
+import com.openai.core.ObjectMappers;
+import org.apache.flink.agents.api.chat.messages.AudioBlock;
+import org.apache.flink.agents.api.chat.messages.Base64Source;
+import org.apache.flink.agents.api.chat.messages.ChatMessage;
+import org.apache.flink.agents.api.chat.messages.ContentBlock;
+import org.apache.flink.agents.api.chat.messages.DocumentBlock;
+import org.apache.flink.agents.api.chat.messages.ImageBlock;
+import org.apache.flink.agents.api.chat.messages.TextBlock;
+import org.apache.flink.agents.api.chat.messages.ToolResultBlock;
+import
org.apache.flink.agents.api.chat.messages.UnsupportedContentBlockException;
+import org.apache.flink.agents.api.chat.messages.UrlSource;
+import org.apache.flink.agents.api.chat.messages.VideoBlock;
+import org.apache.flink.agents.api.resource.ResourceContext;
+import org.apache.flink.agents.api.resource.ResourceDescriptor;
+import org.junit.jupiter.api.DisplayName;
+import org.junit.jupiter.api.Test;
+
+import java.util.List;
+import java.util.Map;
+
+import static org.assertj.core.api.Assertions.assertThat;
+import static org.assertj.core.api.Assertions.assertThatThrownBy;
+
+/** Tests how content blocks become OpenAI Responses API input content. */
+class OpenAIResponsesMultimodalTest {
+
+ private static final String URL = "https://example.com/cat.png?sig=secret";;
+ private static final String BASE64 = "aGVsbG8="; // "hello"
+
+ /** The message's input item, as the SDK serializes it. */
+ private static JsonNode wire(ChatMessage message) {
+ List<com.openai.models.responses.ResponseInputItem> items =
+ OpenAIResponsesUtils.convertSingleMessage(message);
+ assertThat(items).hasSize(1);
+ return ObjectMappers.jsonMapper().valueToTree(items.get(0));
+ }
+
+ @Test
+ @DisplayName("A text-only user message is sent as before, as one
input_text")
+ void testTextOnlyUserMessageIsUnchanged() {
+ JsonNode content =
wire(ChatMessage.user(List.of(TextBlock.of("hi")))).get("content");
+
+ assertThat(content).hasSize(1);
+ assertThat(content.at("/0/type").asText()).isEqualTo("input_text");
+ assertThat(content.at("/0/text").asText()).isEqualTo("hi");
+ }
+
+ @Test
+ @DisplayName("Images become input_image in block order, by URL or as a
data URI")
+ void testImagesBecomeInputImages() {
+ JsonNode content =
+ wire(ChatMessage.user(
+ List.of(
+ TextBlock.of("Compare"),
+ ImageBlock.fromUrl("image/png", URL),
+ ImageBlock.fromBase64("image/png",
BASE64))))
+ .get("content");
+
+ assertThat(content).hasSize(3);
+ assertThat(content.at("/0/type").asText()).isEqualTo("input_text");
+ assertThat(content.at("/0/text").asText()).isEqualTo("Compare");
+ assertThat(content.at("/1/type").asText()).isEqualTo("input_image");
+ assertThat(content.at("/1/image_url").asText()).isEqualTo(URL);
+ assertThat(content.at("/1/detail").asText()).isEqualTo("auto");
+ assertThat(content.at("/2/image_url").asText())
+ .isEqualTo("data:image/png;base64," + BASE64);
+ }
+
+ @Test
+ @DisplayName(
+ "Documents become input_file: Base64 as file_data with a filename,
URLs as file_url")
+ void testDocumentsBecomeInputFiles() {
+ JsonNode content =
+ wire(ChatMessage.user(
+ List.of(
+ new DocumentBlock(
+ "application/pdf",
+ new Base64Source(BASE64),
+ "report.pdf",
+ null,
+ null),
+
DocumentBlock.fromBase64("application/pdf", BASE64),
+ DocumentBlock.fromUrl(
+ "application/pdf",
"https://example.com/a.pdf";),
+ new DocumentBlock(
+ "application/pdf",
+ new
UrlSource("https://example.com/b.pdf";),
+ "b.pdf",
+ null,
+ null))))
+ .get("content");
+
+ assertThat(content.at("/0/type").asText()).isEqualTo("input_file");
+ assertThat(content.at("/0/file_data").asText())
+ .isEqualTo("data:application/pdf;base64," + BASE64);
+ assertThat(content.at("/0/filename").asText()).isEqualTo("report.pdf");
+ assertThat(content.at("/1/filename").asText()).isEqualTo("document");
+
assertThat(content.at("/2/file_url").asText()).isEqualTo("https://example.com/a.pdf";);
+ assertThat(content.get(2).has("file_data")).isFalse();
+ assertThat(content.get(2).has("filename")).isFalse();
+
assertThat(content.at("/3/file_url").asText()).isEqualTo("https://example.com/b.pdf";);
+ assertThat(content.at("/3/filename").asText()).isEqualTo("b.pdf");
+ }
+
+ @Test
+ @DisplayName("Audio and video fail without leaking the source")
+ void testUnsupportedBlocksFailExplicitly() {
+ for (ContentBlock block :
+ List.<ContentBlock>of(
+ AudioBlock.fromBase64("audio/wav", BASE64),
+ VideoBlock.fromUrl("video/mp4", URL))) {
+ assertThatThrownBy(() ->
wire(ChatMessage.user(List.of(TextBlock.of("hi"), block))))
+ .isInstanceOf(UnsupportedContentBlockException.class)
+ .hasMessageStartingWith("OpenAI Responses cannot send a")
+ .hasMessageContaining(" " + block.getType() + " block")
+ .hasMessageNotContaining("secret")
+ .hasMessageNotContaining(BASE64);
+ }
+ }
+
+ @Test
+ @DisplayName("Media in assistant messages and tool results fails
explicitly")
+ void testMediaOutsideUserMessagesFails() {
+ List<ContentBlock> blocks =
+ List.of(TextBlock.of("see"), ImageBlock.fromUrl("image/png",
URL));
+ ChatMessage assistant = ChatMessage.assistant(blocks);
+ ChatMessage tool =
+ ChatMessage.tool(
+ new ToolResultBlock(
+ "call-1",
+ List.of(TextBlock.of("see"),
ImageBlock.fromUrl("image/png", URL)),
+ false));
+
+ for (ChatMessage message : List.of(assistant, tool)) {
+ assertThatThrownBy(() ->
OpenAIResponsesUtils.convertSingleMessage(message))
+ .isInstanceOf(UnsupportedContentBlockException.class)
+ .hasMessageContaining("only user messages can carry
media");
+ }
+ }
+
+ @Test
+ @DisplayName("chat() surfaces media errors with their own types, before
any request")
+ void testChatKeepsMediaErrorTypes() {
+ ResourceDescriptor descriptor =
+ ResourceDescriptor.Builder.newBuilder(
+ OpenAIResponsesModelConnection.class.getName())
+ .addInitialArgument("api_key", "test-key")
+ .addInitialArgument("model", "gpt-4o")
+ .build();
+ OpenAIResponsesModelConnection connection =
+ new OpenAIResponsesModelConnection(
+ descriptor, ResourceContext.fromGetResource((a, b) ->
null));
+ ChatMessage message =
ChatMessage.user(List.of(AudioBlock.fromBase64("audio/wav", BASE64)));
+
+ assertThatThrownBy(() -> connection.chat(List.of(message), List.of(),
Map.of()))
+ .isExactlyInstanceOf(UnsupportedContentBlockException.class);
+ }
+}
![https://example.com/cat.png?sig=secret"](https://example.com/cat.png?sig=secret"){kind=link}