This is an automated email from the ASF dual-hosted git repository.
wenjin272 pushed a commit to branch main
in repository https://gitbox.apache.org/repos/asf/flink-agents.git
The following commit(s) were added to refs/heads/main by this push:
new 62c65b0e9 [integrations][java][python] Skip native structured output
for a Map, JsonNode or dict field (#1265)
62c65b0e9 is described below
commit 62c65b0e928668579f0fd7ae06581646f71d3cca
Author: Weiqing Yang <[email protected]>
AuthorDate: Sun Oct 18 23:24:55 2026 -0700
[integrations][java][python] Skip native structured output for a Map,
JsonNode or dict field (#1265)
Generated-by: GitHub Copilot CLI 1.0.98 (Claude Opus 5.5)
---
.../development/calling-llms/structured-output.md | 15 +-
.../anthropic/AnthropicChatModelConnection.java | 44 ++++--
.../AnthropicChatModelConnectionTest.java | 40 ++++-
.../chatmodels/common/PojoJsonSchemaGenerator.java | 44 +++++-
.../common/PojoJsonSchemaGeneratorTest.java | 167 +++++++++++++++++++++
.../gemini/GeminiChatModelConnection.java | 49 ++++--
.../gemini/GeminiChatModelConnectionTest.java | 32 +++-
.../openai/AzureOpenAIChatModelConnection.java | 20 ++-
.../openai/OpenAIChatCompletionsUtils.java | 24 ++-
.../openai/OpenAICompletionsConnection.java | 15 +-
.../openai/AzureOpenAIChatModelConnectionTest.java | 39 ++++-
.../openai/OpenAICompletionsConnectionTest.java | 34 ++++-
.../openai/VLLMChatModelConnectionTest.java | 13 ++
.../api/agents/tests/test_output_schema_render.py | 132 +++++++++++++++-
python/flink_agents/api/agents/types.py | 51 +++++++
.../chat_models/anthropic/anthropic_chat_model.py | 31 ++--
.../tests/test_anthropic_response_parsing.py | 51 +++++--
.../chat_models/azure/azure_openai_chat_model.py | 25 ++-
.../test_azure_openai_native_structured_output.py | 41 +++--
.../chat_models/openai/openai_chat_model.py | 18 ++-
.../tests/test_openai_native_structured_output.py | 41 +++--
.../chat_models/vllm/tests/test_vllm_chat_model.py | 27 ++++
.../chat_models/vllm/vllm_chat_model.py | 22 ++-
23 files changed, 867 insertions(+), 108 deletions(-)
diff --git a/docs/content/docs/development/calling-llms/structured-output.md
b/docs/content/docs/development/calling-llms/structured-output.md
index d66b30df4..d566fb910 100644
--- a/docs/content/docs/development/calling-llms/structured-output.md
+++ b/docs/content/docs/development/calling-llms/structured-output.md
@@ -209,13 +209,13 @@ Native path: ChatRequestEvent → model call (tool loop,
no schema)
| Provider | Languages | Recommended models (`AUTO` goes native) | Blocked
(besides a non-typed-class schema) |
|---|---|---|---|
-| OpenAI | Java, Python | `gpt-4o-mini`, `gpt-4o-search-preview`, `gpt-4.1`,
`gpt-5` and `o3`, `o4-mini` families (names that start with these), plus
`gpt-4o`, `gpt-4o-2024-08-06`, `gpt-4o-2024-11-20`, `o1`, `o1-2024-12-17`.
Audio, realtime, TTS and transcribe variants are excluded. | |
+| OpenAI | Java, Python | `gpt-4o-mini`, `gpt-4o-search-preview`, `gpt-4.1`,
`gpt-5` and `o3`, `o4-mini` families (names that start with these), plus
`gpt-4o`, `gpt-4o-2024-08-06`, `gpt-4o-2024-11-20`, `o1`, `o1-2024-12-17`.
Audio, realtime, TTS and transcribe variants are excluded. | The schema has an
open-keyed field |
| OpenAI Responses | Java | none | Always. Use the prompt path. |
-| Azure OpenAI | Java, Python | The model named by `model_of_azure_deployment`
is one of `gpt-5.1`, `gpt-5.1-chat`, `gpt-5`, `gpt-5-mini`, `gpt-5-nano`, `o3`,
`o3-mini`, `o4-mini`, `o1`, `gpt-4o-mini`, `gpt-4.1`, `gpt-4.1-mini`,
`gpt-4.1-nano`. | `api_version` older than `2024-08-01`, or not a date |
-| vLLM | Java, Python | Any model | Same as OpenAI |
-| Anthropic | Java, Python | `claude-opus-4-6`, `-4-7`, `-4-8`, `-5`;
`claude-sonnet-4-6`, `-5`; `claude-fable-5`; `claude-mythos-5`;
`claude-mythos-preview`; and the `claude-opus-4-5`, `claude-sonnet-4-5`,
`claude-haiku-4-5` aliases and their dated versions | You pass `output_config`
in `additional_kwargs` |
+| Azure OpenAI | Java, Python | The model named by `model_of_azure_deployment`
is one of `gpt-5.1`, `gpt-5.1-chat`, `gpt-5`, `gpt-5-mini`, `gpt-5-nano`, `o3`,
`o3-mini`, `o4-mini`, `o1`, `gpt-4o-mini`, `gpt-4.1`, `gpt-4.1-mini`,
`gpt-4.1-nano`. | `api_version` older than `2024-08-01`, or not a date. The
schema has an open-keyed field. |
+| vLLM | Java, Python | Any model | Java only: the schema has an open-keyed
field |
+| Anthropic | Java, Python | `claude-opus-4-6`, `-4-7`, `-4-8`, `-5`;
`claude-sonnet-4-6`, `-5`; `claude-fable-5`; `claude-mythos-5`;
`claude-mythos-preview`; and the `claude-opus-4-5`, `claude-sonnet-4-5`,
`claude-haiku-4-5` aliases and their dated versions | You pass `output_config`
in `additional_kwargs`. The schema has an open-keyed field. |
| Amazon Bedrock | Java | A fixed list of Converse models (a regional or
global inference-profile prefix is ignored). ARNs are never recommended. | |
-| Gemini | Java | `gemini-*` text models. Image, TTS, audio, live, transcribe,
embedding and omni variants are excluded. | The request binds tools |
+| Gemini | Java | `gemini-*` text models. Image, TTS, audio, live, transcribe,
embedding and omni variants are excluded. | The request binds tools. The schema
has a `JsonNode` field. |
| Ollama | Java, Python | Any model | |
| watsonx | Java, Python | Any model | |
| DashScope | Python | `qwen3.7-max`, `qwen3.7-max-preview`,
`qwen3.7-max-2026-05-17`, `qwen3.7-max-2026-05-20` | |
@@ -226,6 +226,7 @@ Notes:
- **Gemini** blocks native output when the request binds tools. The conversion
call is tool-free, so this only affects direct connection calls. A newer model
that does not support the schema field fails with `400 INVALID_ARGUMENT`.
Constraints such as `pattern` and `minLength` are ignored by the server.
- **Ollama** needs a server at v0.5.0 or later. Ollama Cloud and the MLX
runner accept the schema but do not enforce it.
- **Bedrock** rejects recursive schemas, and map values are untyped.
+- **An open-keyed field blocks native output on OpenAI, Azure OpenAI,
Anthropic and Java vLLM.** That is a field whose keys are not fixed in advance,
at any depth: a Java `Map` or Jackson `JsonNode` field, or a Python `dict`
field or a Pydantic model with `extra="allow"`. These providers need every
object in the schema to list its keys. Gemini blocks only a `JsonNode` field,
because it can type a `Map`'s values. Python vLLM keeps the field native,
because vLLM's guided decoding enforces [...]
- **A native parameter you set yourself conflicts.** Do not put
`response_format` (OpenAI-compatible providers) or `output_config` (Anthropic)
in `additional_kwargs` or model parameters when you use an output schema. Azure
OpenAI, watsonx and DashScope raise an error when the schema would overwrite
your value. Anthropic treats the request as blocked.
- **Cross-language setups never go native.** A chat model from the other
language, reached through a [cross-language resource]({{< ref
"docs/development/calling-llms/chat-models#using-cross-language-providers"
>}}), cannot carry a schema across the bridge. `AUTO` and `PROMPT` use the
prompt path, and `NATIVE` raises. To use a provider natively, declare the agent
in that provider's language.
@@ -314,12 +315,12 @@ public enum Sentiment {
### Java POJOs
- **Descriptions and required fields.** `@JsonPropertyDescription` and
`@JsonClassDescription` are sent in the native schema, and every field is
required (an `Optional` field is required but nullable on OpenAI, Azure OpenAI
and Anthropic). The ReAct Agent's *prompt text* is rendered by a different
generator, which has no descriptions, and lists a field as required only when
it has `@JsonProperty(required = true)`. The prompt path therefore describes
the schema less fully than the native path.
-- **`Map` fields.** On OpenAI, Azure OpenAI, Anthropic and vLLM, a
`Map<String, V>` field becomes an empty object schema that allows no keys, so
the model can only return `{}`. Avoid `Map` fields with these providers, or use
a nested class. Gemini, Ollama and watsonx type the map values.
+- **`Map` and `JsonNode` fields.** On OpenAI, Azure OpenAI, Anthropic and
vLLM, a schema with either field uses the prompt path under `AUTO` and fails
under `NATIVE`. The native schema would describe the field as an empty object
that allows no keys, so the model could only return `{}`. Gemini keeps a `Map`
native and types its values, but blocks a `JsonNode` field the same way. Ollama
and watsonx type the map values. To stay native everywhere, use a nested class.
- **Recursive types.** The ReAct Agent rejects a self-referential POJO at
construction time. Bedrock also rejects a recursive schema.
### Python Models
-The schema comes from Pydantic's `model_json_schema()`, so field descriptions
and defaults are included. A model that Pydantic cannot express raises a
`TypeError` naming the schema when it is rendered.
+The schema comes from Pydantic's `model_json_schema()`, so field descriptions
and defaults are included. A model that Pydantic cannot express raises a
`TypeError` naming the schema when it is rendered. A `dict` field, or a model
with `extra="allow"`, blocks native output on OpenAI, Azure OpenAI and
Anthropic, as a `Map` does in Java. Use a nested model to stay native there.
vLLM keeps the field native.
## Next Steps
diff --git
a/integrations/chat-models/anthropic/src/main/java/org/apache/flink/agents/integrations/chatmodels/anthropic/AnthropicChatModelConnection.java
b/integrations/chat-models/anthropic/src/main/java/org/apache/flink/agents/integrations/chatmodels/anthropic/AnthropicChatModelConnection.java
index a9498d650..f7df5b32f 100644
---
a/integrations/chat-models/anthropic/src/main/java/org/apache/flink/agents/integrations/chatmodels/anthropic/AnthropicChatModelConnection.java
+++
b/integrations/chat-models/anthropic/src/main/java/org/apache/flink/agents/integrations/chatmodels/anthropic/AnthropicChatModelConnection.java
@@ -28,6 +28,7 @@ import com.anthropic.models.messages.Model;
import com.anthropic.models.messages.OutputConfig;
import com.anthropic.models.messages.TextBlockParam;
import com.anthropic.models.messages.Tool;
+import com.fasterxml.jackson.databind.JsonNode;
import com.github.victools.jsonschema.generator.Option;
import org.apache.flink.agents.api.chat.messages.ChatMessage;
import org.apache.flink.agents.api.chat.messages.ChatResult;
@@ -202,11 +203,13 @@ public class AnthropicChatModelConnection extends
BaseChatModelConnection {
* Whether a request built from these inputs would carry a derived {@code
output_config}, the
* effective model's capability aside.
*
- * <p>Two conditions. Only a POJO {@link Class} has a native translation
here; a {@code
- * RowTypeInfo} wrapped in {@code OutputSchema}, or any other form, has
none. And a
- * caller-supplied {@code output_config} wins: it is the caller being
explicit about the exact
- * parameter the native branch writes, so the schema falls back to prompt
engineering rather
- * than the two competing on one request.
+ * <p>Two conditions. Only a POJO {@link Class} has a native translation
here, and only one
+ * whose schema reaches no {@code Map} or {@code JsonNode}: the native
schema closes every
+ * object, so the model could return only {@code {}} for one. A {@code
RowTypeInfo} wrapped in
+ * {@code OutputSchema}, or any other form, has none. And a
caller-supplied {@code
+ * output_config} wins: it is the caller being explicit about the exact
parameter the native
+ * branch writes, so the schema falls back to prompt engineering rather
than the two competing
+ * on one request.
*
* <p>The tools are not read; a bound tool does not stop this connection
sending a schema.
*
@@ -216,14 +219,16 @@ public class AnthropicChatModelConnection extends
BaseChatModelConnection {
* additional_kwargs}. These must be the parameters as they arrived:
{@code buildRequest}
* removes that key from its own copy before the native branch runs,
so the stripped copy
* would report every caller's {@code output_config} absent
- * @return true if {@code outputSchema} is a POJO {@link Class} and the
caller supplied no
- * {@code output_config} of its own
+ * @return true if {@code outputSchema} is a POJO {@link Class} that
reaches no {@code Map} or
+ * {@code JsonNode} and the caller supplied no {@code output_config}
of its own
*/
private boolean canApplyNativeStructuredOutput(
Object outputSchema,
List<org.apache.flink.agents.api.tools.Tool> tools,
Map<String, Object> modelParams) {
- return outputSchema instanceof Class &&
!carriesCallerOutputConfig(modelParams);
+ return outputSchema instanceof Class
+ && !carriesCallerOutputConfig(modelParams)
+ && canDescribeNatively((Class<?>) outputSchema);
}
/**
@@ -405,6 +410,26 @@ public class AnthropicChatModelConnection extends
BaseChatModelConnection {
return override instanceof Number ? override : topLevel;
}
+ // The native schema closes every object it describes, as structured
outputs require. A Map's
+ // keys are not known in advance, and a Jackson JsonNode holds any JSON
value, which the
+ // generator describes as an object. Closed, either admits only {}, so a
model held to the
+ // schema returns that and the data is lost.
+ private static final Option[] NATIVE_SCHEMA_OPTIONS = {
+ Option.FORBIDDEN_ADDITIONAL_PROPERTIES_BY_DEFAULT
+ };
+
+ private static final List<Class<?>> TYPES_WITHOUT_NATIVE_FORM =
+ List.of(Map.class, JsonNode.class);
+
+ /**
+ * Whether the native {@code output_config} can state {@code
schemaClass}'s contract, which it
+ * cannot once the schema reaches a {@code Map} or a {@code JsonNode}.
+ */
+ private static boolean canDescribeNatively(Class<?> schemaClass) {
+ return !PojoJsonSchemaGenerator.reachesAny(
+ schemaClass, TYPES_WITHOUT_NATIVE_FORM, NATIVE_SCHEMA_OPTIONS);
+ }
+
/**
* Derives the native {@code output_config} for a POJO output schema class.
*
@@ -415,8 +440,7 @@ public class AnthropicChatModelConnection extends
BaseChatModelConnection {
*/
private static OutputConfig toNativeOutputConfig(Class<?> schemaClass) {
Map<String, JsonValue> schema = new LinkedHashMap<>();
- PojoJsonSchemaGenerator.generateAllRequired(
- schemaClass,
Option.FORBIDDEN_ADDITIONAL_PROPERTIES_BY_DEFAULT)
+ PojoJsonSchemaGenerator.generateAllRequired(schemaClass,
NATIVE_SCHEMA_OPTIONS)
.fields()
.forEachRemaining(
member ->
diff --git
a/integrations/chat-models/anthropic/src/test/java/org/apache/flink/agents/integrations/chatmodels/anthropic/AnthropicChatModelConnectionTest.java
b/integrations/chat-models/anthropic/src/test/java/org/apache/flink/agents/integrations/chatmodels/anthropic/AnthropicChatModelConnectionTest.java
index 6ca83790e..46d10c63a 100644
---
a/integrations/chat-models/anthropic/src/test/java/org/apache/flink/agents/integrations/chatmodels/anthropic/AnthropicChatModelConnectionTest.java
+++
b/integrations/chat-models/anthropic/src/test/java/org/apache/flink/agents/integrations/chatmodels/anthropic/AnthropicChatModelConnectionTest.java
@@ -384,6 +384,18 @@ class AnthropicChatModelConnectionTest {
public Answer answer;
}
+ /** Output schema fixture with a {@code Map} field, whose keys are not
known in advance. */
+ public static class Tally {
+ public String label;
+ public Map<String, Integer> counts;
+ }
+
+ /** Output schema fixture with a {@code JsonNode} field, which holds any
JSON value. */
+ public static class Note {
+ public String title;
+ public JsonNode body;
+ }
+
private static Map<String, Object> paramsWithModel(String model, Object
jsonPrefill) {
Map<String, Object> params = params(jsonPrefill);
params.put("model", model);
@@ -497,6 +509,30 @@ class AnthropicChatModelConnectionTest {
assertThat(build(CAPABLE_MODEL, "not-a-class",
null).params.outputConfig()).isEmpty();
}
+ @Test
+ @DisplayName("a schema reaching a Map or a JsonNode keeps the prompt
fallback")
+ void testOpenObjectSchemaKeepsFallback() {
+ // The native schema closes every object, so it would admit only {}
for the map or the
+ // tree and the model would return that. The prompt path keeps the
field's contents, and
+ // the json_prefill it relies on with it.
+ for (Class<?> schema : List.of(Tally.class, Note.class)) {
+ assertThat(build(CAPABLE_MODEL, schema,
null).params.outputConfig())
+ .as(schema.getSimpleName())
+ .isEmpty();
+ assertThat(build(INCAPABLE_MODEL, schema, true).jsonPrefillApplied)
+ .as(schema.getSimpleName())
+ .isTrue();
+ assertThat(
+ connection()
+ .supportsNativeStructuredOutput(
+ schema,
+ List.of(),
+ paramsWithModel(CAPABLE_MODEL,
null)))
+ .as(schema.getSimpleName())
+ .isEqualTo(NativeStructuredOutputSupport.INFEASIBLE);
+ }
+ }
+
@Test
@DisplayName("the native path adds no anthropic-beta header")
void testNativePathSendsNoBetaHeader() {
@@ -1171,7 +1207,9 @@ class AnthropicChatModelConnectionTest {
AnthropicChatModelConnection connection =
new AnthropicChatModelConnection(descriptor(configured),
NOOP);
- for (Object schema : Arrays.asList(Answer.class, "row<name
STRING>", null)) {
+ for (Object schema :
+ Arrays.asList(
+ Answer.class, Tally.class, Note.class, "row<name
STRING>", null)) {
for (boolean callerConfig : List.of(false, true)) {
for (List<Tool> tools :
Arrays.asList(List.<Tool>of(), List.<Tool>of(new
StubTool()), null)) {
diff --git
a/integrations/chat-models/common/src/main/java/org/apache/flink/agents/integrations/chatmodels/common/PojoJsonSchemaGenerator.java
b/integrations/chat-models/common/src/main/java/org/apache/flink/agents/integrations/chatmodels/common/PojoJsonSchemaGenerator.java
index b849e05a7..f3e7d4db9 100644
---
a/integrations/chat-models/common/src/main/java/org/apache/flink/agents/integrations/chatmodels/common/PojoJsonSchemaGenerator.java
+++
b/integrations/chat-models/common/src/main/java/org/apache/flink/agents/integrations/chatmodels/common/PojoJsonSchemaGenerator.java
@@ -36,9 +36,11 @@ import
com.github.victools.jsonschema.generator.impl.PropertySortUtils;
import com.github.victools.jsonschema.module.jackson.JacksonModule;
import com.github.victools.jsonschema.module.jackson.JacksonOption;
+import java.util.Collection;
import java.util.LinkedHashSet;
import java.util.Optional;
import java.util.Set;
+import java.util.concurrent.atomic.AtomicBoolean;
import java.util.function.Predicate;
/** Derives the JSON Schema that chat model connections send as a native
output schema. */
@@ -105,8 +107,48 @@ public final class PojoJsonSchemaGenerator {
return generate(type, field -> true, options);
}
+ /**
+ * Reports whether the schema generated for {@code type} reaches a type
that is an instance of
+ * one of {@code targets}.
+ *
+ * <p>The check runs the same generation as {@link #generate}, so it
reaches exactly the types
+ * that schema describes: {@code type} itself, the type of every field,
the element and value
+ * types the options describe, and every subtype Jackson resolves, but not
a field Jackson
+ * ignores. A caller uses it to find a type its provider's schema cannot
state before sending a
+ * schema that misstates it.
+ *
+ * @param type the class a response is deserialized into
+ * @param targets the types to look for; a subtype or implementation of
one counts
+ * @param options the options the caller generates its schema with. They
decide what the schema
+ * describes: a map's value type, for example, is reached only under
{@link
+ * Option#MAP_VALUES_AS_ADDITIONAL_PROPERTIES}
+ * @return true if the schema reaches an instance of one of {@code targets}
+ */
+ public static boolean reachesAny(
+ Class<?> type, Collection<Class<?>> targets, Option... options) {
+ AtomicBoolean reached = new AtomicBoolean();
+ SchemaGeneratorConfigBuilder builder = configure(field -> true,
options);
+ // This provider is offered each type the schema describes. It returns
null, so generation
+ // proceeds as configured.
+ builder.forTypesInGeneral()
+ .withCustomDefinitionProvider(
+ (javaType, context) -> {
+ if
(targets.stream().anyMatch(javaType::isInstanceOf)) {
+ reached.set(true);
+ }
+ return null;
+ });
+ new SchemaGenerator(builder.build()).generateSchema(type);
+ return reached.get();
+ }
+
private static ObjectNode generate(
Class<?> type, Predicate<FieldScope> requiredCheck, Option...
options) {
+ return new SchemaGenerator(configure(requiredCheck,
options).build()).generateSchema(type);
+ }
+
+ private static SchemaGeneratorConfigBuilder configure(
+ Predicate<FieldScope> requiredCheck, Option... options) {
SchemaGeneratorConfigBuilder builder =
new SchemaGeneratorConfigBuilder(
SchemaVersion.DRAFT_2020_12, OptionPreset.PLAIN_JSON);
@@ -122,7 +164,7 @@ public final class PojoJsonSchemaGenerator {
builder.forTypesInGeneral()
.withPropertySorter(PropertySortUtils.SORT_PROPERTIES_FIELDS_BEFORE_METHODS);
builder.forFields().withRequiredCheck(requiredCheck);
- return new SchemaGenerator(builder.build()).generateSchema(type);
+ return builder;
}
private static CustomDefinition enumDefinition(
diff --git
a/integrations/chat-models/common/src/test/java/org/apache/flink/agents/integrations/chatmodels/common/PojoJsonSchemaGeneratorTest.java
b/integrations/chat-models/common/src/test/java/org/apache/flink/agents/integrations/chatmodels/common/PojoJsonSchemaGeneratorTest.java
index 9bcace4a5..7b12ae6be 100644
---
a/integrations/chat-models/common/src/test/java/org/apache/flink/agents/integrations/chatmodels/common/PojoJsonSchemaGeneratorTest.java
+++
b/integrations/chat-models/common/src/test/java/org/apache/flink/agents/integrations/chatmodels/common/PojoJsonSchemaGeneratorTest.java
@@ -20,9 +20,12 @@ package
org.apache.flink.agents.integrations.chatmodels.common;
import com.fasterxml.jackson.annotation.JsonFormat;
import com.fasterxml.jackson.annotation.JsonIgnore;
import com.fasterxml.jackson.annotation.JsonProperty;
+import com.fasterxml.jackson.annotation.JsonSubTypes;
+import com.fasterxml.jackson.annotation.JsonTypeInfo;
import com.fasterxml.jackson.annotation.JsonValue;
import com.fasterxml.jackson.databind.JsonNode;
import com.fasterxml.jackson.databind.ObjectMapper;
+import com.fasterxml.jackson.databind.node.ArrayNode;
import com.fasterxml.jackson.databind.node.ObjectNode;
import com.github.victools.jsonschema.generator.Option;
import org.junit.jupiter.api.DisplayName;
@@ -30,6 +33,7 @@ import org.junit.jupiter.api.Test;
import java.math.BigDecimal;
import java.util.ArrayList;
+import java.util.HashMap;
import java.util.List;
import java.util.Map;
import java.util.Optional;
@@ -105,6 +109,89 @@ class PojoJsonSchemaGeneratorTest {
}
}
+ /** Output schema fixture reaching a map only through a nested class. */
+ public static class Wrapper {
+ public Report report;
+ }
+
+ /** Output schema fixture reaching maps as list elements. */
+ public static class Rows {
+ public List<Map<String, String>> rows;
+ }
+
+ /** Output schema fixture reaching maps as array elements. */
+ public static class Cells {
+ public Map<String, String>[] cells;
+ }
+
+ /** Output schema fixture reaching a concrete map implementation inside an
Optional. */
+ public static class Lookup {
+ public Optional<HashMap<String, Profile>> byName;
+ }
+
+ /** Output schema fixture reaching a map only through a subtype Jackson
resolves. */
+ public static class Zoo {
+ public Animal animal;
+ }
+
+ /** Polymorphic base type whose subtypes Jackson names in the payload. */
+ @JsonTypeInfo(use = JsonTypeInfo.Id.NAME, property = "kind")
+ @JsonSubTypes({
+ @JsonSubTypes.Type(value = Parrot.class, name = "parrot"),
+ @JsonSubTypes.Type(value = Snake.class, name = "snake")
+ })
+ public abstract static class Animal {
+ public String name;
+ }
+
+ /** Subtype carrying a map. */
+ public static class Parrot extends Animal {
+ public Map<String, Integer> phrases;
+ }
+
+ /** Subtype carrying no map. */
+ public static class Snake extends Animal {
+ public int length;
+ }
+
+ /** Output schema fixture holding any JSON value. */
+ public static class AnyTree {
+ public JsonNode value;
+ }
+
+ /** Output schema fixture holding a JSON array. */
+ public static class ArrayTree {
+ public ArrayNode values;
+ }
+
+ /** Output schema fixture holding JSON values as list elements. */
+ public static class TreeList {
+ public List<JsonNode> values;
+ }
+
+ /** Output schema fixture whose only map is a field Jackson ignores. */
+ public static class IgnoredMap {
+ @JsonIgnore public Map<String, String> internal;
+
+ public String name;
+ }
+
+ /** Output schema fixture referring back to itself. */
+ public static class Chain {
+ public String value;
+ public Chain next;
+ }
+
+ /** Output schema fixture with a field named like the keyword that types a
map's values. */
+ public static class Keyword {
+ public String additionalProperties;
+ }
+
+ /** Output schema fixture whose only JSON tree is a map's value type. */
+ public static class Catalog {
+ public Map<String, JsonNode> entries;
+ }
+
/**
* Output schema fixture for enums the Jackson module's flattened enums
misstate: one maps only
* some of its constants through {@code @JsonProperty}, and one maps its
constants to numbers
@@ -402,6 +489,86 @@ class PojoJsonSchemaGeneratorTest {
assertThat(withoutOption.has("additionalProperties")).isFalse();
}
+ @Test
+ @DisplayName("A map or JSON tree is reached at any depth the schema
describes")
+ void reachesMapsAndTreesAtAnyDepth() {
+ for (Class<?> type :
+ List.of(
+ Report.class,
+ Wrapper.class,
+ Rows.class,
+ Cells.class,
+ Lookup.class,
+ Zoo.class,
+ HashMap.class,
+ AnyTree.class,
+ ArrayTree.class,
+ TreeList.class)) {
+ assertThat(
+ PojoJsonSchemaGenerator.reachesAny(
+ type,
+ List.of(Map.class, JsonNode.class),
+
Option.FORBIDDEN_ADDITIONAL_PROPERTIES_BY_DEFAULT))
+ .as(type.getSimpleName())
+ .isTrue();
+ }
+ }
+
+ @Test
+ @DisplayName("A type the schema does not describe is not reached")
+ void doesNotReachTypesTheSchemaLeavesOut() {
+ // IgnoredMap declares a map that Jackson never reads, so the schema
leaves it out, and
+ // Keyword only names a field after the keyword a map's value schema
is written under.
+ for (Class<?> type :
+ List.of(
+ Profile.class,
+ Ticket.class,
+ IgnoredMap.class,
+ Chain.class,
+ Keyword.class,
+ Snake.class)) {
+ assertThat(
+ PojoJsonSchemaGenerator.reachesAny(
+ type,
+ List.of(Map.class, JsonNode.class),
+
Option.FORBIDDEN_ADDITIONAL_PROPERTIES_BY_DEFAULT))
+ .as(type.getSimpleName())
+ .isFalse();
+ }
+ }
+
+ @Test
+ @DisplayName("A map's value type is reached only under the option that
describes it")
+ void reachesMapValuesOnlyUnderTheirOption() {
+ List<Class<?>> trees = List.of(JsonNode.class);
+
+ assertThat(PojoJsonSchemaGenerator.reachesAny(Catalog.class,
trees)).isFalse();
+ assertThat(
+ PojoJsonSchemaGenerator.reachesAny(
+ Catalog.class, trees,
Option.MAP_VALUES_AS_ADDITIONAL_PROPERTIES))
+ .isTrue();
+ }
+
+ /** Output schema fixture holding a map beside enums the generator defines
itself. */
+ public static class TaggedOrder {
+ public PartlyMapped status;
+ public Sign sign;
+ public Map<String, String> tags;
+ }
+
+ @Test
+ @DisplayName("An enum the generator defines itself does not hide a map
beside it")
+ void reachesMapBesideEnumDefinition() {
+ // The check is offered only the types no earlier definition provider
claims. The enum
+ // definition claims each enum, so the fields beside an enum must
still be reached.
+ List<Class<?>> openKeyed = List.of(Map.class, JsonNode.class);
+ Option strict = Option.FORBIDDEN_ADDITIONAL_PROPERTIES_BY_DEFAULT;
+
+ assertThat(PojoJsonSchemaGenerator.reachesAny(Order.class, openKeyed,
strict)).isFalse();
+ assertThat(PojoJsonSchemaGenerator.reachesAny(TaggedOrder.class,
openKeyed, strict))
+ .isTrue();
+ }
+
private static List<String> fieldNames(JsonNode node) {
List<String> names = new ArrayList<>();
node.fieldNames().forEachRemaining(names::add);
diff --git
a/integrations/chat-models/gemini/src/main/java/org/apache/flink/agents/integrations/chatmodels/gemini/GeminiChatModelConnection.java
b/integrations/chat-models/gemini/src/main/java/org/apache/flink/agents/integrations/chatmodels/gemini/GeminiChatModelConnection.java
index b2471ee62..49c3e2178 100644
---
a/integrations/chat-models/gemini/src/main/java/org/apache/flink/agents/integrations/chatmodels/gemini/GeminiChatModelConnection.java
+++
b/integrations/chat-models/gemini/src/main/java/org/apache/flink/agents/integrations/chatmodels/gemini/GeminiChatModelConnection.java
@@ -255,12 +255,13 @@ public class GeminiChatModelConnection extends
BaseChatModelConnection {
* Whether a config built from these inputs would carry a native {@code
responseJsonSchema}, the
* effective model's capability aside.
*
- * <p>Two conditions. Only a POJO {@link Class} has a native translation
here; a {@code
- * RowTypeInfo} wrapped in {@code OutputSchema}, or any other form, has
none. And the request
- * must bind no tools: outside a documented preview, Gemini answers a
request combining function
- * declarations with a JSON response mime type with {@code 400
INVALID_ARGUMENT}, so a
- * tool-carrying request keeps its tools and drops the schema. This is the
only connection whose
- * answer the tools can move.
+ * <p>Two conditions. Only a POJO {@link Class} has a native translation
here, and only one
+ * whose schema reaches no {@code JsonNode}: the native schema closes the
object it describes
+ * one as, so the model could return only {@code {}} for it. A {@code
RowTypeInfo} wrapped in
+ * {@code OutputSchema}, or any other form, has none. And the request must
bind no tools:
+ * outside a documented preview, Gemini answers a request combining
function declarations with a
+ * JSON response mime type with {@code 400 INVALID_ARGUMENT}, so a
tool-carrying request keeps
+ * its tools and drops the schema. This is the only connection whose
answer the tools can move.
*
* <p>No parameter is read today. It is nonetheless asked with the
caller's parameters rather
* than a copy the request builder has consumed, so that a conjunct added
here later reads the
@@ -269,13 +270,16 @@ public class GeminiChatModelConnection extends
BaseChatModelConnection {
* @param outputSchema the schema the request would carry, or null for an
unconstrained request
* @param tools the tools the request would bind; null and empty alike
mean none
* @param arguments not read
- * @return true if {@code outputSchema} is a POJO {@link Class} and no
tools would be bound
+ * @return true if {@code outputSchema} is a POJO {@link Class} that
reaches no {@code JsonNode}
+ * and no tools would be bound
*/
private boolean canApplyNativeStructuredOutput(
Object outputSchema,
List<org.apache.flink.agents.api.tools.Tool> tools,
Map<String, Object> arguments) {
- return outputSchema instanceof Class && (tools == null ||
tools.isEmpty());
+ return outputSchema instanceof Class
+ && (tools == null || tools.isEmpty())
+ && canDescribeNatively((Class<?>) outputSchema);
}
@Override
@@ -449,15 +453,34 @@ public class GeminiChatModelConnection extends
BaseChatModelConnection {
// Enabling it instead produces a $defs entry referencing another $defs
entry, a shape no
// published Gemini example demonstrates.
private static ObjectNode toNativeJsonSchema(Class<?> schemaClass) {
- ObjectNode schema =
- PojoJsonSchemaGenerator.generate(
- schemaClass,
- Option.MAP_VALUES_AS_ADDITIONAL_PROPERTIES,
- Option.FORBIDDEN_ADDITIONAL_PROPERTIES_BY_DEFAULT);
+ ObjectNode schema = PojoJsonSchemaGenerator.generate(schemaClass,
NATIVE_SCHEMA_OPTIONS);
stripRefSiblings(schema);
return schema;
}
+ // The options listed above toNativeJsonSchema. canDescribeNatively
generates under the same
+ // ones, because whether a type is reached depends on them: a Map's value
type is reached only
+ // under MAP_VALUES_AS_ADDITIONAL_PROPERTIES.
+ private static final Option[] NATIVE_SCHEMA_OPTIONS = {
+ Option.MAP_VALUES_AS_ADDITIONAL_PROPERTIES,
+ Option.FORBIDDEN_ADDITIONAL_PROPERTIES_BY_DEFAULT
+ };
+
+ // A Jackson JsonNode holds any JSON value, but the generator describes it
as an object, which
+ // FORBIDDEN_ADDITIONAL_PROPERTIES_BY_DEFAULT then closes. The schema
admits only {} for it, so
+ // a model held to the schema returns that and the data is lost. A Map
keeps its value schema
+ // under MAP_VALUES_AS_ADDITIONAL_PROPERTIES, so it is not listed.
+ private static final List<Class<?>> TYPES_WITHOUT_NATIVE_FORM =
List.of(JsonNode.class);
+
+ /**
+ * Whether the native {@code responseJsonSchema} can state {@code
schemaClass}'s contract, which
+ * it cannot once the schema reaches a {@code JsonNode}.
+ */
+ private static boolean canDescribeNatively(Class<?> schemaClass) {
+ return !PojoJsonSchemaGenerator.reachesAny(
+ schemaClass, TYPES_WITHOUT_NATIVE_FORM, NATIVE_SCHEMA_OPTIONS);
+ }
+
// Gemini states that a sub-schema setting $ref may set no other property
except those starting
// with a $, so every non-$ sibling is removed wherever a $ref appears.
The generator writes a
// referencing field's own keywords beside the reference, and two shapes
are measured to produce
diff --git
a/integrations/chat-models/gemini/src/test/java/org/apache/flink/agents/integrations/chatmodels/gemini/GeminiChatModelConnectionTest.java
b/integrations/chat-models/gemini/src/test/java/org/apache/flink/agents/integrations/chatmodels/gemini/GeminiChatModelConnectionTest.java
index 1bacb04b4..dc36a1ff1 100644
---
a/integrations/chat-models/gemini/src/test/java/org/apache/flink/agents/integrations/chatmodels/gemini/GeminiChatModelConnectionTest.java
+++
b/integrations/chat-models/gemini/src/test/java/org/apache/flink/agents/integrations/chatmodels/gemini/GeminiChatModelConnectionTest.java
@@ -133,6 +133,17 @@ class GeminiChatModelConnectionTest {
public int total;
}
+ /** Output schema fixture with a {@code JsonNode} field, which holds any
JSON value. */
+ public static class Note {
+ public String title;
+ public JsonNode body;
+ }
+
+ /** Output schema fixture whose map values are {@code JsonNode}s. */
+ public static class Catalog {
+ public Map<String, JsonNode> entries;
+ }
+
/** Nested type reused by two described fields of {@link Addresses}. */
public static class Address {
public String street;
@@ -775,6 +786,24 @@ class GeminiChatModelConnectionTest {
assertThat(config.responseMimeType()).isEmpty();
}
+ @Test
+ @DisplayName("A schema reaching a JsonNode is skipped, directly or as a
map's values")
+ void nativeSchemaSkippedForJsonNode() {
+ // The generator describes a JsonNode as an object and the native
schema closes it, so it
+ // would admit only {} and the model would return that. A map with
typed values keeps its
+ // value schema and stays native, as the Report fixture shows.
+ for (Class<?> schema : List.of(Note.class, Catalog.class)) {
+ GenerateContentConfig config =
+ connection().buildConfig(userMessage(), null, params(),
schema);
+
+
assertThat(config.responseJsonSchema()).as(schema.getSimpleName()).isEmpty();
+
assertThat(config.responseMimeType()).as(schema.getSimpleName()).isEmpty();
+ assertThat(connection().supportsNativeStructuredOutput(schema,
List.of(), params()))
+ .as(schema.getSimpleName())
+ .isEqualTo(NativeStructuredOutputSupport.INFEASIBLE);
+ }
+ }
+
@Test
@DisplayName("The derived schema lists enum constants by the values
Jackson deserializes")
void derivedSchemaListsEnumsByTheirJacksonWireValues() throws Exception {
@@ -1017,7 +1046,8 @@ class GeminiChatModelConnectionTest {
// capable throughout, so the schema form and the bound tools are what
move.
GeminiChatModelConnection connection = connection();
- for (Object schema : Arrays.asList(Report.class, "row<name STRING>",
null)) {
+ for (Object schema :
+ Arrays.asList(Report.class, Note.class, Catalog.class,
"row<name STRING>", null)) {
for (List<Tool> tools :
Arrays.asList(List.<Tool>of(), List.<Tool>of(new
SchemaOnlyTool()), null)) {
NativeStructuredOutputSupport answer =
diff --git
a/integrations/chat-models/openai/src/main/java/org/apache/flink/agents/integrations/chatmodels/openai/AzureOpenAIChatModelConnection.java
b/integrations/chat-models/openai/src/main/java/org/apache/flink/agents/integrations/chatmodels/openai/AzureOpenAIChatModelConnection.java
index 9c69cc5ec..b30eccefd 100644
---
a/integrations/chat-models/openai/src/main/java/org/apache/flink/agents/integrations/chatmodels/openai/AzureOpenAIChatModelConnection.java
+++
b/integrations/chat-models/openai/src/main/java/org/apache/flink/agents/integrations/chatmodels/openai/AzureOpenAIChatModelConnection.java
@@ -281,10 +281,11 @@ public class AzureOpenAIChatModelConnection extends
BaseChatModelConnection {
* Whether {@code outputSchema} is a form this connection could translate
into a native {@code
* response_format}, the effective model's capability aside.
*
- * <p>Two conditions. Only a POJO {@link Class} has a native translation
here; a {@code
- * RowTypeInfo} wrapped in {@code OutputSchema}, or any other form, has
none. And the configured
- * api-version has to reach the structured-output floor, which is a
property of this connection
- * rather than of the request.
+ * <p>Two conditions. Only a POJO {@link Class} has a native translation
here, and only one
+ * whose schema reaches no {@code Map} or {@code JsonNode}: strict mode
closes every object, so
+ * the model could return only {@code {}} for one. A {@code RowTypeInfo}
wrapped in {@code
+ * OutputSchema}, or any other form, has none. And the configured
api-version has to reach the
+ * structured-output floor, which is a property of this connection rather
than of the request.
*
* <p>A {@code true} answer is about those two conditions alone, and is
not a promise that a
* request gets built. A caller-supplied {@code response_format} in {@code
additional_kwargs} is
@@ -300,13 +301,16 @@ public class AzureOpenAIChatModelConnection extends
BaseChatModelConnection {
* @param outputSchema the schema the request would carry, or null for an
unconstrained request
* @param tools not read; bound tools do not stop this connection sending
a native schema
* @param modelParams not read
- * @return true if {@code outputSchema} is a POJO {@link Class} and the
configured api-version
- * reaches the structured-output floor; a caller-supplied {@code
response_format} does not
- * make it false, and the request builder raises on that conflict
+ * @return true if {@code outputSchema} is a POJO {@link Class} that
reaches no {@code Map} or
+ * {@code JsonNode} and the configured api-version reaches the
structured-output floor; a
+ * caller-supplied {@code response_format} does not make it false, and
the request builder
+ * raises on that conflict
*/
private boolean canApplyNativeStructuredOutput(
Object outputSchema, List<Tool> tools, Map<String, Object>
modelParams) {
- return outputSchema instanceof Class &&
apiVersionSupportsStructuredOutput();
+ return outputSchema instanceof Class
+ && apiVersionSupportsStructuredOutput()
+ && OpenAIChatCompletionsUtils.canDescribeNatively((Class<?>)
outputSchema);
}
/**
diff --git
a/integrations/chat-models/openai/src/main/java/org/apache/flink/agents/integrations/chatmodels/openai/OpenAIChatCompletionsUtils.java
b/integrations/chat-models/openai/src/main/java/org/apache/flink/agents/integrations/chatmodels/openai/OpenAIChatCompletionsUtils.java
index c2a96e367..eab73a8a5 100644
---
a/integrations/chat-models/openai/src/main/java/org/apache/flink/agents/integrations/chatmodels/openai/OpenAIChatCompletionsUtils.java
+++
b/integrations/chat-models/openai/src/main/java/org/apache/flink/agents/integrations/chatmodels/openai/OpenAIChatCompletionsUtils.java
@@ -19,6 +19,7 @@ package
org.apache.flink.agents.integrations.chatmodels.openai;
import com.fasterxml.jackson.core.JsonProcessingException;
import com.fasterxml.jackson.core.type.TypeReference;
+import com.fasterxml.jackson.databind.JsonNode;
import com.fasterxml.jackson.databind.ObjectMapper;
import com.github.victools.jsonschema.generator.Option;
import com.openai.core.JsonValue;
@@ -178,6 +179,26 @@ final class OpenAIChatCompletionsUtils {
}
}
+ // Strict mode requires additionalProperties false on every object, so the
native schema
+ // closes every object it describes. A Map's keys are not known in
advance, and a Jackson
+ // JsonNode holds any JSON value, which the generator describes as an
object. Closed, either
+ // admits only {}, so a model held to the schema returns that and the data
is lost.
+ private static final Option[] NATIVE_SCHEMA_OPTIONS = {
+ Option.FORBIDDEN_ADDITIONAL_PROPERTIES_BY_DEFAULT
+ };
+
+ private static final List<Class<?>> TYPES_WITHOUT_STRICT_FORM =
+ List.of(Map.class, JsonNode.class);
+
+ /**
+ * Whether the strict {@code json_schema} response format can state {@code
schemaClass}'s
+ * contract, which it cannot once the schema reaches a {@code Map} or a
{@code JsonNode}.
+ */
+ static boolean canDescribeNatively(Class<?> schemaClass) {
+ return !PojoJsonSchemaGenerator.reachesAny(
+ schemaClass, TYPES_WITHOUT_STRICT_FORM, NATIVE_SCHEMA_OPTIONS);
+ }
+
/**
* Derives the strict {@code json_schema} response format for a POJO
output schema class.
*
@@ -188,8 +209,7 @@ final class OpenAIChatCompletionsUtils {
*/
static ResponseFormatJsonSchema toNativeResponseFormat(Class<?>
schemaClass) {
Map<String, JsonValue> schema = new LinkedHashMap<>();
- PojoJsonSchemaGenerator.generateAllRequired(
- schemaClass,
Option.FORBIDDEN_ADDITIONAL_PROPERTIES_BY_DEFAULT)
+ PojoJsonSchemaGenerator.generateAllRequired(schemaClass,
NATIVE_SCHEMA_OPTIONS)
.fields()
.forEachRemaining(
member ->
diff --git
a/integrations/chat-models/openai/src/main/java/org/apache/flink/agents/integrations/chatmodels/openai/OpenAICompletionsConnection.java
b/integrations/chat-models/openai/src/main/java/org/apache/flink/agents/integrations/chatmodels/openai/OpenAICompletionsConnection.java
index bc3cf9fd5..6c2fd055b 100644
---
a/integrations/chat-models/openai/src/main/java/org/apache/flink/agents/integrations/chatmodels/openai/OpenAICompletionsConnection.java
+++
b/integrations/chat-models/openai/src/main/java/org/apache/flink/agents/integrations/chatmodels/openai/OpenAICompletionsConnection.java
@@ -209,18 +209,23 @@ public class OpenAICompletionsConnection extends
BaseChatModelConnection {
*
* <p>Only a POJO {@link Class} has a native translation here; a {@code
RowTypeInfo} wrapped in
* {@code OutputSchema}, or any other form, has none and keeps the
prompt-engineering fallback.
- * Nothing else about the request constrains the native branch, so neither
the tools nor the
- * parameters are read: this connection sends a native schema alongside
bound tools, and the one
- * parameter that would matter is the model, which is the capability
question this excludes.
+ * Nor does a class whose schema reaches a {@code Map} or a {@code
JsonNode}: strict mode closes
+ * every object, so the model could return only {@code {}} for one, and
the class keeps the
+ * fallback too. Nothing else about the request constrains the native
branch, so neither the
+ * tools nor the parameters are read: this connection sends a native
schema alongside bound
+ * tools, and the one parameter that would matter is the model, which is
the capability question
+ * this excludes.
*
* @param outputSchema the schema the request would carry, or null for an
unconstrained request
* @param tools not read; bound tools do not stop this connection sending
a native schema
* @param modelParams not read
- * @return true if {@code outputSchema} is a POJO {@link Class}
+ * @return true if {@code outputSchema} is a POJO {@link Class} that
reaches no {@code Map} or
+ * {@code JsonNode}
*/
private boolean canApplyNativeStructuredOutput(
Object outputSchema, List<Tool> tools, Map<String, Object>
modelParams) {
- return outputSchema instanceof Class;
+ return outputSchema instanceof Class
+ && OpenAIChatCompletionsUtils.canDescribeNatively((Class<?>)
outputSchema);
}
/**
diff --git
a/integrations/chat-models/openai/src/test/java/org/apache/flink/agents/integrations/chatmodels/openai/AzureOpenAIChatModelConnectionTest.java
b/integrations/chat-models/openai/src/test/java/org/apache/flink/agents/integrations/chatmodels/openai/AzureOpenAIChatModelConnectionTest.java
index a4bd0bfd5..2f0ffd404 100644
---
a/integrations/chat-models/openai/src/test/java/org/apache/flink/agents/integrations/chatmodels/openai/AzureOpenAIChatModelConnectionTest.java
+++
b/integrations/chat-models/openai/src/test/java/org/apache/flink/agents/integrations/chatmodels/openai/AzureOpenAIChatModelConnectionTest.java
@@ -131,6 +131,18 @@ class AzureOpenAIChatModelConnectionTest {
public Status status;
}
+ /** Output schema fixture with a {@code Map} field, whose keys are not
known in advance. */
+ public static class Tally {
+ public String label;
+ public Map<String, Integer> counts;
+ }
+
+ /** Output schema fixture with a {@code JsonNode} field, which holds any
JSON value. */
+ public static class Note {
+ public String title;
+ public JsonNode body;
+ }
+
/** Enum constants deserialized from their {@code @JsonProperty} values. */
public enum Status {
@JsonProperty("in-progress")
@@ -503,6 +515,26 @@ class AzureOpenAIChatModelConnectionTest {
assertThat(request.responseFormat()).isEmpty();
}
+ @Test
+ @DisplayName("Native NOT applied for a schema reaching a Map or a
JsonNode")
+ void testNativeNotAppliedForOpenObjectSchema() {
+ // Strict mode closes every object, so the native schema would admit
only {} for the map or
+ // the tree and the model would return that. The prompt path keeps the
field's contents.
+ for (Class<?> schema : List.of(Tally.class, Note.class)) {
+ ChatCompletionCreateParams request =
+ connection()
+ .buildRequest(userMessage(), List.of(),
params("gpt-4o-mini"), schema);
+
+
assertThat(request.responseFormat()).as(schema.getSimpleName()).isEmpty();
+ assertThat(
+ connection()
+ .supportsNativeStructuredOutput(
+ schema, List.of(),
params("gpt-4o-mini")))
+ .as(schema.getSimpleName())
+ .isEqualTo(NativeStructuredOutputSupport.INFEASIBLE);
+ }
+ }
+
@Test
@DisplayName("Native applied for a POJO even when tools are bound")
void testNativeAppliedEvenWhenToolsBound() {
@@ -596,6 +628,9 @@ class AzureOpenAIChatModelConnectionTest {
Arguments.of(
"non_pojo_schema", CAPABLE_API_VERSION, "gpt-4o-mini",
"row<name STRING>"),
Arguments.of("no_output_schema", CAPABLE_API_VERSION,
"gpt-4o-mini", null),
+ Arguments.of("schema_with_map", CAPABLE_API_VERSION,
"gpt-4o-mini", Tally.class),
+ Arguments.of(
+ "schema_with_json_node", CAPABLE_API_VERSION,
"gpt-4o-mini", Note.class),
Arguments.of(
"api_version_below_floor",
BELOW_FLOOR_API_VERSION,
@@ -833,7 +868,9 @@ class AzureOpenAIChatModelConnectionTest {
for (String apiVersion : List.of(CAPABLE_API_VERSION,
BELOW_FLOOR_API_VERSION)) {
AzureOpenAIChatModelConnection connection = connection(apiVersion);
- for (Object schema : Arrays.asList(Person.class, "row<name
STRING>", null)) {
+ for (Object schema :
+ Arrays.asList(
+ Person.class, Tally.class, Note.class, "row<name
STRING>", null)) {
for (List<Tool> tools :
Arrays.asList(List.<Tool>of(), List.<Tool>of(new
StubTool()), null)) {
for (String backing : Arrays.asList("gpt-4o-mini",
"gpt-3.5-turbo", null)) {
diff --git
a/integrations/chat-models/openai/src/test/java/org/apache/flink/agents/integrations/chatmodels/openai/OpenAICompletionsConnectionTest.java
b/integrations/chat-models/openai/src/test/java/org/apache/flink/agents/integrations/chatmodels/openai/OpenAICompletionsConnectionTest.java
index 05cf78890..7dfb88ce2 100644
---
a/integrations/chat-models/openai/src/test/java/org/apache/flink/agents/integrations/chatmodels/openai/OpenAICompletionsConnectionTest.java
+++
b/integrations/chat-models/openai/src/test/java/org/apache/flink/agents/integrations/chatmodels/openai/OpenAICompletionsConnectionTest.java
@@ -141,6 +141,18 @@ class OpenAICompletionsConnectionTest {
public Person author;
}
+ /** Output schema fixture with a {@code Map} field, whose keys are not
known in advance. */
+ public static class Tally {
+ public String label;
+ public Map<String, Integer> counts;
+ }
+
+ /** Output schema fixture with a {@code JsonNode} field, which holds any
JSON value. */
+ public static class Note {
+ public String title;
+ public JsonNode body;
+ }
+
/** The JSON Schema the request carries, as a JSON tree. */
private static JsonNode nativeSchema(ChatCompletionCreateParams params) {
return ObjectMappers.jsonMapper()
@@ -510,6 +522,25 @@ class OpenAICompletionsConnectionTest {
assertThat(params.responseFormat()).isEmpty();
}
+ @Test
+ @DisplayName("Native NOT applied for a schema reaching a Map or a
JsonNode")
+ void testNativeNotAppliedForOpenObjectSchema() {
+ // Strict mode closes every object, so the native schema would admit
only {} for the map or
+ // the tree and the model would return that. The prompt path keeps the
field's contents.
+ for (Class<?> schema : List.of(Tally.class, Note.class)) {
+ ChatCompletionCreateParams params =
+ connection().buildRequest(userMessage(), List.of(),
params("gpt-4o"), schema);
+
+
assertThat(params.responseFormat()).as(schema.getSimpleName()).isEmpty();
+ assertThat(
+ connection()
+ .supportsNativeStructuredOutput(
+ schema, List.of(),
params("gpt-4o")))
+ .as(schema.getSimpleName())
+ .isEqualTo(NativeStructuredOutputSupport.INFEASIBLE);
+ }
+ }
+
@Test
@DisplayName("Native applied for a POJO even when tools are bound (no
empty-tools gate)")
void testNativeAppliedEvenWhenToolsBound() {
@@ -601,7 +632,8 @@ class OpenAICompletionsConnectionTest {
// capable in every case, so feasibility alone decides the branch.
OpenAICompletionsConnection connection = connection();
- for (Object schema : Arrays.asList(Person.class, "row<name STRING>",
null)) {
+ for (Object schema :
+ Arrays.asList(Person.class, Tally.class, Note.class, "row<name
STRING>", null)) {
for (List<Tool> tools :
Arrays.asList(List.<Tool>of(), List.<Tool>of(new
StubTool()), null)) {
NativeStructuredOutputSupport answer =
diff --git
a/integrations/chat-models/openai/src/test/java/org/apache/flink/agents/integrations/chatmodels/openai/VLLMChatModelConnectionTest.java
b/integrations/chat-models/openai/src/test/java/org/apache/flink/agents/integrations/chatmodels/openai/VLLMChatModelConnectionTest.java
index f09219d96..e8727f325 100644
---
a/integrations/chat-models/openai/src/test/java/org/apache/flink/agents/integrations/chatmodels/openai/VLLMChatModelConnectionTest.java
+++
b/integrations/chat-models/openai/src/test/java/org/apache/flink/agents/integrations/chatmodels/openai/VLLMChatModelConnectionTest.java
@@ -109,6 +109,12 @@ class VLLMChatModelConnectionTest {
public int age;
}
+ /** Output schema fixture with a {@code Map} field, whose keys are not
known in advance. */
+ public static class Tally {
+ public String label;
+ public Map<String, Integer> counts;
+ }
+
@Test
@DisplayName("Native is recommended for any served model, not only OpenAI
model names")
void testQueryRecommendsNativeForServedModels() {
@@ -129,6 +135,13 @@ class VLLMChatModelConnectionTest {
List.of(),
Map.of("model", "Qwen/Qwen2.5-7B-Instruct")))
.isEqualTo(NativeStructuredOutputSupport.INFEASIBLE);
+ // Nor is the strict form's exclusion of a schema reaching a Map,
which vLLM inherits.
+ assertThat(
+ conn.supportsNativeStructuredOutput(
+ Tally.class,
+ List.of(),
+ Map.of("model", "Qwen/Qwen2.5-7B-Instruct")))
+ .isEqualTo(NativeStructuredOutputSupport.INFEASIBLE);
}
private static NativeStructuredOutputSupport support(
diff --git a/python/flink_agents/api/agents/tests/test_output_schema_render.py
b/python/flink_agents/api/agents/tests/test_output_schema_render.py
index 5253fa6cc..bd6d8917a 100644
--- a/python/flink_agents/api/agents/tests/test_output_schema_render.py
+++ b/python/flink_agents/api/agents/tests/test_output_schema_render.py
@@ -15,13 +15,13 @@
# See the License for the specific language governing permissions and
# limitations under the License.
#################################################################################
-from typing import Any, Callable
+from typing import Annotated, Any, Callable, Mapping
import pytest
-from pydantic import BaseModel
+from pydantic import BaseModel, ConfigDict, RootModel, StringConstraints
from pydantic.errors import PydanticInvalidForJsonSchema
-from flink_agents.api.agents.types import render_output_schema
+from flink_agents.api.agents.types import contains_open_object,
render_output_schema
def _render(model: type[BaseModel]) -> dict[str, Any]:
@@ -154,3 +154,129 @@ def test_renderer_failure_raises_chained() -> None:
render_output_schema(Renderable, _render_failing)
assert isinstance(exc_info.value.__cause__, ValueError)
+
+
+class TypedValues(BaseModel):
+ counts: dict[str, int]
+
+
+class AnyDict(BaseModel):
+ attributes: dict[str, Any]
+
+
+class BareDict(BaseModel):
+ attributes: dict
+
+
+class MappingField(BaseModel):
+ counts: Mapping[str, int]
+
+
+class OptionalDict(BaseModel):
+ counts: dict[str, int] | None = None
+
+
+class ListOfDicts(BaseModel):
+ rows: list[dict[str, int]]
+
+
+class DictOfModels(BaseModel):
+ by_name: dict[str, Renderable]
+
+
+class NestsDict(BaseModel):
+ inner: TypedValues
+
+
+class ConstrainedKeys(BaseModel):
+ counts: dict[Annotated[str, StringConstraints(pattern="^[a-z]+$")], int]
+
+
+class TupleWithDict(BaseModel):
+ pair: tuple[int, dict[str, int]]
+
+
+class ModelOrDict(BaseModel):
+ value: Renderable | dict[str, int]
+
+
+class AllowsExtra(BaseModel):
+ model_config = ConfigDict(extra="allow")
+ name: str
+
+
+class DictRoot(RootModel[dict[str, int]]):
+ pass
+
+
+class NestsRenderable(BaseModel):
+ inner: Renderable
+ items: list[Renderable]
+
+
+class AnyField(BaseModel):
+ value: Any
+
+
+class ForbidsExtra(BaseModel):
+ model_config = ConfigDict(extra="forbid")
+ name: str
+
+
+class KeywordNamedField(BaseModel):
+ additionalProperties: int
+ patternProperties: str
+
+
[email protected](
+ "model",
+ [
+ TypedValues,
+ AnyDict,
+ BareDict,
+ MappingField,
+ OptionalDict,
+ ListOfDicts,
+ DictOfModels,
+ NestsDict,
+ ConstrainedKeys,
+ TupleWithDict,
+ ModelOrDict,
+ AllowsExtra,
+ DictRoot,
+ ],
+)
+def test_schema_admitting_undeclared_keys_is_open(model: type[BaseModel]) ->
None:
+ """A dict anywhere in the schema, or an extra-allowing model, is found.
+
+ A strict native schema closes every object, so it would admit only ``{}``
for
+ each of these, or be refused.
+ """
+ assert contains_open_object(model)
+
+
[email protected](
+ "model",
+ [
+ Renderable,
+ NestsRenderable,
+ SelfReferential,
+ MutualA,
+ FieldLess,
+ AnyField,
+ ForbidsExtra,
+ KeywordNamedField,
+ ],
+)
+def test_schema_declaring_every_key_is_not_open(model: type[BaseModel]) ->
None:
+ """A schema whose objects all declare their keys is left to the native
path.
+
+ That includes a field named after a keyword, since keys under
``properties``
+ are names rather than keywords.
+ """
+ assert not contains_open_object(model)
+
+
+def test_model_without_a_json_schema_is_not_reported_open() -> None:
+ """The render reports that failure clearly, so this check leaves it
alone."""
+ assert not contains_open_object(Unrenderable)
diff --git a/python/flink_agents/api/agents/types.py
b/python/flink_agents/api/agents/types.py
index a0f5efffe..381f0da1e 100644
--- a/python/flink_agents/api/agents/types.py
+++ b/python/flink_agents/api/agents/types.py
@@ -128,3 +128,54 @@ def render_output_schema(
)
raise TypeError(msg)
return schema
+
+
+def contains_open_object(model: type[BaseModel]) -> bool:
+ """Whether ``model``'s JSON Schema admits keys it does not declare, at any
depth.
+
+ A ``dict`` field does, wherever it sits, and so does a model configured
with
+ ``extra="allow"``: their keys are not known in advance. A strict native
schema
+ requires every object to be closed, so it cannot state either, and a
connection
+ whose native form is strict keeps the prompt path for such a model.
+
+ Only the keywords that hold subschemas are followed. Keys under
``properties`` and
+ ``$defs`` are names, so a field named ``additionalProperties`` does not
count.
+
+ Args:
+ model: The model class describing the shape the response must take.
+
+ Returns:
+ ``True`` if some object in the schema sets ``additionalProperties`` to
+ anything but ``False``, or sets ``patternProperties``. ``False`` when
the
+ model has no JSON Schema, leaving that failure to
``render_output_schema``,
+ which reports it clearly.
+ """
+ try:
+ schema = model.model_json_schema()
+ except Exception:
+ return False
+ return _admits_undeclared_keys(schema)
+
+
+def _admits_undeclared_keys(schema: Any) -> bool:
+ if not isinstance(schema, dict):
+ return False
+ if schema.get("additionalProperties", False) is not False:
+ return True
+ if "patternProperties" in schema:
+ return True
+ for keyword in ("properties", "$defs"):
+ subschemas = schema.get(keyword)
+ if isinstance(subschemas, dict) and any(
+ _admits_undeclared_keys(subschema) for subschema in
subschemas.values()
+ ):
+ return True
+ for keyword in ("prefixItems", "anyOf", "oneOf", "allOf"):
+ subschemas = schema.get(keyword)
+ if isinstance(subschemas, list) and any(
+ _admits_undeclared_keys(subschema) for subschema in subschemas
+ ):
+ return True
+ return _admits_undeclared_keys(schema.get("items")) or
_admits_undeclared_keys(
+ schema.get("not")
+ )
diff --git
a/python/flink_agents/integrations/chat_models/anthropic/anthropic_chat_model.py
b/python/flink_agents/integrations/chat_models/anthropic/anthropic_chat_model.py
index bef145dbd..7264d5a3c 100644
---
a/python/flink_agents/integrations/chat_models/anthropic/anthropic_chat_model.py
+++
b/python/flink_agents/integrations/chat_models/anthropic/anthropic_chat_model.py
@@ -27,7 +27,11 @@ from anthropic.types import MessageParam, TextBlockParam,
ToolParam
from pydantic import BaseModel, Field, PrivateAttr
from typing_extensions import override
-from flink_agents.api.agents.types import OutputSchema, render_output_schema
+from flink_agents.api.agents.types import (
+ OutputSchema,
+ contains_open_object,
+ render_output_schema,
+)
from flink_agents.api.chat_message import (
Base64Source,
ChatMessage,
@@ -521,11 +525,14 @@ class
AnthropicChatModelConnection(BaseChatModelConnection):
"""Whether a request built from these inputs would carry a derived
``output_config``, leaving the effective model's capability out of the
answer.
- Two conditions. Only a ``BaseModel`` subclass has a native translation
here; a
- ``RowTypeInfo`` wrapped in ``OutputSchema``, or no schema at all, has
none. And
- a caller-supplied ``output_config`` wins: it is the caller being
explicit about
- the exact parameter the native branch writes, so the schema keeps the
- prompt-engineering fallback rather than the two competing on one
request.
+ Two conditions. Only a ``BaseModel`` subclass has a native translation
here,
+ and only one whose schema admits no undeclared keys: a ``dict`` field
admits
+ them, and the derived schema closes every object, so the model could
return
+ only ``{}`` for one. A ``RowTypeInfo`` wrapped in ``OutputSchema``, or
no
+ schema at all, has none. And a caller-supplied ``output_config`` wins:
it is
+ the caller being explicit about the exact parameter the native branch
writes,
+ so the schema keeps the prompt-engineering fallback rather than the two
+ competing on one request.
The tools are not read; a bound tool does not stop this connection
sending a
schema.
@@ -549,11 +556,15 @@ class
AnthropicChatModelConnection(BaseChatModelConnection):
Returns:
-------
bool
- ``True`` if ``output_schema`` wraps a ``BaseModel`` subclass and
the caller
- supplied no ``output_config`` of its own.
+ ``True`` if ``output_schema`` wraps a ``BaseModel`` subclass whose
schema
+ admits no undeclared keys, and the caller supplied no
``output_config`` of
+ its own.
"""
- return _native_output_model(output_schema) is not None and (
- model_kwargs is None or "output_config" not in model_kwargs
+ model = _native_output_model(output_schema)
+ return (
+ model is not None
+ and (model_kwargs is None or "output_config" not in model_kwargs)
+ and not contains_open_object(model)
)
def chat(
diff --git
a/python/flink_agents/integrations/chat_models/anthropic/tests/test_anthropic_response_parsing.py
b/python/flink_agents/integrations/chat_models/anthropic/tests/test_anthropic_response_parsing.py
index 4aad9e30c..8139d1a65 100644
---
a/python/flink_agents/integrations/chat_models/anthropic/tests/test_anthropic_response_parsing.py
+++
b/python/flink_agents/integrations/chat_models/anthropic/tests/test_anthropic_response_parsing.py
@@ -22,7 +22,7 @@ from unittest.mock import MagicMock
import pytest
from anthropic import transform_schema
from anthropic.types import Message, TextBlock, ToolUseBlock, Usage
-from pydantic import BaseModel
+from pydantic import BaseModel, ConfigDict
from pyflink.common.typeinfo import Types
from flink_agents.api.agents.types import OutputSchema
@@ -232,11 +232,19 @@ class _MapsToFieldLess(BaseModel):
class _Labelled(BaseModel):
- """A schema whose only member is a free-form map, a legitimate
constraint."""
+ """A schema whose only member is a map, which the renderer closes empty."""
labels: Dict[str, str]
+class _AllowsExtra(BaseModel):
+ """A schema admitting undeclared keys at its root, which the renderer
closes."""
+
+ model_config = ConfigDict(extra="allow")
+
+ verdict: str
+
+
# A model the provider documents native structured-output support for.
_CAPABLE_MODEL = "claude-sonnet-4-5"
@@ -314,11 +322,11 @@ def test_unrenderable_schema_raises_naming_the_model() ->
None:
)
[email protected]("schema", [_FieldLess, _NestsFieldLess,
_MapsToFieldLess])
[email protected]("schema", [_FieldLess, _NestsFieldLess])
def test_field_less_schema_is_accepted_and_sent_whole(schema) -> None:
# A schema declaring no fields renders, so the provider decides on it, not
this
# connection. The document reaches the request exactly as rendered rather
than
- # being refused here. The nested cases carry the field-less model below
the root,
+ # being refused here. The nested case carries the field-less model below
the root,
# so the assertion covers the whole document rather than only its top
level.
output_config = _request_kwargs(
model=_CAPABLE_MODEL, output_schema=OutputSchema(output_schema=schema)
@@ -327,16 +335,22 @@ def
test_field_less_schema_is_accepted_and_sent_whole(schema) -> None:
assert output_config["format"]["schema"] == transform_schema(schema)
-def test_map_member_schema_is_accepted_and_sent_whole() -> None:
- # This renderer rewrites a map member into an object with an empty
properties. The
- # rewritten document is what reaches the request, so the member survives
the
- # normalization rather than being dropped or flattened.
- output_config = _request_kwargs(
- model=_CAPABLE_MODEL,
output_schema=OutputSchema(output_schema=_Labelled)
- )["output_config"]
[email protected]("schema", [_Labelled, _MapsToFieldLess, _AllowsExtra])
+def test_schema_admitting_undeclared_keys_keeps_the_prompt_path(schema) ->
None:
+ # The renderer closes every object, so a map would reach the provider as
an object
+ # with no properties and come back empty, and an open model would lose its
extra
+ # keys, with no error either way. No output_config is sent, and the query
answers
+ # INFEASIBLE in step, which fails a NATIVE policy before the call is made.
+ assert (
+ _connection().supports_native_structured_output(
+ OutputSchema(output_schema=schema), [], {"model": _CAPABLE_MODEL}
+ )
+ is NativeStructuredOutputSupport.INFEASIBLE
+ )
- assert output_config["format"]["schema"] == transform_schema(_Labelled)
- assert
output_config["format"]["schema"]["properties"]["labels"]["properties"] == {}
+ assert "output_config" not in _request_kwargs(
+ model=_CAPABLE_MODEL, output_schema=OutputSchema(output_schema=schema)
+ )
def test_unlisted_model_answers_feasible_and_still_carries_the_schema() ->
None:
@@ -603,6 +617,16 @@ def test_json_prefill_applied_when_schema_falls_back() ->
None:
) == (True, _COMPLETED)
+def test_json_prefill_applied_when_open_schema_falls_back() -> None:
+ # A schema admitting undeclared keys is a BaseModel, so it passes every
check a
+ # RowTypeInfo fails, and still falls back. The prefill has to follow it
there.
+ assert _prefill_outcome(
+ model=_INCAPABLE_MODEL,
+ json_prefill=True,
+ output_schema=OutputSchema(output_schema=_AllowsExtra),
+ ) == (True, _COMPLETED)
+
+
def test_json_prefill_suppressed_on_prefill_unsupported_model() -> None:
assert _prefill_outcome(model="claude-opus-4-6", json_prefill=True) == (
False,
@@ -872,6 +896,7 @@ def test_query_agrees_with_the_native_branch() -> None:
for schema in (
OutputSchema(output_schema=_Answer),
+ OutputSchema(output_schema=_Labelled),
OutputSchema(output_schema=row_type),
None,
):
diff --git
a/python/flink_agents/integrations/chat_models/azure/azure_openai_chat_model.py
b/python/flink_agents/integrations/chat_models/azure/azure_openai_chat_model.py
index 438a9a167..7820f8135 100644
---
a/python/flink_agents/integrations/chat_models/azure/azure_openai_chat_model.py
+++
b/python/flink_agents/integrations/chat_models/azure/azure_openai_chat_model.py
@@ -29,7 +29,11 @@ from openai.lib._pydantic import to_strict_json_schema
from pydantic import BaseModel, Field, PrivateAttr
from typing_extensions import override
-from flink_agents.api.agents.types import OutputSchema, render_output_schema
+from flink_agents.api.agents.types import (
+ OutputSchema,
+ contains_open_object,
+ render_output_schema,
+)
from flink_agents.api.chat_message import ChatMessage
from flink_agents.api.chat_models.chat_model import (
BaseChatModelConnection,
@@ -304,10 +308,12 @@ class
AzureOpenAIChatModelConnection(BaseChatModelConnection):
"""Whether a request built from these inputs would carry a native
``response_format``, leaving the effective model's capability out of
the answer.
- Two conditions. Only a ``BaseModel`` subclass has a native translation
here; a
- ``RowTypeInfo`` wrapped in ``OutputSchema``, or no schema at all, has
none. And
- the configured api-version has to reach the structured-output floor,
which is a
- property of this connection rather than of the request.
+ Two conditions. Only a ``BaseModel`` subclass has a native translation
here,
+ and only one whose schema admits no undeclared keys: a ``dict`` field
admits
+ them, and strict mode requires every object to be closed. A
``RowTypeInfo``
+ wrapped in ``OutputSchema``, or no schema at all, has none. And the
configured
+ api-version has to reach the structured-output floor, which is a
property of
+ this connection rather than of the request.
A caller-supplied ``response_format`` is deliberately not a condition:
the
branch sets the format and then raises on that conflict rather than
skipping,
@@ -334,12 +340,15 @@ class
AzureOpenAIChatModelConnection(BaseChatModelConnection):
Returns:
-------
bool
- ``True`` if ``output_schema`` wraps a ``BaseModel`` subclass and
the
- configured api-version reaches the structured-output floor.
+ ``True`` if ``output_schema`` wraps a ``BaseModel`` subclass whose
schema
+ admits no undeclared keys, and the configured api-version reaches
the
+ structured-output floor.
"""
+ model = _native_output_model(output_schema)
return (
- _native_output_model(output_schema) is not None
+ model is not None
and self._api_version_supports_structured_output()
+ and not contains_open_object(model)
)
def chat(
diff --git
a/python/flink_agents/integrations/chat_models/azure/tests/test_azure_openai_native_structured_output.py
b/python/flink_agents/integrations/chat_models/azure/tests/test_azure_openai_native_structured_output.py
index 0ab06ed2b..1d549f4f1 100644
---
a/python/flink_agents/integrations/chat_models/azure/tests/test_azure_openai_native_structured_output.py
+++
b/python/flink_agents/integrations/chat_models/azure/tests/test_azure_openai_native_structured_output.py
@@ -20,7 +20,7 @@ from unittest.mock import MagicMock
import pytest
from openai.lib._pydantic import to_strict_json_schema
-from pydantic import BaseModel
+from pydantic import BaseModel, ConfigDict
from pyflink.common.typeinfo import Types
from flink_agents.api.agents.types import OutputSchema
@@ -74,11 +74,19 @@ class MapsToFieldLess(BaseModel):
class Labelled(BaseModel):
- """A schema whose only member is a free-form map, a legitimate
constraint."""
+ """A schema whose only member is a map, which strict mode cannot
express."""
labels: dict[str, str]
+class AllowsExtra(BaseModel):
+ """A schema admitting undeclared keys at its root, which strict mode
forbids."""
+
+ model_config = ConfigDict(extra="allow")
+
+ name: str
+
+
ROW_TYPE = Types.ROW_NAMED(["name"], [Types.STRING()])
@@ -492,12 +500,12 @@ def test_unrenderable_schema_raises_naming_the_model() ->
None:
_chat_with_schema(_connection(), Unrenderable)
[email protected]("schema", [FieldLess, NestsFieldLess,
MapsToFieldLess])
[email protected]("schema", [FieldLess, NestsFieldLess])
def test_field_less_schema_is_accepted_and_sent_whole(schema: type[BaseModel])
-> None:
"""A schema declaring no fields renders, so the provider decides on it,
not us.
The document reaches the request exactly as rendered rather than being
refused
- here. The nested cases carry the field-less model below the root, so the
+ here. The nested case carries the field-less model below the root, so the
assertion covers the whole document rather than only its top level.
"""
conn = _connection()
@@ -506,12 +514,26 @@ def
test_field_less_schema_is_accepted_and_sent_whole(schema: type[BaseModel]) -
assert response_format["json_schema"]["schema"] ==
to_strict_json_schema(schema)
-def test_map_member_schema_is_accepted_and_sent_whole() -> None:
- """A free-form map is a legitimate constraint and reaches the request
intact."""
[email protected]("schema", [Labelled, MapsToFieldLess, AllowsExtra])
+def test_schema_admitting_undeclared_keys_keeps_the_prompt_path(
+ schema: type[BaseModel],
+) -> None:
+ """A map or an open model has no strict form, so no response_format is
sent.
+
+ Strict mode requires every object to forbid undeclared keys, and these keep
+ theirs open, so the provider would reject the request. The query answers
+ INFEASIBLE in step, which fails a NATIVE policy before the call is made.
+ """
conn = _connection()
- _chat_with_schema(conn, Labelled)
- response_format = _create_call_kwargs(conn)["response_format"]
- assert response_format["json_schema"]["schema"] ==
to_strict_json_schema(Labelled)
+ assert (
+ conn.supports_native_structured_output(
+ OutputSchema(output_schema=schema), [],
_model_kwargs("gpt-4o-mini")
+ )
+ is NativeStructuredOutputSupport.INFEASIBLE
+ )
+
+ _chat_with_schema(conn, schema)
+ assert "response_format" not in _create_call_kwargs(conn)
def _model_kwargs(backing_model: str | None = None) -> dict[str, Any]:
@@ -572,6 +594,7 @@ def test_query_agrees_with_the_native_branch() -> None:
for schema in (
OutputSchema(output_schema=Person),
+ OutputSchema(output_schema=Labelled),
OutputSchema(output_schema=ROW_TYPE),
None,
):
diff --git
a/python/flink_agents/integrations/chat_models/openai/openai_chat_model.py
b/python/flink_agents/integrations/chat_models/openai/openai_chat_model.py
index afdcd70e4..9865c5195 100644
--- a/python/flink_agents/integrations/chat_models/openai/openai_chat_model.py
+++ b/python/flink_agents/integrations/chat_models/openai/openai_chat_model.py
@@ -28,7 +28,11 @@ from openai.lib._pydantic import to_strict_json_schema
from pydantic import BaseModel, Field, PrivateAttr
from typing_extensions import override
-from flink_agents.api.agents.types import OutputSchema, render_output_schema
+from flink_agents.api.agents.types import (
+ OutputSchema,
+ contains_open_object,
+ render_output_schema,
+)
from flink_agents.api.chat_message import ChatMessage
from flink_agents.api.chat_models.chat_model import (
BaseChatModelConnection,
@@ -285,8 +289,10 @@ class OpenAIChatModelConnection(BaseChatModelConnection):
Only a ``BaseModel`` subclass has a native translation here; a
``RowTypeInfo``
wrapped in ``OutputSchema``, or no schema at all, has none and keeps
the
- prompt-engineering fallback. Nothing else about the request constrains
the
- native branch, so neither the tools nor the parameters are read: this
connection
+ prompt-engineering fallback. So does a model whose schema admits
undeclared
+ keys, such as one with a ``dict`` field, since strict mode requires
every
+ object to be closed. Nothing else about the request constrains the
native
+ branch, so neither the tools nor the parameters are read: this
connection
sends a native schema alongside bound tools, and the one parameter
that would
bear on the answer is the model, which is the capability question this
excludes.
@@ -307,9 +313,11 @@ class OpenAIChatModelConnection(BaseChatModelConnection):
Returns:
-------
bool
- ``True`` if ``output_schema`` wraps a ``BaseModel`` subclass.
+ ``True`` if ``output_schema`` wraps a ``BaseModel`` subclass whose
schema
+ admits no undeclared keys.
"""
- return _native_output_model(output_schema) is not None
+ model = _native_output_model(output_schema)
+ return model is not None and not contains_open_object(model)
def chat(
self,
diff --git
a/python/flink_agents/integrations/chat_models/openai/tests/test_openai_native_structured_output.py
b/python/flink_agents/integrations/chat_models/openai/tests/test_openai_native_structured_output.py
index 42abf8fbb..6f160d6c6 100644
---
a/python/flink_agents/integrations/chat_models/openai/tests/test_openai_native_structured_output.py
+++
b/python/flink_agents/integrations/chat_models/openai/tests/test_openai_native_structured_output.py
@@ -20,7 +20,7 @@ from unittest.mock import MagicMock
import pytest
from openai.lib._pydantic import to_strict_json_schema
-from pydantic import BaseModel
+from pydantic import BaseModel, ConfigDict
from pyflink.common.typeinfo import Types
from flink_agents.api.agents.types import OutputSchema
@@ -63,11 +63,19 @@ class MapsToFieldLess(BaseModel):
class Labelled(BaseModel):
- """A schema whose only member is a free-form map, a legitimate
constraint."""
+ """A schema whose only member is a map, which strict mode cannot
express."""
labels: dict[str, str]
+class AllowsExtra(BaseModel):
+ """A schema admitting undeclared keys at its root, which strict mode
forbids."""
+
+ model_config = ConfigDict(extra="allow")
+
+ name: str
+
+
def _connection() -> OpenAIChatModelConnection:
conn = OpenAIChatModelConnection(
api_key="test-key", api_base_url="http://localhost";
@@ -271,12 +279,12 @@ def test_unrenderable_schema_raises_naming_the_model() ->
None:
_chat_with_schema(_connection(), Unrenderable)
[email protected]("schema", [FieldLess, NestsFieldLess,
MapsToFieldLess])
[email protected]("schema", [FieldLess, NestsFieldLess])
def test_field_less_schema_is_accepted_and_sent_whole(schema: type[BaseModel])
-> None:
"""A schema declaring no fields renders, so the provider decides on it,
not us.
The document reaches the request exactly as rendered rather than being
refused
- here. The nested cases carry the field-less model below the root, so the
+ here. The nested case carries the field-less model below the root, so the
assertion covers the whole document rather than only its top level.
"""
conn = _connection()
@@ -285,12 +293,26 @@ def
test_field_less_schema_is_accepted_and_sent_whole(schema: type[BaseModel]) -
assert response_format["json_schema"]["schema"] ==
to_strict_json_schema(schema)
-def test_map_member_schema_is_accepted_and_sent_whole() -> None:
- """A free-form map is a legitimate constraint and reaches the request
intact."""
[email protected]("schema", [Labelled, MapsToFieldLess, AllowsExtra])
+def test_schema_admitting_undeclared_keys_keeps_the_prompt_path(
+ schema: type[BaseModel],
+) -> None:
+ """A map or an open model has no strict form, so no response_format is
sent.
+
+ Strict mode requires every object to forbid undeclared keys, and these keep
+ theirs open, so the provider would reject the request. The query answers
+ INFEASIBLE in step, which fails a NATIVE policy before the call is made.
+ """
conn = _connection()
- _chat_with_schema(conn, Labelled)
- response_format = _create_call_kwargs(conn)["response_format"]
- assert response_format["json_schema"]["schema"] ==
to_strict_json_schema(Labelled)
+ assert (
+ conn.supports_native_structured_output(
+ OutputSchema(output_schema=schema), [], {"model": "gpt-4o"}
+ )
+ is NativeStructuredOutputSupport.INFEASIBLE
+ )
+
+ _chat_with_schema(conn, schema)
+ assert "response_format" not in _create_call_kwargs(conn)
def _judging_connection() -> tuple[OpenAIChatModelConnection, list[str |
None]]:
@@ -366,6 +388,7 @@ def test_query_agrees_with_the_native_branch() -> None:
for schema in (
OutputSchema(output_schema=Person),
+ OutputSchema(output_schema=Labelled),
OutputSchema(output_schema=row_type),
None,
):
diff --git
a/python/flink_agents/integrations/chat_models/vllm/tests/test_vllm_chat_model.py
b/python/flink_agents/integrations/chat_models/vllm/tests/test_vllm_chat_model.py
index 9547e1b4d..643ca3faa 100644
---
a/python/flink_agents/integrations/chat_models/vllm/tests/test_vllm_chat_model.py
+++
b/python/flink_agents/integrations/chat_models/vllm/tests/test_vllm_chat_model.py
@@ -83,6 +83,12 @@ class _Person(BaseModel):
age: int
+class _Tally(BaseModel):
+ """A schema carrying a map, which vLLM's guided decoding can enforce."""
+
+ counts: dict[str, int]
+
+
@pytest.mark.parametrize(
("model", "expected"),
[
@@ -138,6 +144,27 @@ def test_native_response_format_applied_for_qwen_model()
-> None:
assert kwargs["response_format"]["type"] == "json_schema"
+def test_map_schema_is_sent_natively_with_its_value_type() -> None:
+ # OpenAI keeps a map off the native path because strict mode rejects it.
vLLM
+ # does not apply strict mode, so the map is sent and keeps its value type.
+ connection, mock_client = _connection_with_mock_client()
+ schema = OutputSchema(output_schema=_Tally)
+ model = "Qwen/Qwen2.5-7B-Instruct"
+
+ assert (
+ connection.supports_native_structured_output(schema, [], {"model":
model})
+ is NativeStructuredOutputSupport.NATIVE_RECOMMENDED
+ )
+
+ connection.chat(
+ [ChatMessage.of(MessageRole.USER, "hi")], model=model,
output_schema=schema
+ )
+
+ kwargs = mock_client.chat.completions.create.call_args.kwargs
+ sent = kwargs["response_format"]["json_schema"]["schema"]
+ assert sent["properties"]["counts"]["additionalProperties"] == {"type":
"integer"}
+
+
def test_capability_override_shapes_only_the_query() -> None:
# A blank model is the one input the vLLM capability check declines. The
query
# reports it FEASIBLE, and the request still carries the schema it was
handed.
diff --git
a/python/flink_agents/integrations/chat_models/vllm/vllm_chat_model.py
b/python/flink_agents/integrations/chat_models/vllm/vllm_chat_model.py
index 1dba9347f..a427801d1 100644
--- a/python/flink_agents/integrations/chat_models/vllm/vllm_chat_model.py
+++ b/python/flink_agents/integrations/chat_models/vllm/vllm_chat_model.py
@@ -15,13 +15,16 @@
# See the License for the specific language governing permissions and
# limitations under the License.
#################################################################################
-from typing import Any
+from typing import Any, List, Mapping
from typing_extensions import override
+from flink_agents.api.agents.types import OutputSchema
+from flink_agents.api.tools.tool import Tool
from flink_agents.integrations.chat_models.openai.openai_chat_model import (
OpenAIChatModelConnection,
OpenAIChatModelSetup,
+ _native_output_model,
)
DEFAULT_VLLM_API_BASE_URL = "http://localhost:8000/v1";
@@ -86,6 +89,23 @@ class VLLMChatModelConnection(OpenAIChatModelConnection):
"""
return bool(effective_model and effective_model.strip())
+ @override
+ def _can_apply_native_structured_output(
+ self,
+ output_schema: OutputSchema | None,
+ tools: List[Tool] | None,
+ model_kwargs: Mapping[str, Any] | None,
+ ) -> bool:
+ """Whether a request built from these inputs would carry a native
+ ``response_format``, leaving the effective model's capability out of
the answer.
+
+ Unlike OpenAI, a schema that admits undeclared keys, such as one with a
+ ``dict`` field, still applies: vLLM does not enforce OpenAI's
strict-mode
+ rules, and its guided decoding honours ``additionalProperties``, so a
map
+ keeps its value type. Only a ``BaseModel`` subclass has a native
translation.
+ """
+ return _native_output_model(output_schema) is not None
+
class VLLMChatModelSetup(OpenAIChatModelSetup):
"""Settings for a chat model served by vLLM.
