This is an automated email from the ASF dual-hosted git repository.
wenjin272 pushed a commit to branch main
in repository https://gitbox.apache.org/repos/asf/flink-agents.git
The following commit(s) were added to refs/heads/main by this push:
new 31c0259a [api][java][python] Name the model the capability predicate
judges (#1128)
31c0259a is described below
commit 31c0259aace0f9123036946a0c38a3e34bd32e51
Author: Weiqing Yang <[email protected]>
AuthorDate: Wed Sep 16 20:18:53 2026 -0700
[api][java][python] Name the model the capability predicate judges (#1128)
Generated-by: Claude Code 2.1.272 (Claude Opus 5)
---
.../api/chat/model/BaseChatModelConnection.java | 46 +++++++--
.../agents/api/chat/model/BaseChatModelTest.java | 28 ++++++
.../anthropic/AnthropicChatModelConnection.java | 14 +++
.../AnthropicChatModelConnectionTest.java | 43 +++++++++
.../bedrock/BedrockChatModelConnection.java | 16 ++++
.../bedrock/BedrockChatModelConnectionTest.java | 56 +++++++++++
.../gemini/GeminiChatModelConnection.java | 17 ++++
.../gemini/GeminiChatModelConnectionTest.java | 70 ++++++++++++++
.../openai/AzureOpenAIChatModelConnection.java | 14 +++
.../openai/OpenAICompletionsConnection.java | 13 +++
.../openai/OpenAIResponsesModelConnection.java | 13 +++
.../openai/AzureOpenAIChatModelConnectionTest.java | 63 +++++++++++++
.../openai/OpenAICompletionsConnectionTest.java | 54 +++++++++++
.../openai/OpenAIResponsesModelConnectionTest.java | 12 +++
python/flink_agents/api/chat_models/chat_model.py | 52 ++++++++--
.../api/chat_models/tests/test_chat_model_base.py | 53 +++++++++++
.../tests/test_anthropic_response_parsing.py | 56 +++++++++++
.../chat_models/azure/azure_openai_chat_model.py | 17 +++-
.../test_azure_openai_native_structured_output.py | 105 +++++++++++++++++++++
.../tests/test_openai_native_structured_output.py | 56 +++++++++++
.../tests/test_ollama_native_structured_output.py | 64 +++++++++++++
.../tests/test_tongyi_native_structured_output.py | 89 +++++++++++++++++
.../integrations/chat_models/tongyi_chat_model.py | 18 +++-
.../tests/test_watsonx_native_structured_output.py | 95 +++++++++++++++++++
.../chat_models/watsonx/watsonx_chat_model.py | 18 +++-
25 files changed, 1065 insertions(+), 17 deletions(-)
diff --git
a/api/src/main/java/org/apache/flink/agents/api/chat/model/BaseChatModelConnection.java
b/api/src/main/java/org/apache/flink/agents/api/chat/model/BaseChatModelConnection.java
index 57d0a93b..79bb5ed6 100644
---
a/api/src/main/java/org/apache/flink/agents/api/chat/model/BaseChatModelConnection.java
+++
b/api/src/main/java/org/apache/flink/agents/api/chat/model/BaseChatModelConnection.java
@@ -52,21 +52,53 @@ public abstract class BaseChatModelConnection extends
Resource {
* model.
*
* <p>Capability is <b>model-dependent</b>, not connection-wide: a single
provider connection
- * commonly serves both models that accept a native schema parameter and
models that do not. It
- * is therefore evaluated against the <i>effective</i> model at
request-build time — the model
- * actually being called, which per-request parameters may override.
+ * commonly serves both models that accept a native schema parameter and
models that do not. The
+ * model to ask about is whatever {@link #effectiveModelFor(Map)} returns
for the parameters a
+ * request would be built from, and a caller outside the connection asks
that hook and nothing
+ * else. Such a caller must not substitute the identifier the request is
issued against: on a
+ * deployment-based provider the request targets a deployment name the
user chose while
+ * capability belongs to the model backing it, so the two disagree in both
directions.
*
- * <p>The default {@code false} keeps a connection on the
prompt-engineering fallback. An
- * unrecognized model must report {@code false} so that it degrades to the
fallback rather than
- * failing at the provider.
+ * <p>The default {@code false} keeps a connection on the
prompt-engineering fallback. A
+ * connection that classifies by model name must report {@code false} for
a name it does not
+ * recognize, so that it degrades to the fallback rather than failing at
the provider. A
+ * connection whose capability belongs to the endpoint rather than to the
model answers for the
+ * endpoint instead, and may report {@code true} for a name it has never
seen.
*
- * @param effectiveModel the model the request will be issued against, may
be null
+ * @param effectiveModel the model whose capability is being asked about,
as returned by {@link
+ * #effectiveModelFor(Map)}, may be null
* @return true if a schema can be applied natively for {@code
effectiveModel}
*/
protected boolean supportsNativeStructuredOutput(String effectiveModel) {
return false;
}
+ /**
+ * The model whose capability {@link
#supportsNativeStructuredOutput(String)} should be asked
+ * about, derived from the parameters a request would be built from.
+ *
+ * <p>Overriding this is how a connection whose effective model is not the
{@code model}
+ * parameter verbatim keeps the capability answer and the request in
agreement. A connection
+ * that falls back to a configured default model when the parameter is
absent applies that
+ * fallback here, and a deployment-based provider returns the model
backing the deployment
+ * rather than the deployment the request targets.
+ *
+ * <p>Answers for whatever it is given rather than validating: a model
that does not resolve
+ * comes back null, and {@link #supportsNativeStructuredOutput(String)}
must accept null rather
+ * than throwing. What a null model resolves to is that predicate's own
answer; the default, and
+ * every override that classifies by name, reports it not capable.
+ *
+ * <p>An override must read {@code modelParams} without consuming it, so
that the same map still
+ * builds the request the answer was about.
+ *
+ * @param modelParams the parameters a request would be built from, may be
null
+ * @return the model to ask the capability predicate about, or null if
none resolves
+ */
+ @Nullable
+ protected String effectiveModelFor(@Nullable Map<String, Object>
modelParams) {
+ return modelParams == null ? null : (String) modelParams.get("model");
+ }
+
/**
* Process a chat request and return a chat response.
*
diff --git
a/api/src/test/java/org/apache/flink/agents/api/chat/model/BaseChatModelTest.java
b/api/src/test/java/org/apache/flink/agents/api/chat/model/BaseChatModelTest.java
index d27d79cb..bb030f77 100644
---
a/api/src/test/java/org/apache/flink/agents/api/chat/model/BaseChatModelTest.java
+++
b/api/src/test/java/org/apache/flink/agents/api/chat/model/BaseChatModelTest.java
@@ -460,4 +460,32 @@ class BaseChatModelTest {
assertNotNull(response);
assertTrue(response.getContent().length() > 0);
}
+
+ @Test
+ @DisplayName("Effective model defaults to the model parameter a request
would be built from")
+ void testEffectiveModelForReadsTheModelParameter() {
+ RecordingConnection connection = new RecordingConnection();
+ Map<String, Object> modelParams = new HashMap<>();
+ modelParams.put("model", "gpt-4o");
+
+ assertEquals("gpt-4o", connection.effectiveModelFor(modelParams));
+ }
+
+ @Test
+ @DisplayName("Effective model is null when the parameters name no model")
+ void testEffectiveModelForReturnsNullWhenModelParameterAbsent() {
+ RecordingConnection connection = new RecordingConnection();
+
+ // A connection carrying no default of its own has no model to
resolve, and the capability
+ // predicate reports a null model not capable rather than throwing.
+ assertNull(connection.effectiveModelFor(Map.of("temperature", 0.5)));
+ }
+
+ @Test
+ @DisplayName("Effective model is null for null parameters")
+ void testEffectiveModelForReturnsNullForNullParameters() {
+ RecordingConnection connection = new RecordingConnection();
+
+ assertNull(connection.effectiveModelFor(null));
+ }
}
diff --git
a/integrations/chat-models/anthropic/src/main/java/org/apache/flink/agents/integrations/chatmodels/anthropic/AnthropicChatModelConnection.java
b/integrations/chat-models/anthropic/src/main/java/org/apache/flink/agents/integrations/chatmodels/anthropic/AnthropicChatModelConnection.java
index 8b613b02..116b3ad1 100644
---
a/integrations/chat-models/anthropic/src/main/java/org/apache/flink/agents/integrations/chatmodels/anthropic/AnthropicChatModelConnection.java
+++
b/integrations/chat-models/anthropic/src/main/java/org/apache/flink/agents/integrations/chatmodels/anthropic/AnthropicChatModelConnection.java
@@ -182,6 +182,20 @@ public class AnthropicChatModelConnection extends
BaseChatModelConnection {
||
effectiveModel.startsWith(prefix + "-"));
}
+ /**
+ * The {@code model} parameter, falling back to the model configured on
the connection when the
+ * call names none, which is how the request itself resolves the model it
is issued against.
+ */
+ @Override
+ protected String effectiveModelFor(Map<String, Object> modelParams) {
+ Object modelObj = modelParams != null ? modelParams.get("model") :
null;
+ String modelName = modelObj != null ? modelObj.toString() : null;
+ if (modelName == null || modelName.isBlank()) {
+ return this.defaultModel;
+ }
+ return modelName;
+ }
+
// Models Anthropic documents as rejecting assistant-message prefilling.
Source of truth:
//
https://platform.claude.com/docs/en/build-with-claude/working-with-messages#putting-words-in-claudes-mouth
//
diff --git
a/integrations/chat-models/anthropic/src/test/java/org/apache/flink/agents/integrations/chatmodels/anthropic/AnthropicChatModelConnectionTest.java
b/integrations/chat-models/anthropic/src/test/java/org/apache/flink/agents/integrations/chatmodels/anthropic/AnthropicChatModelConnectionTest.java
index daeb1084..0246161b 100644
---
a/integrations/chat-models/anthropic/src/test/java/org/apache/flink/agents/integrations/chatmodels/anthropic/AnthropicChatModelConnectionTest.java
+++
b/integrations/chat-models/anthropic/src/test/java/org/apache/flink/agents/integrations/chatmodels/anthropic/AnthropicChatModelConnectionTest.java
@@ -396,6 +396,49 @@ class AnthropicChatModelConnectionTest {
assertThat(build(CAPABLE_MODEL, null,
null).params.outputConfig()).isEmpty();
}
+ @Test
+ @DisplayName("the effective model falls back to the configured default")
+ void testEffectiveModelForFallsBackToTheConfiguredDefault() {
+ // buildRequest resolves the model the same way before feeding the
capability predicate,
+ // and connection() is configured with claude-sonnet-4-20250514.
+ assertThat(connection().effectiveModelFor(params(null)))
+ .isEqualTo("claude-sonnet-4-20250514");
+
assertThat(connection().effectiveModelFor(paramsWithModel(CAPABLE_MODEL, null)))
+ .isEqualTo(CAPABLE_MODEL);
+ // A blank model is not a model. The builder substitutes the default
for it, so the hook
+ // has to as well or the two disagree on exactly this input.
+ assertThat(connection().effectiveModelFor(paramsWithModel(" ",
null)))
+ .isEqualTo("claude-sonnet-4-20250514");
+ }
+
+ @Test
+ @DisplayName("the model the request builder judges is the one the hook
names")
+ void testEffectiveModelForNamesTheModelTheBuilderJudges() {
+ // The hook duplicates the builder's resolution rather than
centralizing it, so only
+ // capturing what the builder feeds the predicate keeps the two from
drifting apart.
+ AtomicReference<String> judged = new AtomicReference<>();
+ AnthropicChatModelConnection connection =
+ new
AnthropicChatModelConnection(descriptor("claude-sonnet-4-20250514"), NOOP) {
+ @Override
+ protected boolean supportsNativeStructuredOutput(String
effectiveModel) {
+ judged.set(effectiveModel);
+ return
super.supportsNativeStructuredOutput(effectiveModel);
+ }
+ };
+
+ for (Map<String, Object> modelParams :
+ List.<Map<String, Object>>of(
+ paramsWithModel(CAPABLE_MODEL, null),
+ paramsWithModel(" ", null),
+ params(null))) {
+ String named = connection.effectiveModelFor(modelParams);
+
+ connection.buildRequest(userMessage(), List.of(), modelParams,
Answer.class);
+
+ assertThat(judged.get()).isEqualTo(named);
+ }
+ }
+
@Test
@DisplayName("a schema that is not a Class keeps the prompt fallback")
void testNonClassSchemaKeepsFallback() {
diff --git
a/integrations/chat-models/bedrock/src/main/java/org/apache/flink/agents/integrations/chatmodels/bedrock/BedrockChatModelConnection.java
b/integrations/chat-models/bedrock/src/main/java/org/apache/flink/agents/integrations/chatmodels/bedrock/BedrockChatModelConnection.java
index 29989ab6..97ca7fd7 100644
---
a/integrations/chat-models/bedrock/src/main/java/org/apache/flink/agents/integrations/chatmodels/bedrock/BedrockChatModelConnection.java
+++
b/integrations/chat-models/bedrock/src/main/java/org/apache/flink/agents/integrations/chatmodels/bedrock/BedrockChatModelConnection.java
@@ -237,6 +237,22 @@ public class BedrockChatModelConnection extends
BaseChatModelConnection {
return profile.matches() &&
NATIVE_STRUCTURED_OUTPUT_MODELS.contains(profile.group(1));
}
+ /**
+ * The {@code model} parameter, falling back to the model configured on
the connection when the
+ * call names none, which is how the request itself resolves the model it
is issued against.
+ *
+ * <p>Resolving to nothing comes back null rather than raising the way
{@code resolveModel}
+ * does, because the capability predicate reports a null model not capable.
+ */
+ @Override
+ protected String effectiveModelFor(Map<String, Object> modelParams) {
+ String model = modelParams != null ? (String) modelParams.get("model")
: null;
+ if (model == null || model.isBlank()) {
+ return this.defaultModel;
+ }
+ return model;
+ }
+
@Override
public ChatMessage chat(
List<ChatMessage> messages, List<Tool> tools, Map<String, Object>
modelParams) {
diff --git
a/integrations/chat-models/bedrock/src/test/java/org/apache/flink/agents/integrations/chatmodels/bedrock/BedrockChatModelConnectionTest.java
b/integrations/chat-models/bedrock/src/test/java/org/apache/flink/agents/integrations/chatmodels/bedrock/BedrockChatModelConnectionTest.java
index 334c94a7..3cd315c9 100644
---
a/integrations/chat-models/bedrock/src/test/java/org/apache/flink/agents/integrations/chatmodels/bedrock/BedrockChatModelConnectionTest.java
+++
b/integrations/chat-models/bedrock/src/test/java/org/apache/flink/agents/integrations/chatmodels/bedrock/BedrockChatModelConnectionTest.java
@@ -46,6 +46,7 @@ import
software.amazon.awssdk.services.bedrockruntime.model.Message;
import software.amazon.awssdk.services.bedrockruntime.model.OutputFormatType;
import java.util.*;
+import java.util.concurrent.atomic.AtomicReference;
import java.util.stream.Stream;
import static org.assertj.core.api.Assertions.assertThat;
@@ -130,6 +131,61 @@ class BedrockChatModelConnectionTest {
.isInstanceOf(RuntimeException.class);
}
+ @Test
+ @DisplayName("the effective model falls back to the configured default")
+ void testEffectiveModelForFallsBackToTheConfiguredDefault() {
+ // buildRequest resolves the model the same way before feeding the
capability predicate.
+ assertThat(connection().effectiveModelFor(new HashMap<>()))
+ .isEqualTo("us.anthropic.claude-sonnet-4-20250514-v1:0");
+
assertThat(connection().effectiveModelFor(params("qwen.qwen3-32b-v1:0")))
+ .isEqualTo("qwen.qwen3-32b-v1:0");
+ // A blank model is not a model. resolveModel substitutes the default
for it, so the hook
+ // has to as well or the two disagree on exactly this input.
+ assertThat(connection().effectiveModelFor(params(" ")))
+ .isEqualTo("us.anthropic.claude-sonnet-4-20250514-v1:0");
+ }
+
+ @Test
+ @DisplayName("the model the request builder judges is the one the hook
names")
+ void testEffectiveModelForNamesTheModelTheBuilderJudges() {
+ // The hook duplicates resolveModel rather than calling it, so only
capturing what the
+ // builder feeds the predicate keeps the two from drifting apart.
+ AtomicReference<String> judged = new AtomicReference<>();
+ BedrockChatModelConnection connection =
+ new BedrockChatModelConnection(
+ descriptor("us-east-1",
"us.anthropic.claude-sonnet-4-20250514-v1:0"),
+ NOOP) {
+ @Override
+ protected boolean supportsNativeStructuredOutput(String
effectiveModel) {
+ judged.set(effectiveModel);
+ return
super.supportsNativeStructuredOutput(effectiveModel);
+ }
+ };
+
+ for (Map<String, Object> modelParams :
+ List.<Map<String, Object>>of(
+ params("qwen.qwen3-32b-v1:0"), params(" "), new
HashMap<>())) {
+ String named = connection.effectiveModelFor(modelParams);
+
+ connection.buildRequest(
+ List.of(ChatMessage.user("hello")), null, modelParams,
Profile.class);
+
+ assertThat(judged.get()).isEqualTo(named);
+ }
+ }
+
+ @Test
+ @DisplayName("the effective model is null rather than throwing when none
resolves")
+ void testEffectiveModelForReturnsNullWhenNoModelResolves() {
+ // resolveModel rejects an unresolvable model before a request is
built. This query is
+ // part of the connection contract and answers for whatever it is
given, so it reports
+ // null, which the capability predicate treats as not capable.
+ BedrockChatModelConnection conn =
+ new BedrockChatModelConnection(descriptor("us-east-1", null),
NOOP);
+
+ assertThat(conn.effectiveModelFor(new HashMap<>())).isNull();
+ }
+
@Test
@DisplayName("chat accepts a POJO output schema instead of rejecting it as
untranslatable")
void testChatAcceptsPojoOutputSchema() {
diff --git
a/integrations/chat-models/gemini/src/main/java/org/apache/flink/agents/integrations/chatmodels/gemini/GeminiChatModelConnection.java
b/integrations/chat-models/gemini/src/main/java/org/apache/flink/agents/integrations/chatmodels/gemini/GeminiChatModelConnection.java
index 3c34c431..1f72f524 100644
---
a/integrations/chat-models/gemini/src/main/java/org/apache/flink/agents/integrations/chatmodels/gemini/GeminiChatModelConnection.java
+++
b/integrations/chat-models/gemini/src/main/java/org/apache/flink/agents/integrations/chatmodels/gemini/GeminiChatModelConnection.java
@@ -234,6 +234,23 @@ public class GeminiChatModelConnection extends
BaseChatModelConnection {
&& effectiveModel.length() >
NATIVE_STRUCTURED_OUTPUT_FAMILY_PREFIX.length();
}
+ /**
+ * The {@code model} parameter, falling back to the model configured on
the connection when the
+ * call names none, which is how the request itself resolves the model it
is issued against.
+ *
+ * <p>Resolving to nothing comes back null rather than raising the way
request building does,
+ * because the capability predicate reports a null model not capable.
+ */
+ @Override
+ protected String effectiveModelFor(Map<String, Object> modelParams) {
+ Object modelObj = modelParams != null ? modelParams.get("model") :
null;
+ String modelName = modelObj != null ? modelObj.toString() : null;
+ if (modelName == null || modelName.isBlank()) {
+ return this.defaultModel;
+ }
+ return modelName;
+ }
+
@Override
public ChatMessage chat(
List<ChatMessage> messages,
diff --git
a/integrations/chat-models/gemini/src/test/java/org/apache/flink/agents/integrations/chatmodels/gemini/GeminiChatModelConnectionTest.java
b/integrations/chat-models/gemini/src/test/java/org/apache/flink/agents/integrations/chatmodels/gemini/GeminiChatModelConnectionTest.java
index 1cadda4e..ea27d4ca 100644
---
a/integrations/chat-models/gemini/src/test/java/org/apache/flink/agents/integrations/chatmodels/gemini/GeminiChatModelConnectionTest.java
+++
b/integrations/chat-models/gemini/src/test/java/org/apache/flink/agents/integrations/chatmodels/gemini/GeminiChatModelConnectionTest.java
@@ -53,6 +53,7 @@ import java.util.HashMap;
import java.util.List;
import java.util.Map;
import java.util.Optional;
+import java.util.concurrent.atomic.AtomicReference;
import static org.assertj.core.api.Assertions.assertThat;
import static org.assertj.core.api.Assertions.assertThatThrownBy;
@@ -920,4 +921,73 @@ class GeminiChatModelConnectionTest {
.isInstanceOf(IllegalArgumentException.class)
.hasMessageContaining("model name must be provided");
}
+
+ @Test
+ @DisplayName("The effective model falls back to the configured default")
+ void effectiveModelForFallsBackToTheConfiguredDefault() {
+ // chat resolves the model the same way before feeding the capability
predicate, and
+ // connection() is configured with gemini-3-pro-preview.
+ Map<String, Object> modelParams = params();
+
+
assertThat(connection().effectiveModelFor(modelParams)).isEqualTo("gemini-3-pro-preview");
+
+ modelParams.put("model", CAPABLE_MODEL);
+
assertThat(connection().effectiveModelFor(modelParams)).isEqualTo(CAPABLE_MODEL);
+
+ // A blank model is not a model. The request path substitutes the
default for it, so the
+ // hook has to as well or the two disagree on exactly this input.
+ modelParams.put("model", " ");
+
assertThat(connection().effectiveModelFor(modelParams)).isEqualTo("gemini-3-pro-preview");
+ }
+
+ /** Stops the request after the config is assembled, so no call reaches
the provider. */
+ private static final class StopBeforeRequest extends RuntimeException {}
+
+ private static Map<String, Object> paramsWithModel(String model) {
+ Map<String, Object> modelParams = params();
+ modelParams.put("model", model);
+ return modelParams;
+ }
+
+ @Test
+ @DisplayName("The model the request builder judges is the one the hook
names")
+ void effectiveModelForNamesTheModelTheBuilderJudges() {
+ // chat resolves the model and hands it to buildConfig, which is where
the predicate is fed.
+ // Overriding buildConfig to stop once it has run binds the hook to
the request without
+ // reaching the provider. Asserting each side against a literal would
let the two drift in
+ // step, which is the one failure this has to catch.
+ AtomicReference<String> judged = new AtomicReference<>();
+ GeminiChatModelConnection connection =
+ new GeminiChatModelConnection(
+ descriptor("test-key", null, "gemini-3-pro-preview"),
NOOP) {
+ @Override
+ protected boolean supportsNativeStructuredOutput(String
effectiveModel) {
+ judged.set(effectiveModel);
+ return
super.supportsNativeStructuredOutput(effectiveModel);
+ }
+
+ @Override
+ GenerateContentConfig buildConfig(
+ List<ChatMessage> messages,
+ List<Tool> tools,
+ Map<String, Object> arguments,
+ String modelName,
+ Object outputSchema) {
+ super.buildConfig(messages, tools, arguments,
modelName, outputSchema);
+ throw new StopBeforeRequest();
+ }
+ };
+
+ for (Map<String, Object> modelParams :
+ List.<Map<String, Object>>of(
+ paramsWithModel(CAPABLE_MODEL), paramsWithModel("
"), params())) {
+ String named = connection.effectiveModelFor(modelParams);
+
+ assertThatThrownBy(
+ () -> connection.chat(userMessage(), null,
modelParams, Report.class))
+ .hasRootCauseInstanceOf(StopBeforeRequest.class);
+
+ assertThat(judged.get()).isEqualTo(named);
+ }
+ }
}
diff --git
a/integrations/chat-models/openai/src/main/java/org/apache/flink/agents/integrations/chatmodels/openai/AzureOpenAIChatModelConnection.java
b/integrations/chat-models/openai/src/main/java/org/apache/flink/agents/integrations/chatmodels/openai/AzureOpenAIChatModelConnection.java
index 4ce122ad..440a7c8d 100644
---
a/integrations/chat-models/openai/src/main/java/org/apache/flink/agents/integrations/chatmodels/openai/AzureOpenAIChatModelConnection.java
+++
b/integrations/chat-models/openai/src/main/java/org/apache/flink/agents/integrations/chatmodels/openai/AzureOpenAIChatModelConnection.java
@@ -223,6 +223,20 @@ public class AzureOpenAIChatModelConnection extends
BaseChatModelConnection {
return NATIVE_STRUCTURED_OUTPUT_MODELS.contains(effectiveModel);
}
+ /**
+ * The model backing the deployment, read from {@code
model_of_azure_deployment}.
+ *
+ * <p>The {@code model} parameter carries the deployment name the request
is issued against.
+ * That name is chosen by the user, and although it commonly echoes the
model behind it, nothing
+ * keeps the two in step once the deployment is repointed, so it is never
the answer here.
+ * Leaving the backing model unset keeps even a capable deployment on the
prompt-engineering
+ * fallback rather than classifying a deployment name on its spelling.
+ */
+ @Override
+ protected String effectiveModelFor(Map<String, Object> modelParams) {
+ return modelParams == null ? null : (String)
modelParams.get("model_of_azure_deployment");
+ }
+
/**
* Whether the configured api-version reaches the structured-output floor.
*
diff --git
a/integrations/chat-models/openai/src/main/java/org/apache/flink/agents/integrations/chatmodels/openai/OpenAICompletionsConnection.java
b/integrations/chat-models/openai/src/main/java/org/apache/flink/agents/integrations/chatmodels/openai/OpenAICompletionsConnection.java
index 4864769e..da8b2dc2 100644
---
a/integrations/chat-models/openai/src/main/java/org/apache/flink/agents/integrations/chatmodels/openai/OpenAICompletionsConnection.java
+++
b/integrations/chat-models/openai/src/main/java/org/apache/flink/agents/integrations/chatmodels/openai/OpenAICompletionsConnection.java
@@ -171,6 +171,19 @@ public class OpenAICompletionsConnection extends
BaseChatModelConnection {
|| NATIVE_STRUCTURED_OUTPUT_MODELS.contains(effectiveModel);
}
+ /**
+ * The {@code model} parameter, falling back to the model configured on
the connection when the
+ * call names none, which is how the request itself resolves the model it
is issued against.
+ */
+ @Override
+ protected String effectiveModelFor(Map<String, Object> modelParams) {
+ String modelName = modelParams != null ? (String)
modelParams.get("model") : null;
+ if (modelName == null || modelName.isBlank()) {
+ return this.defaultModel;
+ }
+ return modelName;
+ }
+
/**
* Returns the model response. When the provider reports a finish reason
it is carried verbatim
* in {@code extraArgs} under {@code finish_reason}, including values
outside the documented
diff --git
a/integrations/chat-models/openai/src/main/java/org/apache/flink/agents/integrations/chatmodels/openai/OpenAIResponsesModelConnection.java
b/integrations/chat-models/openai/src/main/java/org/apache/flink/agents/integrations/chatmodels/openai/OpenAIResponsesModelConnection.java
index 5dca0b6a..77d16c21 100644
---
a/integrations/chat-models/openai/src/main/java/org/apache/flink/agents/integrations/chatmodels/openai/OpenAIResponsesModelConnection.java
+++
b/integrations/chat-models/openai/src/main/java/org/apache/flink/agents/integrations/chatmodels/openai/OpenAIResponsesModelConnection.java
@@ -124,6 +124,19 @@ public class OpenAIResponsesModelConnection extends
BaseChatModelConnection {
this.client = builder.build();
}
+ /**
+ * The {@code model} parameter, falling back to the model configured on
the connection when the
+ * call names none, which is how the request itself resolves the model it
is issued against.
+ */
+ @Override
+ protected String effectiveModelFor(Map<String, Object> modelParams) {
+ String modelName = modelParams != null ? (String)
modelParams.get("model") : null;
+ if (modelName == null || modelName.isBlank()) {
+ return this.defaultModel;
+ }
+ return modelName;
+ }
+
@Override
public ChatMessage chat(
List<ChatMessage> messages,
diff --git
a/integrations/chat-models/openai/src/test/java/org/apache/flink/agents/integrations/chatmodels/openai/AzureOpenAIChatModelConnectionTest.java
b/integrations/chat-models/openai/src/test/java/org/apache/flink/agents/integrations/chatmodels/openai/AzureOpenAIChatModelConnectionTest.java
index 24d448a7..f23cca2e 100644
---
a/integrations/chat-models/openai/src/test/java/org/apache/flink/agents/integrations/chatmodels/openai/AzureOpenAIChatModelConnectionTest.java
+++
b/integrations/chat-models/openai/src/test/java/org/apache/flink/agents/integrations/chatmodels/openai/AzureOpenAIChatModelConnectionTest.java
@@ -357,6 +357,69 @@ class AzureOpenAIChatModelConnectionTest {
assertThat(request.responseFormat()).isEmpty();
}
+ @Test
+ @DisplayName("Effective model is the model backing the deployment, not the
deployment")
+ void testEffectiveModelForReturnsBackingModelNotDeployment() {
+ // params() addresses DEPLOYMENT under "model". Capability belongs to
the model backing
+ // that deployment, which is why buildRequest feeds the predicate
+ // model_of_azure_deployment instead.
+
assertThat(connection().effectiveModelFor(params("gpt-4o-mini"))).isEqualTo("gpt-4o-mini");
+ }
+
+ @Test
+ @DisplayName(
+ "The model the request builder judges is the one the hook names,
never the deployment")
+ void testEffectiveModelForNamesTheModelTheBuilderJudges() {
+ // The builder reads the backing model under its own key and feeds
that to the predicate.
+ // Capturing the value it actually judges is what ties the hook to the
request; comparing
+ // each against a literal would let the two drift in step.
+ AtomicReference<String> judged = new AtomicReference<>();
+ ResourceDescriptor desc =
+ connectionDescriptor()
+ .addInitialArgument("api_key", "test-key")
+ .addInitialArgument("api_version", CAPABLE_API_VERSION)
+ .addInitialArgument("azure_endpoint",
"https://example.openai.azure.com";)
+ .build();
+ AzureOpenAIChatModelConnection connection =
+ new AzureOpenAIChatModelConnection(desc, NOOP) {
+ @Override
+ protected boolean supportsNativeStructuredOutput(String
effectiveModel) {
+ judged.set(effectiveModel);
+ return
super.supportsNativeStructuredOutput(effectiveModel);
+ }
+ };
+
+ Map<String, Object> backed = params("gpt-4o-mini");
+ String named = connection.effectiveModelFor(backed);
+ connection.buildRequest(userMessage(), List.of(), backed,
Person.class);
+ assertThat(judged.get()).isEqualTo(named).isNotEqualTo(DEPLOYMENT);
+
+ // The unset case has to agree too: the builder judges nothing, and so
does the hook.
+ Map<String, Object> unbacked = params(null);
+ assertThat(connection.effectiveModelFor(unbacked)).isNull();
+ connection.buildRequest(userMessage(), List.of(), unbacked,
Person.class);
+ assertThat(judged.get()).isNull();
+ }
+
+ @Test
+ @DisplayName("Effective model reads the backing model without consuming
it")
+ void testEffectiveModelForDoesNotConsumeTheBackingModel() {
+ // The request builder removes this very key. An override copying that
idiom would hand the
+ // builder a map with no backing model, and the native branch would
silently disappear.
+ Map<String, Object> modelParams = params("gpt-4o-mini");
+
+
assertThat(connection().effectiveModelFor(modelParams)).isEqualTo("gpt-4o-mini");
+ assertThat(modelParams).containsEntry("model_of_azure_deployment",
"gpt-4o-mini");
+ }
+
+ @Test
+ @DisplayName("Effective model is null when the backing model is unset")
+ void testEffectiveModelForReturnsNullWhenBackingModelUnset() {
+ // Falling back to the deployment name would classify a user-chosen
name on nothing but
+ // its spelling, so an unset backing model resolves to nothing at all.
+ assertThat(connection().effectiveModelFor(params(null))).isNull();
+ }
+
@Test
@DisplayName("Native NOT applied for a backing model outside the
allowlist")
void testNativeNotAppliedForUnknownDeploymentModel() {
diff --git
a/integrations/chat-models/openai/src/test/java/org/apache/flink/agents/integrations/chatmodels/openai/OpenAICompletionsConnectionTest.java
b/integrations/chat-models/openai/src/test/java/org/apache/flink/agents/integrations/chatmodels/openai/OpenAICompletionsConnectionTest.java
index 851e9004..76b6b0ac 100644
---
a/integrations/chat-models/openai/src/test/java/org/apache/flink/agents/integrations/chatmodels/openai/OpenAICompletionsConnectionTest.java
+++
b/integrations/chat-models/openai/src/test/java/org/apache/flink/agents/integrations/chatmodels/openai/OpenAICompletionsConnectionTest.java
@@ -344,6 +344,60 @@ class OpenAICompletionsConnectionTest {
assertThat(params.responseFormat()).isEmpty();
}
+ @Test
+ @DisplayName("Effective model falls back to the connection default when
the parameter is unset")
+ void testEffectiveModelForFallsBackToTheDefaultModel() {
+ // buildRequest applies the same fallback before feeding the
capability predicate, so an
+ // answer taken without it would disagree with the model the request
is issued against.
+ assertThat(connection().effectiveModelFor(new
HashMap<>())).isEqualTo("gpt-4o");
+
assertThat(connection().effectiveModelFor(params(null))).isEqualTo("gpt-4o");
+ assertThat(connection().effectiveModelFor(params("
"))).isEqualTo("gpt-4o");
+ }
+
+ @Test
+ @DisplayName("The model the request builder judges is the one the hook
names")
+ void testEffectiveModelForNamesTheModelTheBuilderJudges() {
+ // The hook duplicates the builder's own model resolution rather than
centralizing it, so
+ // capturing what the builder actually feeds the predicate is the only
thing that keeps the
+ // two from drifting apart. Asserting each against a literal would let
them drift in step.
+ AtomicReference<String> judged = new AtomicReference<>();
+ OpenAICompletionsConnection connection =
+ new OpenAICompletionsConnection(
+ ResourceDescriptor.Builder.newBuilder(
+
OpenAICompletionsConnection.class.getName())
+ .addInitialArgument("api_key", "test-key")
+ .addInitialArgument("model", "gpt-4o")
+ .build(),
+ NOOP) {
+ @Override
+ protected boolean supportsNativeStructuredOutput(String
effectiveModel) {
+ judged.set(effectiveModel);
+ return
super.supportsNativeStructuredOutput(effectiveModel);
+ }
+ };
+
+ for (Map<String, Object> modelParams :
+ List.<Map<String, Object>>of(
+ params("gpt-4o-mini"), params(" "), new
HashMap<>())) {
+ String named = connection.effectiveModelFor(modelParams);
+
+ connection.buildRequest(userMessage(), List.of(), modelParams,
Person.class);
+
+ assertThat(judged.get()).isEqualTo(named);
+ }
+ }
+
+ @Test
+ @DisplayName("Effective model reads the model parameter without consuming
it")
+ void testEffectiveModelForDoesNotConsumeTheModelParameter() {
+ // buildRequest resolves the model with remove(). A query doing the
same would leave the
+ // caller's parameters without a model for the request it is about to
build.
+ Map<String, Object> modelParams = params("gpt-4o-mini");
+
+
assertThat(connection().effectiveModelFor(modelParams)).isEqualTo("gpt-4o-mini");
+ assertThat(modelParams).containsEntry("model", "gpt-4o-mini");
+ }
+
@Test
@DisplayName("Native NOT applied for a non-POJO schema form (POJO-only
scope)")
void testNativeNotAppliedForNonPojoSchema() {
diff --git
a/integrations/chat-models/openai/src/test/java/org/apache/flink/agents/integrations/chatmodels/openai/OpenAIResponsesModelConnectionTest.java
b/integrations/chat-models/openai/src/test/java/org/apache/flink/agents/integrations/chatmodels/openai/OpenAIResponsesModelConnectionTest.java
index dfb8d625..ac018f2d 100644
---
a/integrations/chat-models/openai/src/test/java/org/apache/flink/agents/integrations/chatmodels/openai/OpenAIResponsesModelConnectionTest.java
+++
b/integrations/chat-models/openai/src/test/java/org/apache/flink/agents/integrations/chatmodels/openai/OpenAIResponsesModelConnectionTest.java
@@ -79,6 +79,18 @@ class OpenAIResponsesModelConnectionTest {
return List.of(new ChatMessage(MessageRole.USER, "hi"));
}
+ @Test
+ @DisplayName("Effective model falls back to the connection default when
the parameter is unset")
+ void testEffectiveModelForFallsBackToTheDefaultModel() {
+ // This connection extends the base directly, so it inherits no
override from the chat
+ // completions connection. Its request builder resolves the model
against the configured
+ // default, and the capability answer has to be about that same model.
+ assertThat(connection().effectiveModelFor(new
HashMap<>())).isEqualTo("gpt-4o");
+
assertThat(connection().effectiveModelFor(params(null))).isEqualTo("gpt-4o");
+ assertThat(connection().effectiveModelFor(params("
"))).isEqualTo("gpt-4o");
+
assertThat(connection().effectiveModelFor(params("gpt-4o-mini"))).isEqualTo("gpt-4o-mini");
+ }
+
@Test
@DisplayName("Constructor throws when api_key is missing")
void testConstructorMissingApiKey() {
diff --git a/python/flink_agents/api/chat_models/chat_model.py
b/python/flink_agents/api/chat_models/chat_model.py
index 11cf6579..133e57ae 100644
--- a/python/flink_agents/api/chat_models/chat_model.py
+++ b/python/flink_agents/api/chat_models/chat_model.py
@@ -135,19 +135,25 @@ class BaseChatModelConnection(Resource, ABC):
Capability is *model-dependent*, not connection-wide: a single provider
connection commonly serves both models that accept a native schema
parameter and
- models that do not, so it is evaluated against the *effective* model at
- request-build time — the model actually being called, which per-request
- parameters may override.
+ models that do not. The model to ask about is whatever
``effective_model_for``
+ returns for the parameters a request would be built from, and a caller
outside
+ the connection asks that hook and nothing else. Such a caller must not
+ substitute the identifier the request is issued against: on a
deployment-based
+ provider the request targets a deployment name the user chose while
capability
+ belongs to the model backing it, so the two disagree in both
directions.
The default ``False`` keeps a connection on the prompt-engineering
fallback. A
- connection that translates a schema into a native provider parameter
overrides
- this; an unrecognized model must report ``False`` so it degrades to
the fallback
- rather than failing at the provider.
+ connection that classifies by model name must report ``False`` for a
name it
+ does not recognize, so that it degrades to the fallback rather than
failing at
+ the provider. A connection whose capability belongs to the endpoint
rather than
+ to the model answers for the endpoint instead, and may report ``True``
for a
+ name it has never seen.
Parameters
----------
effective_model : str | None
- The model the request will be issued against, may be ``None``.
+ The model whose capability is being asked about, as returned by
+ ``effective_model_for``, may be ``None``.
Returns:
-------
@@ -156,6 +162,38 @@ class BaseChatModelConnection(Resource, ABC):
"""
return False
+ def effective_model_for(self, model_kwargs: Mapping[str, Any] | None) ->
str | None:
+ """The model ``supports_native_structured_output`` should be asked
about.
+
+ Derived from the parameters a request would be built from. Overriding
this is
+ how a connection whose effective model is not the ``model`` parameter
verbatim
+ keeps the capability answer and the request in agreement: a connection
that
+ falls back to a configured default model when the parameter is absent
applies
+ that fallback here, and a deployment-based provider returns the model
backing
+ the deployment rather than the deployment the request targets.
+
+ Answers for whatever it is given rather than validating it: a model
that does
+ not resolve comes back ``None``, and
``supports_native_structured_output`` must
+ accept ``None`` rather than raising. What a ``None`` model resolves to
is that
+ predicate's own answer; the default, and every override that
classifies by
+ name, reports it not capable.
+
+ An override must read ``model_kwargs`` without consuming it, so that
the same
+ mapping still builds the request the answer was about.
+
+ Parameters
+ ----------
+ model_kwargs : Mapping[str, Any] | None
+ The parameters a request would be built from, may be ``None``.
+
+ Returns:
+ -------
+ str | None
+ The model to ask the capability predicate about, or ``None`` if
none
+ resolves.
+ """
+ return None if model_kwargs is None else model_kwargs.get("model")
+
def _reject_unsupported_output_schema(
self, output_schema: OutputSchema | None
) -> None:
diff --git a/python/flink_agents/api/chat_models/tests/test_chat_model_base.py
b/python/flink_agents/api/chat_models/tests/test_chat_model_base.py
index a7524973..cd6417ee 100644
--- a/python/flink_agents/api/chat_models/tests/test_chat_model_base.py
+++ b/python/flink_agents/api/chat_models/tests/test_chat_model_base.py
@@ -273,3 +273,56 @@ def test_setup_rejects_unrecognized_constructor_argument()
-> None:
"""Same guarantee as the connection, for BaseChatModelSetup."""
with pytest.raises(ValidationError, match="not_a_real_field"):
_RecordingChatModelSetup(connection="c", model="m",
not_a_real_field="oops")
+
+
+def test_effective_model_for_reads_the_model_param() -> None:
+ """The default effective model is the ``model`` parameter a request
carries."""
+ connection = _RecordingConnection()
+
+ assert connection.effective_model_for({"model": "gpt-4o"}) == "gpt-4o"
+
+
+def test_effective_model_for_returns_none_when_no_model_param() -> None:
+ """A connection carrying no default of its own has no model to resolve.
+
+ The capability predicate reports a ``None`` model not capable rather than
raising,
+ so answering ``None`` degrades to the prompt-engineering fallback.
+ """
+ connection = _RecordingConnection()
+
+ assert connection.effective_model_for({}) is None
+ assert connection.effective_model_for({"temperature": 0.5}) is None
+
+
+def test_effective_model_for_returns_none_for_none_params() -> None:
+ """No parameters at all resolve to no model, rather than raising."""
+ connection = _RecordingConnection()
+
+ assert connection.effective_model_for(None) is None
+
+
+def test_effective_model_for_does_not_normalize_a_blank_model() -> None:
+ """The hook answers with what it was given rather than validating it.
+
+ Substituting a default for a blank model is what the connections carrying a
+ default model do, and it belongs to their override because it is what their
+ request builder does. Doing it here would invent a fallback for
connections that
+ have none.
+ """
+ connection = _RecordingConnection()
+
+ assert connection.effective_model_for({"model": " "}) == " "
+
+
+def test_effective_model_for_does_not_consume_the_params() -> None:
+ """Reading the model leaves the mapping able to build the request it
described.
+
+ An override copying a request builder's ``pop`` idiom would hand that
builder a
+ map with the model already removed, so the contract is pinned on the
default too.
+ """
+ connection = _RecordingConnection()
+ model_kwargs = {"model": "gpt-4o", "temperature": 0.5}
+
+ connection.effective_model_for(model_kwargs)
+
+ assert model_kwargs == {"model": "gpt-4o", "temperature": 0.5}
diff --git
a/python/flink_agents/integrations/chat_models/anthropic/tests/test_anthropic_response_parsing.py
b/python/flink_agents/integrations/chat_models/anthropic/tests/test_anthropic_response_parsing.py
index 33466cea..811f3648 100644
---
a/python/flink_agents/integrations/chat_models/anthropic/tests/test_anthropic_response_parsing.py
+++
b/python/flink_agents/integrations/chat_models/anthropic/tests/test_anthropic_response_parsing.py
@@ -753,3 +753,59 @@ def test_sampling_and_prefill_boundaries_differ() -> None:
assert _supports_json_prefill("claude-sonnet-4-6") is False
assert _supports_sampling_params("claude-sonnet-4-6") is True
assert _sent_sampling("claude-sonnet-4-6", temperature=0.1)["temperature"]
== 0.1
+
+
+def _judging_connection() -> tuple[AnthropicChatModelConnection, list]:
+ """A connection recording every model its request path judges for
capability.
+
+ Subclassing keeps the predicate itself under test rather than standing a
stub in
+ for it: the override notes what it was asked about and delegates to the
real one.
+ """
+ judged: list = []
+
+ class _JudgingConnection(AnthropicChatModelConnection):
+ def supports_native_structured_output(
+ self, effective_model: str | None
+ ) -> bool:
+ judged.append(effective_model)
+ return super().supports_native_structured_output(effective_model)
+
+ connection = _JudgingConnection(api_key="dummy")
+ client = MagicMock()
+ client.messages.create.return_value = Message(
+ id="m",
+ model="claude",
+ role="assistant",
+ type="message",
+ stop_reason="end_turn",
+ content=[TextBlock(type="text", text="ok")],
+ usage=_usage(),
+ )
+ connection._client = client
+ return connection, judged
+
+
[email protected](
+ "model_kwargs",
+ [{"model": _CAPABLE_MODEL}, {"model": _INCAPABLE_MODEL}, {"model": ""},
{}],
+ ids=["capable", "incapable", "blank", "absent"],
+)
+def test_effective_model_for_names_the_model_the_request_judges(
+ model_kwargs: Dict[str, Any],
+) -> None:
+ """The hook names exactly the model the request path asks the predicate
about.
+
+ This connection reads the parameter without a fallback, so the inherited
hook is
+ already the right answer. Pinning it against what the builder judges is
what would
+ catch a fallback being added here without a matching override.
+ """
+ connection, judged = _judging_connection()
+
+ named = connection.effective_model_for(model_kwargs)
+ connection.chat(
+ [ChatMessage(role=MessageRole.USER, content="hi")],
+ output_schema=OutputSchema(output_schema=_Answer),
+ **model_kwargs,
+ )
+
+ assert judged == [named]
diff --git
a/python/flink_agents/integrations/chat_models/azure/azure_openai_chat_model.py
b/python/flink_agents/integrations/chat_models/azure/azure_openai_chat_model.py
index 65a614c0..2e72b8eb 100644
---
a/python/flink_agents/integrations/chat_models/azure/azure_openai_chat_model.py
+++
b/python/flink_agents/integrations/chat_models/azure/azure_openai_chat_model.py
@@ -17,7 +17,7 @@
#################################################################################
import logging
import re
-from typing import Any, Dict, List, Sequence
+from typing import Any, Dict, List, Mapping, Sequence
from openai import NOT_GIVEN, AzureOpenAI
@@ -231,6 +231,21 @@ class
AzureOpenAIChatModelConnection(BaseChatModelConnection):
return False
return effective_model in _NATIVE_STRUCTURED_OUTPUT_MODELS
+ @override
+ def effective_model_for(self, model_kwargs: Mapping[str, Any] | None) ->
str | None:
+ """The model backing the deployment, read from
``model_of_azure_deployment``.
+
+ The ``model`` parameter carries the deployment name the request is
issued
+ against. That name is chosen by the user, and although it commonly
echoes the
+ model behind it, nothing keeps the two in step once the deployment is
+ repointed, so it is never the answer here. Leaving the backing model
unset
+ keeps even a capable deployment on the prompt-engineering fallback
rather than
+ classifying a deployment name on its spelling.
+ """
+ if model_kwargs is None:
+ return None
+ return model_kwargs.get("model_of_azure_deployment")
+
def _api_version_supports_structured_output(self) -> bool:
"""Whether the configured api-version reaches the structured-output
floor.
diff --git
a/python/flink_agents/integrations/chat_models/azure/tests/test_azure_openai_native_structured_output.py
b/python/flink_agents/integrations/chat_models/azure/tests/test_azure_openai_native_structured_output.py
index ade5c536..e15b3f62 100644
---
a/python/flink_agents/integrations/chat_models/azure/tests/test_azure_openai_native_structured_output.py
+++
b/python/flink_agents/integrations/chat_models/azure/tests/test_azure_openai_native_structured_output.py
@@ -512,3 +512,108 @@ def test_map_member_schema_is_accepted_and_sent_whole()
-> None:
_chat_with_schema(conn, Labelled)
response_format = _create_call_kwargs(conn)["response_format"]
assert response_format["json_schema"]["schema"] ==
to_strict_json_schema(Labelled)
+
+
+def _model_kwargs(backing_model: str | None = None) -> dict[str, Any]:
+ """The parameter map a setup hands the connection for one request.
+
+ ``model`` is the deployment, matching every other call in this module; the
backing
+ model travels under its own key and is omitted entirely when unset.
+ """
+ params: dict[str, Any] = {"model": DEPLOYMENT}
+ if backing_model is not None:
+ params["model_of_azure_deployment"] = backing_model
+ return params
+
+
+def _judging_connection() -> tuple[AzureOpenAIChatModelConnection, list[str |
None]]:
+ """A connection recording every model its request path judges for
capability.
+
+ Subclassing keeps the predicate itself under test rather than standing a
stub in
+ for it: the override notes what it was asked about and delegates to the
real one.
+ """
+ judged: list[str | None] = []
+
+ class _JudgingConnection(AzureOpenAIChatModelConnection):
+ def supports_native_structured_output(
+ self, effective_model: str | None
+ ) -> bool:
+ judged.append(effective_model)
+ return super().supports_native_structured_output(effective_model)
+
+ conn = _JudgingConnection(
+ api_key="test-key",
+ azure_endpoint="https://example.openai.azure.com";,
+ api_version=CAPABLE_API_VERSION,
+ )
+ mock_client = MagicMock()
+ mock_message = MagicMock()
+ mock_message.role = "assistant"
+ mock_message.content = "ok"
+ mock_message.tool_calls = None
+ mock_client.chat.completions.create.return_value.choices = [
+ MagicMock(message=mock_message)
+ ]
+ mock_client.chat.completions.create.return_value.usage = None
+ conn._client = mock_client
+ return conn, judged
+
+
+def test_effective_model_for_returns_backing_model() -> None:
+ """Capability is asked about the model behind the deployment, not the
deployment."""
+ assert (
+ _connection().effective_model_for(_model_kwargs("gpt-4o-mini")) ==
"gpt-4o-mini"
+ )
+
+
+def test_effective_model_for_returns_none_when_backing_model_unset() -> None:
+ """An unset backing model resolves to nothing rather than to the
deployment.
+
+ Falling back to the deployment would classify a user-chosen name on
nothing but
+ its spelling, and that name stops tracking the model behind it the moment
the
+ deployment is repointed.
+ """
+ assert _connection().effective_model_for(_model_kwargs()) is None
+
+
+def test_effective_model_for_does_not_consume_the_backing_model() -> None:
+ """The hook reads the key that ``chat`` pops, and has to leave it in place.
+
+ An override copying the builder's ``pop`` idiom would hand the builder a
map with
+ no backing model, and the native branch would silently disappear.
+ """
+ model_kwargs = _model_kwargs("gpt-4o-mini")
+
+ _connection().effective_model_for(model_kwargs)
+
+ assert model_kwargs["model_of_azure_deployment"] == "gpt-4o-mini"
+
+
[email protected](
+ "backing_model",
+ ["gpt-4o-mini", "some-unknown-model", None],
+ ids=["capable", "unknown", "unset"],
+)
+def test_effective_model_for_names_the_model_the_request_judges(
+ backing_model: str | None,
+) -> None:
+ """The hook names exactly the model the request path asks the predicate
about.
+
+ The hook duplicates the builder's resolution rather than centralizing it,
so only
+ capturing what the request feeds the predicate keeps the two from drifting
apart;
+ comparing each against a literal would let them drift in step. A fresh
connection
+ per case is what makes the single-element comparison also an assertion
that the
+ predicate was reached at all.
+ """
+ conn, judged = _judging_connection()
+ model_kwargs = _model_kwargs(backing_model)
+
+ named = conn.effective_model_for(model_kwargs)
+ conn.chat(
+ [ChatMessage(role=MessageRole.USER, content="hi")],
+ output_schema=OutputSchema(output_schema=Person),
+ **model_kwargs,
+ )
+
+ assert judged == [named]
+ assert judged[0] != DEPLOYMENT
diff --git
a/python/flink_agents/integrations/chat_models/openai/tests/test_openai_native_structured_output.py
b/python/flink_agents/integrations/chat_models/openai/tests/test_openai_native_structured_output.py
index d8b1d2c0..3b8d0f49 100644
---
a/python/flink_agents/integrations/chat_models/openai/tests/test_openai_native_structured_output.py
+++
b/python/flink_agents/integrations/chat_models/openai/tests/test_openai_native_structured_output.py
@@ -268,3 +268,59 @@ def test_map_member_schema_is_accepted_and_sent_whole() ->
None:
_chat_with_schema(conn, Labelled)
response_format = _create_call_kwargs(conn)["response_format"]
assert response_format["json_schema"]["schema"] ==
to_strict_json_schema(Labelled)
+
+
+def _judging_connection() -> tuple[OpenAIChatModelConnection, list[str |
None]]:
+ """A connection recording every model its request path judges for
capability.
+
+ Subclassing keeps the predicate itself under test rather than standing a
stub in
+ for it: the override notes what it was asked about and delegates to the
real one.
+ """
+ judged: list[str | None] = []
+
+ class _JudgingConnection(OpenAIChatModelConnection):
+ def supports_native_structured_output(
+ self, effective_model: str | None
+ ) -> bool:
+ judged.append(effective_model)
+ return super().supports_native_structured_output(effective_model)
+
+ conn = _JudgingConnection(api_key="test-key",
api_base_url="http://localhost";)
+ mock_client = MagicMock()
+ mock_message = MagicMock()
+ mock_message.role = "assistant"
+ mock_message.content = "ok"
+ mock_message.tool_calls = None
+ mock_message.refusal = None
+ mock_client.chat.completions.create.return_value.choices = [
+ MagicMock(message=mock_message)
+ ]
+ mock_client.chat.completions.create.return_value.usage = None
+ conn._client = mock_client
+ return conn, judged
+
+
[email protected](
+ "model_kwargs",
+ [{"model": "gpt-4o-mini"}, {"model": "an-unknown-model"}, {"model": ""},
{}],
+ ids=["capable", "unknown", "blank", "absent"],
+)
+def test_effective_model_for_names_the_model_the_request_judges(
+ model_kwargs: dict[str, Any],
+) -> None:
+ """The hook names exactly the model the request path asks the predicate
about.
+
+ This connection reads the parameter without a fallback, so the inherited
hook is
+ already the right answer. Pinning it against what the builder judges is
what would
+ catch a fallback being added here without a matching override.
+ """
+ conn, judged = _judging_connection()
+
+ named = conn.effective_model_for(model_kwargs)
+ conn.chat(
+ [ChatMessage(role=MessageRole.USER, content="hi")],
+ output_schema=OutputSchema(output_schema=Person),
+ **model_kwargs,
+ )
+
+ assert judged == [named]
diff --git
a/python/flink_agents/integrations/chat_models/tests/test_ollama_native_structured_output.py
b/python/flink_agents/integrations/chat_models/tests/test_ollama_native_structured_output.py
index 0d1d2934..1cf559d6 100644
---
a/python/flink_agents/integrations/chat_models/tests/test_ollama_native_structured_output.py
+++
b/python/flink_agents/integrations/chat_models/tests/test_ollama_native_structured_output.py
@@ -149,3 +149,67 @@ def test_schema_accepted_not_rejected() -> None:
_messages(), model="qwen3",
output_schema=OutputSchema(output_schema=row_type)
)
assert response.content == "ok"
+
+
+def _judging_connection() -> tuple[OllamaChatModelConnection, list]:
+ """A connection recording every model its request path judges for
capability.
+
+ Subclassing keeps the predicate itself under test rather than standing a
stub in
+ for it: the override notes what it was asked about and delegates to the
real one.
+ """
+ judged: list = []
+
+ class _JudgingConnection(OllamaChatModelConnection):
+ def supports_native_structured_output(
+ self, effective_model: str | None
+ ) -> bool:
+ judged.append(effective_model)
+ return super().supports_native_structured_output(effective_model)
+
+ conn = _JudgingConnection()
+ response = MagicMock()
+ response.message.role = "assistant"
+ response.message.content = "ok"
+ response.message.tool_calls = None
+ response.prompt_eval_count = 1
+ response.eval_count = 2
+ mock_client = MagicMock()
+ mock_client.chat.return_value = response
+ conn._OllamaChatModelConnection__client = mock_client
+ return conn, judged
+
+
[email protected](
+ "model_kwargs",
+ [{"model": "qwen3"}, {"model": ""}],
+ ids=["named", "blank"],
+)
+def test_effective_model_for_names_the_model_the_request_judges(
+ model_kwargs: Dict[str, Any],
+) -> None:
+ """The hook names exactly the model the request path asks the predicate
about.
+
+ Only parameter maps that name a model are exercised: this builder pops
``model``
+ with no fallback, so an absent one is a request that cannot be built
rather than a
+ disagreement about which model to judge. The hook still answers ``None``
there,
+ since it resolves rather than validates.
+ """
+ conn, judged = _judging_connection()
+
+ named = conn.effective_model_for(model_kwargs)
+ conn.chat(
+ _messages(),
+ output_schema=OutputSchema(output_schema=Person),
+ **model_kwargs,
+ )
+
+ assert judged == [named]
+
+
+def test_effective_model_for_resolves_nothing_without_a_model_param() -> None:
+ """A parameter map naming no model resolves to nothing rather than raising.
+
+ The request builder refuses that map, but the hook answers for whatever it
is
+ given, and the capability predicate accepts ``None`` without raising.
+ """
+ assert _connection().effective_model_for({}) is None
diff --git
a/python/flink_agents/integrations/chat_models/tests/test_tongyi_native_structured_output.py
b/python/flink_agents/integrations/chat_models/tests/test_tongyi_native_structured_output.py
index c249de51..2979330b 100644
---
a/python/flink_agents/integrations/chat_models/tests/test_tongyi_native_structured_output.py
+++
b/python/flink_agents/integrations/chat_models/tests/test_tongyi_native_structured_output.py
@@ -260,3 +260,92 @@ def
test_row_type_info_leaves_a_caller_response_format_alone(monkeypatch) -> Non
)
assert response.content == "ok"
assert kwargs["response_format"] == caller_format
+
+
+# The connection's default model, written out here rather than imported for
the same
+# reason as the lists above: a changed default is then a disagreement between
two
+# values rather than one value both sides read.
+_DEFAULT_MODEL = "qwen-plus"
+
+
+def _judging_connection() -> tuple[TongyiChatModelConnection, list[str |
None]]:
+ """A connection recording every model its request path judges for
capability.
+
+ Subclassing keeps the predicate itself under test rather than standing a
stub in
+ for it: the override notes what it was asked about and delegates to the
real one.
+ """
+ judged: list[str | None] = []
+
+ class _JudgingConnection(TongyiChatModelConnection):
+ def supports_native_structured_output(
+ self, effective_model: str | None
+ ) -> bool:
+ judged.append(effective_model)
+ return super().supports_native_structured_output(effective_model)
+
+ return _JudgingConnection(api_key="fake-key"), judged
+
+
+def test_effective_model_for_applies_the_default_model() -> None:
+ """A call naming no model resolves to the model the request would be
issued to.
+
+ Reading the parameter alone answers ``None`` for every such call and
reports the
+ default model incapable without ever asking about it, which is the failure
the
+ comment beside the request builder's own lookup warns about.
+ """
+ assert _connection().effective_model_for({}) == _DEFAULT_MODEL
+ assert _connection().effective_model_for(None) == _DEFAULT_MODEL
+
+
+def test_effective_model_for_reads_an_explicit_model() -> None:
+ """A named model is answered as given, not replaced by the default."""
+ assert (
+ _connection().effective_model_for({"model": _CAPABLE_MODEL}) ==
_CAPABLE_MODEL
+ )
+
+
+def test_effective_model_for_keeps_a_present_but_empty_model() -> None:
+ """The default stands in for an absent model only, matching the request
builder.
+
+ The builder's fallback is a ``pop`` default, which applies when the key is
missing
+ and not when it is present and empty. Substituting the default for an
empty value
+ would make the hook and the request disagree on exactly that input.
+ """
+ assert _connection().effective_model_for({"model": ""}) == ""
+
+
+def test_effective_model_for_does_not_consume_the_model() -> None:
+ """The hook reads the key that ``chat`` pops, and has to leave it in
place."""
+ model_kwargs = {"model": _CAPABLE_MODEL}
+
+ _connection().effective_model_for(model_kwargs)
+
+ assert model_kwargs == {"model": _CAPABLE_MODEL}
+
+
[email protected](
+ "model_kwargs",
+ [{"model": _CAPABLE_MODEL}, {"model": "qwen-turbo"}, {"model": ""}, {}],
+ ids=["capable", "incapable", "blank", "absent"],
+)
+def test_effective_model_for_names_the_model_the_request_judges(
+ monkeypatch, model_kwargs: dict[str, Any]
+) -> None:
+ """The hook names exactly the model the request path asks the predicate
about.
+
+ The hook duplicates the builder's resolution rather than centralizing it,
so only
+ capturing what the request feeds the predicate keeps the two from drifting
apart.
+ A fresh connection per case is what makes the single-element comparison
also an
+ assertion that the predicate was reached at all.
+ """
+ conn, judged = _judging_connection()
+ _patched_call(monkeypatch)
+
+ named = conn.effective_model_for(model_kwargs)
+ conn.chat(
+ _messages(),
+ output_schema=OutputSchema(output_schema=Person),
+ **model_kwargs,
+ )
+
+ assert judged == [named]
diff --git a/python/flink_agents/integrations/chat_models/tongyi_chat_model.py
b/python/flink_agents/integrations/chat_models/tongyi_chat_model.py
index 0c05af5f..27879873 100644
--- a/python/flink_agents/integrations/chat_models/tongyi_chat_model.py
+++ b/python/flink_agents/integrations/chat_models/tongyi_chat_model.py
@@ -19,7 +19,7 @@ import contextlib
import json
import os
import uuid
-from typing import Any, Dict, List, Sequence, cast
+from typing import Any, Dict, List, Mapping, Sequence, cast
from dashscope import Generation
from pydantic import BaseModel, Field
@@ -184,6 +184,22 @@ class TongyiChatModelConnection(BaseChatModelConnection):
"""
return effective_model in _NATIVE_STRUCTURED_OUTPUT_MODELS
+ @override
+ def effective_model_for(self, model_kwargs: Mapping[str, Any] | None) ->
str | None:
+ """The ``model`` parameter, falling back to ``DEFAULT_MODEL`` when
absent.
+
+ ``chat`` resolves the model it calls the same way, so reading the
parameter
+ alone would answer ``None`` for every call that names no model, and
report the
+ default model incapable without ever asking about it.
+
+ The fallback stands in for an absent parameter only, matching the
request: a
+ parameter that is present but empty is passed through, so the hook and
the
+ request agree on that input too.
+ """
+ if model_kwargs is None:
+ return DEFAULT_MODEL
+ return model_kwargs.get("model", DEFAULT_MODEL)
+
def chat(
self,
messages: Sequence[ChatMessage],
diff --git
a/python/flink_agents/integrations/chat_models/watsonx/tests/test_watsonx_native_structured_output.py
b/python/flink_agents/integrations/chat_models/watsonx/tests/test_watsonx_native_structured_output.py
index 92a8ca18..3e8430f1 100644
---
a/python/flink_agents/integrations/chat_models/watsonx/tests/test_watsonx_native_structured_output.py
+++
b/python/flink_agents/integrations/chat_models/watsonx/tests/test_watsonx_native_structured_output.py
@@ -356,3 +356,98 @@ def test_chat_with_output_schema() -> None:
parsed = json.loads(response.content)
assert set(parsed) == {"verdict", "score"}
assert Answer(**parsed).verdict is not None
+
+
+def _judging_connection(
+ monkeypatch: pytest.MonkeyPatch,
+) -> Tuple[WatsonxChatModelConnection, list]:
+ """A mocked connection recording every model its request path judges.
+
+ Subclassing keeps the predicate itself under test rather than standing a
stub in
+ for it: the override notes what it was asked about and delegates to the
real one.
+ """
+ judged: list = []
+
+ class _JudgingConnection(WatsonxChatModelConnection):
+ def supports_native_structured_output(
+ self, effective_model: str | None
+ ) -> bool:
+ judged.append(effective_model)
+ return super().supports_native_structured_output(effective_model)
+
+ provider_model = MagicMock()
+ provider_model.chat.return_value = CHAT_RESPONSE
+ monkeypatch.setattr(
+
"flink_agents.integrations.chat_models.watsonx.watsonx_chat_model.ModelInference",
+ MagicMock(return_value=provider_model),
+ )
+ connection = _JudgingConnection(
+ url="https://us-south.ml.cloud.ibm.com";,
+ api_key="fake-key",
+ project_id="fake-project",
+ )
+ connection._client = MagicMock()
+ return connection, judged
+
+
+def test_effective_model_for_applies_the_default_model() -> None:
+ """A call naming no model resolves to the model the request would be
issued to.
+
+ Reading the parameter alone would answer ``None`` where the request in
fact goes
+ to the default model, so the hook and the request would disagree on every
call
+ that names no model.
+ """
+ assert _connection().effective_model_for({}) == DEFAULT_MODEL
+ assert _connection().effective_model_for(None) == DEFAULT_MODEL
+
+
+def test_effective_model_for_reads_an_explicit_model() -> None:
+ """A named model is answered as given, not replaced by the default."""
+ assert _connection().effective_model_for({"model": "no-such-model"}) == (
+ "no-such-model"
+ )
+
+
+def test_effective_model_for_keeps_a_present_but_empty_model() -> None:
+ """The default stands in for an absent model only, matching the request
builder.
+
+ The builder's fallback is a ``pop`` default, which applies when the key is
missing
+ and not when it is present and empty.
+ """
+ assert _connection().effective_model_for({"model": ""}) == ""
+
+
+def test_effective_model_for_does_not_consume_the_model() -> None:
+ """The hook reads the key that ``chat`` pops, and has to leave it in
place."""
+ model_kwargs = {"model": "no-such-model"}
+
+ _connection().effective_model_for(model_kwargs)
+
+ assert model_kwargs == {"model": "no-such-model"}
+
+
[email protected](
+ "model_kwargs",
+ [{"model": "no-such-model"}, {"model": ""}, {}],
+ ids=["named", "blank", "absent"],
+)
+def test_effective_model_for_names_the_model_the_request_judges(
+ monkeypatch: pytest.MonkeyPatch, model_kwargs: Dict[str, Any]
+) -> None:
+ """The hook names exactly the model the request path asks the predicate
about.
+
+ Capability here is unconditional, so the predicate's answer cannot reveal a
+ disagreement; only the argument it was handed can. A fresh connection per
case is
+ what makes the single-element comparison also an assertion that the
predicate was
+ reached at all.
+ """
+ connection, judged = _judging_connection(monkeypatch)
+
+ named = connection.effective_model_for(model_kwargs)
+ connection.chat(
+ [ChatMessage(role=MessageRole.USER, content="Hello!")],
+ output_schema=OutputSchema(output_schema=Answer),
+ **model_kwargs,
+ )
+
+ assert judged == [named]
diff --git
a/python/flink_agents/integrations/chat_models/watsonx/watsonx_chat_model.py
b/python/flink_agents/integrations/chat_models/watsonx/watsonx_chat_model.py
index 414ff6f9..51c7742d 100644
--- a/python/flink_agents/integrations/chat_models/watsonx/watsonx_chat_model.py
+++ b/python/flink_agents/integrations/chat_models/watsonx/watsonx_chat_model.py
@@ -22,7 +22,7 @@ import logging
import os
import time
import uuid
-from typing import Any, Dict, List, Sequence
+from typing import Any, Dict, List, Mapping, Sequence
import httpx
from ibm_watsonx_ai import APIClient, Credentials
@@ -404,6 +404,22 @@ class WatsonxChatModelConnection(BaseChatModelConnection):
"""
return True
+ @override
+ def effective_model_for(self, model_kwargs: Mapping[str, Any] | None) ->
str | None:
+ """The ``model`` parameter, falling back to ``DEFAULT_MODEL`` when
absent.
+
+ ``chat`` resolves the model it calls the same way, so reading the
parameter
+ alone would answer ``None`` where the request in fact goes to the
default
+ model.
+
+ The fallback stands in for an absent parameter only, matching the
request: a
+ parameter that is present but empty is passed through, so the hook and
the
+ request agree on that input too.
+ """
+ if model_kwargs is None:
+ return DEFAULT_MODEL
+ return model_kwargs.get("model", DEFAULT_MODEL)
+
def chat(
self,
messages: Sequence[ChatMessage],
