wenjin272 commented on code in PR #1040:
URL: https://github.com/apache/flink-agents/pull/1040#discussion_r3859345748
##########
plan/src/main/java/org/apache/flink/agents/plan/actions/ChatModelAction.java:
##########
@@ -576,8 +579,40 @@ private static Map<String, Object> takeRoutingMetadata(
return routing;
}
+ /**
+ * Rejects a response the provider did not finish emitting. A finish
reason reporting the
+ * content as cut off by the token budget or withheld by content filtering
raises {@link
+ * IllegalStateException}; any other reason, and an absent one, are
accepted.
+ */
+ private static void rejectIncompleteResponse(ChatMessage response) {
+ Object finishReason = response.getExtraArgs().get(FINISH_REASON);
+ if (TRUNCATED_FINISH_REASON.equals(finishReason)) {
+ throw new IllegalStateException(
+ String.format(
+ "ChatModel response is truncated
(finish_reason='%s'): it"
+ + " exhausted the completion token budget
before the model"
+ + " finished, so the content is incomplete
and cannot yield"
+ + " structured output. Raise the model's
max output tokens,"
+ + " or ask for a smaller output.",
+ finishReason));
+ }
+ if (CONTENT_FILTERED_FINISH_REASON.equals(finishReason)) {
+ throw new IllegalStateException(
+ String.format(
+ "ChatModel response was withheld by the provider's
content"
+ + " filter (finish_reason='%s'), so the
content is incomplete"
+ + " and cannot yield structured output.
Adjust the prompt or"
+ + " the provider's content filtering
configuration.",
+ finishReason));
+ }
+ }
+
static ChatMessage generateStructuredOutputWithReport(
RunnerContext ctx, ChatMessage response, Object outputSchema)
throws Exception {
+ // Precedes the start report: when this rejects, parsing is never
attempted, so there is no
+ // parser execution to report.
+ rejectIncompleteResponse(response);
Review Comment:
Could we move this check to the common chat-response path rather than
limiting it to structured output? Since Flink Agents currently only handles
complete, non-streaming responses, we can validate every response before
processing it as text, structured output, or tool calls. This would also
prevent truncated responses such as finish_reason=length from being treated as
successful.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
