weiqingy commented on issue #912: URL: https://github.com/apache/flink-agents/issues/912#issuecomment-5691033218
Picking this up now. Every provider PR under #280 has merged, including the shared schema generator, so this is the last piece. Today nothing passes an output schema to a connection in either language, so an agent `output_schema` still takes the prompt path. Design summary, and I would rather adjust it before writing code than after. **Where the call goes.** Finalization happens in the chat invoker, at the point the model stops requesting tools. That site already owns retries, candidate fallback, reporters and token metrics. The action branch owns none of them, and by the time it runs the response has already been parsed. **What decides native versus prompt.** Policy, then capability, then feasibility. A capability predicate is only an upper bound, since the api version, bound tools, a caller supplied response format and the schema form each gate the native branch as well. So each connection answers a request shaped question, from the same code path that builds the request, rather than the framework guessing. **The schema form is checked first.** A `RowTypeInfo` schema is not natively translatable on any connection, so it keeps the prompt instruction and issues no extra call. That is what the end to end tests exercise today, and their behavior does not change. **The extra call carries no tools,** because Gemini drops the schema when tools are bound, and it does not re-prepend the bound prompt. **The instruction is suppressed only when the native call is known to apply,** so a request is never left with no schema channel at all. That matters for the connections whose capability predicate is unconditional. **A forced `NATIVE`** on a model the connection treats as incapable passes the schema through and lets the provider answer, which honors the intent. A forced `NATIVE` that the request cannot express fails fast and names the reason. **Sequencing.** Predicate contract and effective model hook first, then the feasibility query, then the setup level entry point, then the finalization call, then the instruction gate. Java and Python move together in each step. Three things I would like your view on. 1. Should the extra round trip happen only when it resolves to native? That keeps every current user at today's cost, but the loop's final turn is still instruction shaped on the prompt path. 2. Should a parse failure after a native call still consume the retry budget? The parse sits inside the retry loop today, so it re-runs the model call. 3. The finalization call needs a short directive message in both languages, which is user visible, so the wording is worth agreeing on early. One correction to what I wrote on #1120: I am doing this wiring before the structured output documentation. The docs would otherwise describe a path that no agent caller can reach. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
