weiqingy commented on PR #919: URL: https://github.com/apache/flink-agents/pull/919#issuecomment-5086222123
One thing worth calling out explicitly, since it is deliberately left out of this PR. `StructuredOutputStrategy.NATIVE` resolves to native regardless of the connection's reported capability, so a caller with an explicit `NATIVE` intent expects the schema to be applied even on a model the connection does not recognize as capable. This PR's OpenAI `chat` re-checks capability itself and skips `response_format` when the effective model is not capable or the schema is a `RowTypeInfo`, which under that intent would return an unconstrained response rather than an error. That is the same shape as the silent drop fixed in #843. I left it unfixed here on purpose. The connection never receives the strategy, so it cannot tell an explicit `NATIVE` apart from an `AUTO` that happens to resolve to native, and any decision made inside the connection is a guess at caller intent. The composition site is where the setup's policy meets the connection's capability, which is #912. Fixing it there also means dropping or bypassing this connection-side re-check, otherwise the setup honoring `NATIVE` would still be downgraded here. I have recorded that on my side so it lands with the wiring. Nothing dispatches on the strategy today, so this is dormant on `main`: no path passes a schema on the chat call yet, and the native path here is exercised only by direct callers and the unit tests. Happy to pull the fix into this PR instead if you would rather it not wait for #912. It would change the two fallback tests, which currently assert the silent degrade. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
