pankajkoti opened a new pull request, #74473: URL: https://github.com/apache/airflow/pull/74473
When a reviewer edits the output of an `LLMOperator` approval gate, the edited text is validated back into the operator's `output_type`. If validation failed, `rehydrate_pydantic_output` silently returned the raw string. The task succeeded and downstream tasks received a `str` instead of an `int`, a `list` or a model. For example `output_type=int` with the edit "thirty-one" produced the string 'thirty-one'. It now raises `ReviewedOutputValidationError`, a `ValueError` subclass in the provider's exceptions module. The message names the output type, the text received and the pydantic validation message, so the task fails with a readable error. Details: - `output_type=str` still accepts any text. - Types whose value is plain text (a `str` `Literal`, a `str` Enum, `str | None`) still accept bare text, so an unedited approval keeps working. - An `output_type` that pydantic has no schema for still passes the text through, because there is nothing to validate against. `AgentOperator` review serialized non-model outputs with `str()`, which produces a Python repr for a list and would now fail an unedited approval. It serializes to JSON instead, for the first output and for regenerated ones. Values pydantic cannot serialize keep `str()`. `LLMSchemaCompareOperator` re-validated the string after the parent returned and raised its own `ValueError`. That fallback is no longer reachable, so the override is removed. The new exception is still a `ValueError`. Behavior change: an invalid reviewer edit now fails the task instead of passing a string downstream. Related open PRs: #70132 also edits `rehydrate_pydantic_output` and `mixins/hitl_review.py` (output type handling for `AgentOperator` review), and #74450 also edits `mixins/hitl_review.py` and `operators/llm.py`. The changes are compatible in intent but overlap in those files, so whichever merges later will need a rebase. This PR does not handle pydantic-ai output markers or output functions, which #70132 covers. Follow-up idea, not part of this PR: validate the edit when the reviewer submits the form, so they see the error immediately instead of the task failing after approval. --- ##### Was generative AI tooling used to co-author this PR? - [X] Yes (Claude Code, Sonnet 5.5) Generated-by: Claude Code (Sonnet 5.5) following [the guidelines](https://github.com/apache/airflow/blob/main/contributing-docs/05_pull_requests.rst#gen-ai-assisted-contributions) --- * Read the **[Pull Request Guidelines](https://github.com/apache/airflow/blob/main/contributing-docs/05_pull_requests.rst#pull-request-guidelines)** for more information. Note: commit author/co-author name and email in commits become permanently public when merged. * For fundamental code changes, an Airflow Improvement Proposal ([AIP](https://cwiki.apache.org/confluence/display/AIRFLOW/Airflow+Improvement+Proposals)) is needed. * When adding dependency, check compliance with the [ASF 3rd Party License Policy](https://www.apache.org/legal/resolved.html#category-x). * For significant user-facing changes create newsfragment: `{pr_number}.significant.rst`, in [airflow-core/newsfragments](https://github.com/apache/airflow/tree/main/airflow-core/newsfragments). You can add this file in a follow-up commit after the PR is created so you know the PR number. 🤖 Generated with [Claude Code](https://claude.com/claude-code) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
