caiocv18 opened a new pull request, #76753:
URL: https://github.com/apache/airflow/pull/76753
A `HITLOperator` form field without a default, or with a default that does
not pass its own schema, fails the task as soon as it starts, before the
request reaches the reviewer:
```
airflow.sdk.exceptions.ParamValidationError: Invalid input for param flag:
No value passed and Param has no default value
```
Reproduction (fails on Airflow 3.4.0 / task-sdk 1.4.0 and on `main`):
```python
from airflow.providers.standard.operators.hitl import HITLEntryOperator
from airflow.sdk import DAG, Param
from airflow.sdk.definitions.param import process_params
with DAG(dag_id="hitl_form", schedule=None) as dag:
ask = HITLEntryOperator(task_id="ask", subject="Fill in the form",
params={"flag": Param(type="boolean")})
# What the Task SDK runs to build the task context, before execute():
process_params(dag, ask, None, suppress_exception=False) # raises
ParamValidationError
```
`Param("maybe", type="boolean")` fails the same way (`'maybe' is not of type
'boolean'`).
related: #59551
### Why
The form fields are stored in the task's `params`, and the Task SDK
validates every task param when it builds the task context
(`RuntimeTaskInstance.get_template_context` -> `process_params`). That happens
in `startup()`, before `execute()`, and again on every resume. #64108 took the
validation out of `__init__` and documented it as deferred to
`validate_params_input` after the human submits the form, but the execution
path still validates the fields first.
### What changes
- `HITLOperator` keeps its form fields in `self.hitl_params` and leaves
`self.params` empty, so `process_params` has none of them to validate at start.
Dag-level params still reach `context["params"]`.
- `serialized_params` sends `None` instead of `NOTSET` for a field without a
default. Without this, moving the fields only moves the failure into
`execute()`: the request to the API server cannot encode it
(`PydanticSerializationError: Unable to serialize unknown type: <class
'airflow.serialization.definitions.notset.ArgNotSet'>`, seen in the new test
before this change). `request_input()` already does the same for the kwargs it
stores with the paused task.
- `validate_params`, `validate_params_input` and `generate_link_to_ui` read
`hitl_params`.
- The example notifier prints `{{ task.hitl_params }}`, and the provider
changelog has a note.
User-visible: the form fields are no longer part of `{{ params }}`,
`context["params"]` or `{{ task.params }}`. Before the reviewer answers they
could only hold the defaults; the answer is still in `params_input` of the
task's return value.
### Alternative considered
Keep the fields in `task.params` and let the operator opt out of the
start-time validation in the Task SDK (for example, a class attribute that
`process_params` reads). That would also cover a mapped
`HITLOperator.partial(params=...).expand(...)`: from reading the code, it still
fails at start with this PR, because the fields end up in
`MappedOperator.params` (not tested). I went with the provider-only change
because it works with the Task SDK versions already released and does not touch
`process_params`, which #75918 and #72499 are changing. Happy to switch if you
prefer the Task SDK route.
### Same issue in `request_input()`
`request_input()` sends the raw serialized params to `upsert_hitl_detail`,
so `request_input(..., params={"reason": Param(type="string")})` fails the task
with the same `PydanticSerializationError` (checked with `dag_maker.run_ti`;
the existing test mocks the supervisor comms, so it never reaches the encoder).
I left it out to keep this PR on the operator and can open a follow-up.
### Tests
- `test_form_fields_are_not_validated_when_the_task_starts`: builds the real
task context; fails on `main` with the `ParamValidationError` above.
- `test_execute_with_form_fields_without_a_valid_default`: runs the task
with `dag_maker.run_ti` until it parks and checks the stored form value
(`None`, or the invalid default as is).
- `test_serialzed_params`: new case for a field without a default.
- `test_param_value_validation_deferred_to_runtime` (from #64108): now
checks `hitl_params`.
The new tests are parametrized with both cases (no default, default outside
its own schema). Locally (WSL2, Python 3.11, `uv run --project
providers/standard`): `operators/test_hitl.py`,
`operators/test_hitl_request_input.py`, `triggers/test_hitl.py` and
`utils/test_hitl.py` give 135 passed, 1 skipped. `prek run --stage pre-commit`
on the changed files passed with `SKIP_BREEZE_PREK_HOOKS=true`. I did not run
the Breeze-based `mypy-providers` hook; `mypy` from the venv reported no issues
in `hitl.py` and `test_hitl.py`.
---
##### Was generative AI tooling used to co-author this PR?
- [X] Yes (please specify the tool below)
Generated-by: Claude Code (Claude Opus 5.5) following [the
guidelines](https://github.com/apache/airflow/blob/main/contributing-docs/05_pull_requests.rst#gen-ai-assisted-contributions)
The code and tests were written with Claude Code at my direction. The design
and the behaviour were my decisions, and I reviewed the diff and ran the tests
locally.
---
* Read the **[Pull Request
Guidelines](https://github.com/apache/airflow/blob/main/contributing-docs/05_pull_requests.rst#pull-request-guidelines)**
for more information. Note: commit author/co-author name and email in commits
become permanently public when merged.
* For fundamental code changes, an Airflow Improvement Proposal
([AIP](https://cwiki.apache.org/confluence/display/AIRFLOW/Airflow+Improvement+Proposals))
is needed.
* When adding dependency, check compliance with the [ASF 3rd Party License
Policy](https://www.apache.org/legal/resolved.html#category-x).
* For significant user-facing changes create newsfragment:
`{pr_number}.significant.rst`, in
[airflow-core/newsfragments](https://github.com/apache/airflow/tree/main/airflow-core/newsfragments).
You can add this file in a follow-up commit after the PR is created so you
know the PR number.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
