muditagrawal2007 opened a new pull request, #74920:
URL: https://github.com/apache/airflow/pull/74920
<!--
Please provide above a brief description of the changes made in this pull
request.
-->
Operators can now declare additional fields that are both Jinja-templated
and `XComArg`-capable through a new class-level `template_fields_extra`
attribute, on top of the existing `template_fields`. This is the first step
toward addressing #27285, which asked for `.output` references to work on
operator fields that are not part of `template_fields`.
### Why
Today, assigning an `XComArg` (e.g. `some_task.output`) to an operator field
only wires up the upstream relationship and renders the field at runtime if
that field is listed in `template_fields`. Operators that intentionally keep a
string field out of `template_fields` (for example, to keep the field out of
the rendered-template UI table or to avoid Jinja-rendering a value that may
legitimately contain `{{ ... }}` patterns) cannot receive `XComArg` references
without modifying the upstream operator or the call site. The previous PR
#29034 (closed) proposed instance-level opt-in via constructor arguments;
maintainers (`@uranusjr`, `@potiuk`) preferred a class-level configuration.
`template_fields_extra` is that class-level hook.
### What changes
* `BaseOperator.template_fields_extra` — a `ClassVar[frozenset[str]] =
frozenset()` on the operator base class. Subclasses populate it with the
*additional* field names that should participate in template rendering and XCom
argument handling.
* `BaseOperator.__init__` merges `template_fields_extra` into the instance
`template_fields` (preserving order, deduplicating). Because
`render_template_fields`, `_set_xcomargs_dependencies`, and `__setattr__` all
read `self.template_fields`, the merged value flows through automatically with
no other code changes downstream.
* `MappedOperator` and `SerializedBaseOperator` thread
`template_fields_extra` through so mapped and serialized tasks round-trip the
override.
* `get_serialized_fields()` includes `template_fields_extra` so it survives
DAG parsing and the metadata DB.
### Usage
```python
from airflow.sdk import BaseOperator
class MyOperator(BaseOperator):
template_fields = ("name",)
template_fields_extra = frozenset({"world"})
def __init__(self, name, world, **kwargs):
super().__init__(**kwargs)
self.name = name
self.world = world
```
With this in place, `world` is rendered with Jinja and accepts
`some_task.output` exactly the same way `name` does — including creating an
upstream relationship when assigned `XComArg`.
### Tests
Added four tests in `task-sdk/tests/task_sdk/bases/test_operator.py`:
* `test_template_fields_extra_class_level_merges_into_template_fields` —
verifies the merge order and dedup
* `test_template_fields_extra_renders_extra_string_fields` — verifies Jinja
rendering picks up extras
* `test_template_fields_extra_creates_xcomarg_upstream_dependency` —
verifies `XComArg` in an extra field wires up the upstream relationship at
construction time
* `test_template_fields_extra_empty_frozenset_is_default` — verifies the
default does not change existing operators' behavior
closes: #27285
---
##### Was generative AI tooling used to co-author this PR?
- [X] Yes
<!-- Generated-by: opencode (minimax/MiniMax-M3) following
https://github.com/apache/airflow/blob/main/contributing-docs/05_pull_requests.rst
-->
* Read the **[Pull Request
Guidelines](https://github.com/apache/airflow/blob/main/contributing-docs/05_pull_requests.rst#pull-request-guidelines)**
for more information.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
