dabla commented on code in PR #75423:
URL: https://github.com/apache/airflow/pull/75423#discussion_r4640186369
##########
airflow-core/src/airflow/jobs/triggerer_job_runner.py:
##########
@@ -1467,6 +1475,34 @@ def create_runtime_ti(
),
)
+ async def record_rendered_fields(self, trigger: BaseTrigger) -> None:
+ """
+ Record the template fields a task that starts from the triggerer
renders here.
+
+ Such a task never starts in a worker, so nothing else records its
rendered fields and the
+ Rendered Templates tab stays empty for the whole run. Only the
operator fields that
+ ``start_trigger_args.trigger_kwargs`` carries reach the triggerer, and
those are exactly
+ the ones ``BaseTrigger`` renders, so those are what this records.
Rendering the rest would
+ need the Dag file, which the triggerer deliberately does not load.
+
+ Call this after ``trigger.render_template_fields()``, so the values
are the rendered ones.
+ """
+ # Reused rather than reimplemented so core cannot drift from the
redaction the worker applies.
+ from airflow.sdk.execution_time.task_runner import
_serialize_rendered_fields # noqa: SDK001
+
+ if not trigger.template_fields:
+ return
+
+ try:
+ rendered_fields = _serialize_rendered_fields(trigger) # type:
ignore[arg-type]
+ except Exception:
+ # A trigger that cannot be serialized still has work to do, so let
it run.
+ self.log.exception("Failed to record rendered template fields",
task_id=trigger.task_id)
+ return
+
+ if rendered_fields:
+ await
self.comms_decoder.asend(SetRenderedFields(rendered_fields=rendered_fields))
Review Comment:
**[warning]** A failed RTIF write fails the trigger, and with it the task.
The `try` above only covers serialization, but this `asend` is the step that
can actually fail. The supervisor's `set_rtif` can return a 410 when the TI was
cleared in the meantime, or a transient DB/API error. `handle_requests` turns
that into an `ErrorResponse`, `asend` raises it, the exception leaves
`record_rendered_fields` inside `run_trigger`'s `try`, and the trigger ends
with it. That fails the task over a value that only feeds the UI. I checked
this by having the supervisor return an error for `SetRenderedFields` in the
end-to-end test below: the trigger task finished with that exception instead of
firing its event.
The comment on the `except` already gives the rule ("still has work to do,
so let it run"), so the send should fall under it too. For example:
```python
try:
rendered_fields = _serialize_rendered_fields(trigger)
if rendered_fields:
await
self.comms_decoder.asend(SetRenderedFields(rendered_fields=rendered_fields))
except Exception:
self.log.exception("Failed to record rendered template fields",
task_id=trigger.task_id)
```
##########
airflow-core/src/airflow/jobs/triggerer_job_runner.py:
##########
@@ -1467,6 +1475,34 @@ def create_runtime_ti(
),
)
+ async def record_rendered_fields(self, trigger: BaseTrigger) -> None:
+ """
+ Record the template fields a task that starts from the triggerer
renders here.
+
+ Such a task never starts in a worker, so nothing else records its
rendered fields and the
+ Rendered Templates tab stays empty for the whole run. Only the
operator fields that
+ ``start_trigger_args.trigger_kwargs`` carries reach the triggerer, and
those are exactly
+ the ones ``BaseTrigger`` renders, so those are what this records.
Rendering the rest would
+ need the Dag file, which the triggerer deliberately does not load.
+
+ Call this after ``trigger.render_template_fields()``, so the values
are the rendered ones.
+ """
+ # Reused rather than reimplemented so core cannot drift from the
redaction the worker applies.
+ from airflow.sdk.execution_time.task_runner import
_serialize_rendered_fields # noqa: SDK001
Review Comment:
**[nit]** This import can move to the top of the module.
The module already imports `RuntimeTaskInstance` from
`airflow.sdk.execution_time.task_runner` at the top, so a local import buys
nothing here (imports at the top of the file, [PEP
8](https://peps.python.org/pep-0008/#imports)). It keeps the `# noqa: SDK001`
there, since `check_sdk_imports_in_core.py` counts every `airflow.sdk` import
line. The `# type: ignore[arg-type]` on the call is the other cost of reusing a
worker-side private helper typed for operators. Fine for this fix, but worth a
sentence in the PR body.
##########
airflow-core/tests/unit/jobs/test_triggerer_job.py:
##########
@@ -4007,3 +4008,86 @@ async def fake_run():
log.get("event") == "Trigger fired event payload" and
log.get("result") == TriggerEvent(payload)
for log in debug_logs
), "Full event payload must be logged at DEBUG level"
+
+
[email protected]
[email protected]("testing_dag_bundle")
+async def test_start_from_trigger_task_records_its_rendered_fields(session,
supervisor_builder, mocker):
Review Comment:
**[nit]** `mocker` is unused in this test.
##########
airflow-core/tests/unit/jobs/test_triggerer_job.py:
##########
@@ -4007,3 +4008,86 @@ async def fake_run():
log.get("event") == "Trigger fired event payload" and
log.get("result") == TriggerEvent(payload)
for log in debug_logs
), "Full event payload must be logged at DEBUG level"
+
+
[email protected]
[email protected]("testing_dag_bundle")
+async def test_start_from_trigger_task_records_its_rendered_fields(session,
supervisor_builder, mocker):
Review Comment:
**[warning]** This test never reaches `set_rtif`: the supervisor drops the
write without a word.
`InProcessTriggerComms.asend` builds `_RequestFrame(id=..., body=...)`
without `ti_id`, unlike the real `TriggerCommsDecoder.asend`, which sets it
from `_current_trigger_ti`. So `_handling_trigger_ti_id` is `None` while the
supervisor handles `SetRenderedFields`, the new branch skips `set_rtif` and
still answers `OKResponse`. With `set_rtif` patched, this test records no call.
It passes because it only checks that the runner *sent* the message. The
supervisor test sets the contextvar by hand, so no test checks that the TI's
row is written end to end.
Could this test assert `set_rtif` was called with the trigger's TI id and
`{"target_time": ...}`? That probably needs the harness to put `ti_id` on its
frames the way the real decoder does. A test where `set_rtif` raises would
cover [1]. Related: when the supervisor gets `SetRenderedFields` with no TI it
answers OK and drops the fields. A `log.warning` there would make the next
silent drop visible.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]