The GitHub Actions job "Tests" on airflow.git has failed.
Run started by GitHub user ashb (triggered by ashb).

Head commit for run:
c660a892e29e9f240e9e1510b5167b3150d37408 / Ash Berlin-Taylor <[email protected]>
Convert the LocalExecutor to run tasks using new Task SDK supervisor code

This also lays the groundwork for a more general purpose "workload" execution
system, make a single interface for executors to run tasks and callbacks.

Also in this PR we set up the supervise function to send Task logs to a file,
and handle the task log template rendering in the scheduler before queueing
the workload.

Additionally we don't pass the activity directly to `supervise()` but instead
the properties/fields of it to reduce the coupling between SDK and Executor.
(More separation will appear in PRs over the next few weeks.)

The big change of note here is that rather than sending an airflow command
line to execute (`["airflow", "tasks", "run", ...]`) and going back in via the
CLI parser we go directly to a special purpose function. Much simpler.

It doesn't remove any of the old behaviour (CeleryExecutor still uses
LocalTaskJob via the CLI parser etc.), nor does anything currently send
callback requests via this new workload mechanism.

The `airflow.executors.workloads` module currently needs to be shared between
the Scheduler (or more specifically the Executor) and the "worker" side of
things. In the future these will be separate python dists and this module will
need to live somewhere else.

Right now we check the if `executor.queue_workload` is different from the
BaseExecutor version (which just raises an error right now) to see which
executors support this new version. That check will be removed as soon as all
the in-tree executors have been migrated.

Report URL: https://github.com/apache/airflow/actions/runs/12058808525

With regards,
GitHub Actions via GitBox


---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]

Reply via email to