The GitHub Actions job "Tests" on airflow.git/main has succeeded.
Run started by GitHub user potiuk (triggered by potiuk).

Head commit for run:
58f785a34e1bbb978fe0bc97508b4a087f5235ee / Niko Oliveira <[email protected]>
AIP-67 - Multi Team: Update Celery Executor to support multi team (#60675)

* Update Celery Executor to support multi team

Updating the Celery executor to read from team based config and also
support multiple instances running concurrently.

The latter is the largest source of changes. Much of the celery
configuration (both Airflow config and Celery config) was module based.
Modules are cached and shared in Python. So the majority of the changes
are moving that module level config code to be function based (while
trying to also maintain backwards compatibility).

The way Celery tasks are sent to workers also changed as a consequence
of this. Since sending tasks is parallelized with multiple processes
(which do not share memory with the parent) the send task logic now
re-creates a celery app in the sub processes (since the pickling and
unpickling that python does to try pass state to the sub processes was
not reliably creating the correct celery app objects).

* Fixes from PR CI

* Mypy sometimes makes code worse

* Fallback to global conf if we're not running on 3.2+ airflow

As inferred by the presence of the correct ExecutorConf methods being
available.

* More backcompat

* Skip multi team tests if not 3.2

* 3.2 not 3.1

* Add type annotation for create_celery_app

* Conditional or TYPE_CHECKING imports of ExercutorConfig

* More type fixups

* Use explicit version compat checks rather than trying to infer

* Test back compat on celery command

* fixup

* Apply suggestions from code review

Co-authored-by: Dheeraj Turaga <[email protected]>

* New exception for unit test

---------

Co-authored-by: Dheeraj Turaga <[email protected]>

Report URL: https://github.com/apache/airflow/actions/runs/21528296673

With regards,
GitHub Actions via GitBox


---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]

Reply via email to