potiuk commented on code in PR #32526:
URL: https://github.com/apache/airflow/pull/32526#discussion_r1260600525
##########
airflow/executors/executor_loader.py:
##########
@@ -60,8 +60,9 @@ class ExecutorLoader:
LOCAL_EXECUTOR: "airflow.executors.local_executor.LocalExecutor",
LOCAL_KUBERNETES_EXECUTOR:
"airflow.executors.local_kubernetes_executor.LocalKubernetesExecutor",
SEQUENTIAL_EXECUTOR:
"airflow.executors.sequential_executor.SequentialExecutor",
- CELERY_EXECUTOR: "airflow.executors.celery_executor.CeleryExecutor",
- CELERY_KUBERNETES_EXECUTOR:
"airflow.executors.celery_kubernetes_executor.CeleryKubernetesExecutor",
+ CELERY_EXECUTOR:
"airflow.providers.celery.executors.celery_executor.CeleryExecutor",
+ CELERY_KUBERNETES_EXECUTOR: "airflow.providers.celery."
+ "executors.celery_kubernetes_executor.CeleryKubernetesExecutor",
Review Comment:
I think Celery and Kubernetest yes - mostly because of tradition and being
"vendor agnostic". While we are mviing them to providers, they will become
still availabile in the core and pre-installed (we are adding both celery and
kubernetes packages as pre-installed). In the way those executors are special.
Always available, described in "apache-airflow" part of our documentaiton, not
in "apache-airflow-providers-". I think it will stay like that "forever".
So in the sense, no - we are not going to add Amazon Executors in here. Even
more - this has not been discussed yet, but I am going to propose to move
"DASK_EXECUTOR" out of the core and out of this list as well - strictly to a
new dask provider we are going to have. Precisely for the same reason.
##########
airflow/executors/executor_loader.py:
##########
@@ -60,8 +60,9 @@ class ExecutorLoader:
LOCAL_EXECUTOR: "airflow.executors.local_executor.LocalExecutor",
LOCAL_KUBERNETES_EXECUTOR:
"airflow.executors.local_kubernetes_executor.LocalKubernetesExecutor",
SEQUENTIAL_EXECUTOR:
"airflow.executors.sequential_executor.SequentialExecutor",
- CELERY_EXECUTOR: "airflow.executors.celery_executor.CeleryExecutor",
- CELERY_KUBERNETES_EXECUTOR:
"airflow.executors.celery_kubernetes_executor.CeleryKubernetesExecutor",
+ CELERY_EXECUTOR:
"airflow.providers.celery.executors.celery_executor.CeleryExecutor",
+ CELERY_KUBERNETES_EXECUTOR: "airflow.providers.celery."
+ "executors.celery_kubernetes_executor.CeleryKubernetesExecutor",
Review Comment:
I think Celery and Kubernetest yes - mostly because of tradition and being
"vendor agnostic" should stay here.
While we are mviing them to providers, they will become still availabile in
the core and pre-installed (we are adding both celery and kubernetes packages
as pre-installed). In the way those executors are special. Always available,
described in "apache-airflow" part of our documentaiton, not in
"apache-airflow-providers-". I think it will stay like that "forever".
So in the sense, no - we are not going to add Amazon Executors in here. Even
more - this has not been discussed yet, but I am going to propose to move
"DASK_EXECUTOR" out of the core and out of this list as well - strictly to a
new dask provider we are going to have. Precisely for the same reason.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
