kaxil commented on a change in pull request #16700:
URL: https://github.com/apache/airflow/pull/16700#discussion_r660477307
##########
File path: airflow/jobs/base_job.py
##########
@@ -94,7 +94,8 @@ class BaseJob(Base, LoggingMixin):
def __init__(self, executor=None, heartrate=None, *args, **kwargs):
self.hostname = get_hostname()
- self.executor = executor or ExecutorLoader.get_default_executor()
+ if self.__class__.__name__ != "LocalTaskJob":
+ self.executor = executor or ExecutorLoader.get_default_executor()
Review comment:
>i do also wonder.... why is it that we only get this issue with CKE? is
there perhaps something about the way in which CKE is designed that causes this
problem?
The issue is that it tries to create an instance of `KubernetesExecutor`
inside `CeleryExecutor` and `KubernetesExecutor` creates a multiprocessing
Manager & Queue in its `__init__` which creates issues with Celery as
explained in https://github.com/celery/celery/issues/4525
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
