Kalyanaraman Santhanam created AIRFLOW-5240:
-----------------------------------------------
Summary: latest version of Kombu is breaking airflow
Key: AIRFLOW-5240
URL: https://issues.apache.org/jira/browse/AIRFLOW-5240
Project: Apache Airflow
Issue Type: Bug
Components: celery, cli
Affects Versions: 1.10.4
Reporter: Kalyanaraman Santhanam
I believe kombu 4.6.4 released on Aug 14th (2 days ago) is breaking
CeleryExecutor
[https://github.com/celery/kombu/releases/tag/4.6.4]
I was able to resolve it by pinning kombu at 4.6.3
Refer to the stack trace below:
{code:java}
Traceback (most recent call last):
File "/usr/local/bin/airflow", line 22, in <module>
from airflow.bin.cli import CLIFactory
File "/usr/local/lib/python2.7/dist-packages/airflow/bin/cli.py", line 55, in
<module>
from airflow import jobs, settings
File "/usr/local/lib/python2.7/dist-packages/airflow/jobs/__init__.py", line
21, in <module>
from airflow.jobs.base_job import BaseJob # noqa: F401
File "/usr/local/lib/python2.7/dist-packages/airflow/jobs/base_job.py", line
48, in <module>
class BaseJob(Base, LoggingMixin):
File "/usr/local/lib/python2.7/dist-packages/airflow/jobs/base_job.py", line
82, in BaseJob
executor=executors.get_default_executor(),
File "/usr/local/lib/python2.7/dist-packages/airflow/executors/__init__.py",
line 48, in get_default_executor
DEFAULT_EXECUTOR = _get_executor(executor_name)
File "/usr/local/lib/python2.7/dist-packages/airflow/executors/__init__.py",
line 76, in _get_executor
from airflow.executors.celery_executor import CeleryExecutor
File
"/usr/local/lib/python2.7/dist-packages/airflow/executors/celery_executor.py",
line 27, in <module>
from celery import Celery
File "/usr/local/lib/python2.7/dist-packages/celery/local.py", line 509, in
__getattr__
module = __import__(self._object_origins[name], None, None, [name])
File "/usr/local/lib/python2.7/dist-packages/celery/app/__init__.py", line 5,
in <module>
from celery import _state
File "/usr/local/lib/python2.7/dist-packages/celery/_state.py", line 17, in
<module>
from celery.utils.threads import LocalStack
File "/usr/local/lib/python2.7/dist-packages/celery/utils/__init__.py", line
9, in <module>
from .nodenames import worker_direct, nodename, nodesplit
File "/usr/local/lib/python2.7/dist-packages/celery/utils/nodenames.py", line
9, in <module>
from kombu.entity import Exchange, Queue
File "/usr/local/lib/python2.7/dist-packages/kombu/entity.py", line 9, in
<module>
from .serialization import prepare_accept_content
File "/usr/local/lib/python2.7/dist-packages/kombu/serialization.py", line
456, in <module>
for ep, args in entrypoints('kombu.serializers'): # pragma: no cover
File "/usr/local/lib/python2.7/dist-packages/kombu/utils/compat.py", line 89,
in entrypoints
for ep in importlib_metadata.entry_points().get(namespace, [])
File "/usr/local/lib/python2.7/dist-packages/importlib_metadata/__init__.py",
line 456, in entry_points
ordered = sorted(eps, key=by_group)
File "/usr/local/lib/python2.7/dist-packages/importlib_metadata/__init__.py",
line 454, in <genexpr>
dist.entry_points for dist in distributions())
File "/usr/local/lib/python2.7/dist-packages/importlib_metadata/__init__.py",
line 364, in <genexpr>
cls._search_path(path, pattern)
File "/usr/local/lib/python2.7/dist-packages/importlib_metadata/__init__.py",
line 373, in _switch_path
return pathlib.Path(path)
File "/usr/local/lib/python2.7/dist-packages/pathlib2/__init__.py", line
1256, in __new__
self = cls._from_parts(args, init=False)
File "/usr/local/lib/python2.7/dist-packages/pathlib2/__init__.py", line 898,
in _from_parts
drv, root, parts = self._parse_args(args)
File "/usr/local/lib/python2.7/dist-packages/pathlib2/__init__.py", line 891,
in _parse_args
return cls._flavour.parse_parts(parts)
File "/usr/local/lib/python2.7/dist-packages/pathlib2/__init__.py", line 250,
in parse_parts
parsed.append(intern(x))
TypeError: can't intern subclass of string
{code}
--
This message was sent by Atlassian JIRA
(v7.6.14#76016)
