Abhishek Ray created AIRFLOW-4452:
-------------------------------------
Summary: Webserver and Scheduler keep crashing because of
slackclient update
Key: AIRFLOW-4452
URL: https://issues.apache.org/jira/browse/AIRFLOW-4452
Project: Apache Airflow
Issue Type: Bug
Components: scheduler, webserver
Affects Versions: 1.10.1
Reporter: Abhishek Ray
Webserver and Scheduler get into a crash loop if Airflow is installed with
slack dependencies.
Airflow relies on slackclient which released a new major version (2.0.0) today
([https://pypi.org/project/slackclient/#history]). This new version seems to be
incompatible with Airflow causing the webserver to get into a crash loop.
The root cause of the issue is that Airflow doesn't pin requirements for
slackclient:
[https://github.com/apache/airflow/blob/v1-10-stable/setup.py#L229]
{code:java}
slack = ['slackclient>=1.0.0']{code}
This is the exception in the logs due to this error:
{code:java}
File "/Users/abhishek.ray/airflow/dags/test_dag.py", line 3, in <module>
from airflow.operators import SlackAPIPostOperator
File
"/Users/abhishek.ray/.virtualenvs/airflow-test/lib/python3.6/site-packages/airflow/utils/helpers.py",
line 372, in __getattr__
loaded_attribute = self._load_attribute(attribute)
File
"/Users/abhishek.ray/.virtualenvs/airflow-test/lib/python3.6/site-packages/airflow/utils/helpers.py",
line 336, in _load_attribute
self._loaded_modules[module] = imp.load_module(module, f, filename,
description)
File "/Users/abhishek.ray/.virtualenvs/airflow-test/lib/python3.6/imp.py",
line 235, in load_module
return load_source(name, filename, file)
File "/Users/abhishek.ray/.virtualenvs/airflow-test/lib/python3.6/imp.py",
line 172, in load_source
module = _load(spec)
File
"/Users/abhishek.ray/.virtualenvs/airflow-test/lib/python3.6/site-packages/airflow/operators/slack_operator.py",
line 24, in <module>
from airflow.hooks.slack_hook import SlackHook
File
"/Users/abhishek.ray/.virtualenvs/airflow-test/lib/python3.6/site-packages/airflow/hooks/slack_hook.py",
line 20, in <module>
from slackclient import SlackClient
File
"/Users/abhishek.ray/.virtualenvs/airflow-test/lib/python3.6/site-packages/slackclient/__init__.py",
line 1, in <module>
from .client import SlackClient # noqa
File
"/Users/abhishek.ray/.virtualenvs/airflow-test/lib/python3.6/site-packages/slackclient/client.py",
line 8, in <module>
from .server import Server
File
"/Users/abhishek.ray/.virtualenvs/airflow-test/lib/python3.6/site-packages/slackclient/server.py",
line 14, in <module>
from websocket import create_connection
ModuleNotFoundError: No module named 'websocket'
{code}
This is how to reproduce this issue:
Install apache airflow with slack:
{code:java}
pip install apache-airflow[slack]==1.10.1{code}
Create a DAG which uses *SlackAPIPostOperator*
{code:java}
from airflow import DAG
from airflow.operators.bash_operator import BashOperator
from airflow.operators import SlackAPIPostOperator
dag_default_args = {
"owner": "airflow",
"depends_on_past": False,
"start_date": datetime(2019, 4, 22),
"email": ["[email protected]"],
"email_on_failure": False,
"email_on_retry": False,
"retries": 1,
"catchup": True,
}
dag = DAG("test_dag", default_args=dag_default_args, schedule_interval="@daily")
BashOperator(task_id="print_date", bash_command="date", dag=dag){code}
I think the fix should be pretty straightforward to add a max version for
slackclient.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
