Stefan Seelmann created AIRFLOW-3353:
----------------------------------------
Summary: redis-py 3.0.0 dependency breaks celery executor
Key: AIRFLOW-3353
URL: https://issues.apache.org/jira/browse/AIRFLOW-3353
Project: Apache Airflow
Issue Type: Bug
Components: celery
Affects Versions: 1.10.0
Reporter: Stefan Seelmann
redis-py 3.0.0 was just released. Airflow 1.10.0 defines redis>=2.10.5 so
installs redis-py 3.0.0 now.
Error in worker below.
Workaround: Pin redis==2.10.6 (e.g. in constraints.txt)
{code}
[2018-11-15 12:06:18,441: CRITICAL/MainProcess] Unrecoverable error:
AttributeError("'float' object has no attribute 'items'",)
Traceback (most recent call last):
File "/usr/local/lib/python3.6/site-packages/celery/worker/worker.py", line
205, in start
self.blueprint.start(self)
File "/usr/local/lib/python3.6/site-packages/celery/bootsteps.py", line 119,
in start
step.start(parent)
File "/usr/local/lib/python3.6/site-packages/celery/bootsteps.py", line 369,
in start
return self.obj.start()
File
"/usr/local/lib/python3.6/site-packages/celery/worker/consumer/consumer.py",
line 317, in start
blueprint.start(self)
File "/usr/local/lib/python3.6/site-packages/celery/bootsteps.py", line 119,
in start
step.start(parent)
File
"/usr/local/lib/python3.6/site-packages/celery/worker/consumer/consumer.py",
line 593, in start
c.loop(*c.loop_args())
File "/usr/local/lib/python3.6/site-packages/celery/worker/loops.py", line
91, in asynloop
next(loop)
File "/usr/local/lib/python3.6/site-packages/kombu/asynchronous/hub.py", line
354, in create_loop
cb(*cbargs)
File "/usr/local/lib/python3.6/site-packages/kombu/transport/redis.py", line
1040, in on_readable
self.cycle.on_readable(fileno)
File "/usr/local/lib/python3.6/site-packages/kombu/transport/redis.py", line
337, in on_readable
chan.handlers[type]()
File "/usr/local/lib/python3.6/site-packages/kombu/transport/redis.py", line
724, in _brpop_read
self.connection._deliver(loads(bytes_to_str(item)), dest)
File
"/usr/local/lib/python3.6/site-packages/kombu/transport/virtual/base.py", line
983, in _deliver
callback(message)
File
"/usr/local/lib/python3.6/site-packages/kombu/transport/virtual/base.py", line
632, in _callback
self.qos.append(message, message.delivery_tag)
File "/usr/local/lib/python3.6/site-packages/kombu/transport/redis.py", line
149, in append
pipe.zadd(self.unacked_index_key, time(), delivery_tag) \
File "/usr/local/lib/python3.6/site-packages/redis/client.py", line 2263, in
zadd
for pair in iteritems(mapping):
File "/usr/local/lib/python3.6/site-packages/redis/_compat.py", line 123, in
iteritems
return iter(x.items())
AttributeError: 'float' object has no attribute 'items'
{code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
