hadoopBrogrammers opened a new issue #15799:
URL: https://github.com/apache/airflow/issues/15799
airflow version 2.0.2
rabbitmqctl version: 3.8.16
celery worker version : 4.4.7 (cliffs)
airflow installed on Kubernetes ..
1. It is working fine but after some run worker node got stuck with
following error ....
File "/home/airflow/.local/lib/python3.6/site-packages/amqp/channel.py",
line 280, in _on_close
reply_code, reply_text, (class_id, method_id), ChannelError,
amqp.exceptions.PreconditionFailed: (0, 0): (406) PRECONDITION_FAILED -
consumer ack timed out on channel 1
2. When I checked Rabitmq log then I see following error....
2021-05-12 10:31:13.882 [info] <0.19109.0> Closing all channels from
connection '*.*.*.216:37624 -> *.*.*.214:5672' because it has been closed
2021-05-12 10:31:14.135 [info] <0.19114.0> accepting AMQP connection
<0.19114.0> (*.*.*.216:37644 -> *.*.*.214:5672)
2021-05-12 10:31:14.137 [info] <0.19114.0> connection <0.19114.0>
(*.*.*.216:37644 -> *.*.*.214:5672): user 'airflow' authenticated and granted
access to vhost 'airflow'
2021-05-12 12:15:01.523 [warning] <0.2188.0> Consumer None4 on channel 1 has
timed out waiting on consumer acknowledgement. Timeout used: 900000 ms
2021-05-12 12:15:01.525 [error] <0.2188.0> Channel error on connection
<0.2173.0> (*.*.*.217:51938 -> *.*.*.214:5672, vhost: 'airflow', user:
'airflow'), channel 1:
operation none caused a channel exception precondition_failed: consumer ack
timed out on channel 1
2021-05-12 12:17:33.050 [error] <0.2173.0> closing AMQP connection
<0.2173.0> (*.*.*.217:51938 -> *.*.*.214:5672):
missed heartbeats from client, timeout: 60s
2021-05-12 12:17:33.050 [info] <0.20579.0> Closing all channels from
connection '*.*.*.217:51938 -> *.*.*.214:5672' because it has been closed
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
