stoiandl commented on issue #14261:
URL: https://github.com/apache/airflow/issues/14261#issuecomment-780616603
15 mins later the scheduler comes back up:
```
[2021-02-17 14:47:02,093] {connectionpool.py:751} WARNING - Retrying
(Retry(total=2, connect=None, read=None, redirect=None, status=None)) after
connection broken by 'ProtocolError('Connection aborted.', OSError("(110,
'ETIMEDOUT')"))':
/api/v1/namespaces/airflow/pods?fieldSelector=status.phase%3DSucceeded&labelSelector=kubernetes_executor%3DTrue
```
Returns a bunch of pods config jsons in debug mode. Then:
```
2021-02-17 14:47:03,060] {scheduler_job.py:1390} DEBUG - Next timed event is
in 4.999955
[2021-02-17 14:47:03,060] {scheduler_job.py:1392} DEBUG - Ran scheduling
loop in 940.21 seconds
[2021-02-17 14:47:04,090] {scheduler_job.py:933} DEBUG - No tasks to
consider for execution.
[2021-02-17 14:47:04,096] {base_executor.py:147} DEBUG - 0 running task
instances
[2021-02-17 14:47:04,096] {base_executor.py:148} DEBUG - 0 in queue
[2021-02-17 14:47:04,096] {base_executor.py:149} DEBUG - 32 open slots
[2021-02-17 14:47:04,096] {base_executor.py:158} DEBUG - Calling the <class
'airflow.executors.kubernetes_executor.KubernetesExecutor'> sync method
[2021-02-17 14:47:04,096] {kubernetes_executor.py:337} DEBUG - Syncing
KubernetesExecutor
[2021-02-17 14:47:04,096] {kubernetes_executor.py:263} DEBUG -
KubeJobWatcher alive, continuing
[2021-02-17 14:47:04,098] {dag_processing.py:383} DEBUG - Received message
of type DagParsingStat
[2021-02-17 14:47:04,099] {dag_processing.py:383} DEBUG - Received message
of type DagParsingStat
[2021-02-17 14:47:04,099] {dag_processing.py:383} DEBUG - Received message
of type DagParsingStat
[2021-02-17 14:47:04,099] {dag_processing.py:383} DEBUG - Received message
of type DagParsingStat
[2021-02-17 14:47:04,099] {dag_processing.py:383} DEBUG - Received message
of type DagParsingStat
[2021-02-17 14:47:04,099] {dag_processing.py:383} DEBUG - Received message
of type DagParsingStat
[2021-02-17 14:47:04,099] {dag_processing.py:383} DEBUG - Received message
of type DagParsingStat
[2021-02-17 14:47:04,100] {dag_processing.py:383} DEBUG - Received message
of type DagParsingStat
[2021-02-17 14:47:04,100] {dag_processing.py:383} DEBUG - Received message
of type DagParsingStat
[2021-02-17 14:47:04,100] {dag_processing.py:383} DEBUG - Received message
of type DagParsingStat
[2021-02-17 14:47:04,100] {dag_processing.py:383} DEBUG - Received message
of type DagParsingStat
[2021-02-17 14:47:04,100] {dag_processing.py:383} DEBUG - Received message
of type DagParsingStat
[2021-02-17 14:47:04,101] {dag_processing.py:383} DEBUG - Received message
of type DagParsingStat
[2021-02-17 14:47:04,101] {dag_processing.py:383} DEBUG - Received message
of type DagParsingStat
[2021-02-17 14:47:04,101] {dag_processing.py:383} DEBUG - Received message
of type DagParsingStat
[2021-02-17 14:47:04,101] {dag_processing.py:383} DEBUG - Received message
of type DagParsingStat
[2021-02-17 14:47:04,101] {dag_processing.py:383} DEBUG - Received message
of type DagParsingStat
[2021-02-17 14:47:04,101] {dag_processing.py:383} DEBUG - Received message
of type DagParsingStat
[2021-02-17 14:47:04,101] {dag_processing.py:383} DEBUG - Received message
of type DagParsingStat
[2021-02-17 14:47:04,102] {dag_processing.py:383} DEBUG - Received message
of type DagParsingStat
[2021-02-17 14:47:04,102] {dag_processing.py:383} DEBUG - Received message
of type DagParsingStat
[2021-02-17 14:47:04,102] {dag_processing.py:383} DEBUG - Received message
of type DagParsingStat
[2021-02-17 14:47:04,102] {dag_processing.py:383} DEBUG - Received message
of type DagParsingStat
[2021-02-17 14:47:04,102] {dag_processing.py:383} DEBUG - Received message
of type DagParsingStat
[2021-02-17 14:47:04,102] {dag_processing.py:383} DEBUG - Received message
of type DagParsingStat
[2021-02-17 14:47:04,102] {dag_processing.py:383} DEBUG - Received message
of type DagParsingStat
[2021-02-17 14:47:04,103] {dag_processing.py:383} DEBUG - Received message
of type DagParsingStat
[2021-02-17 14:47:04,103] {dag_processing.py:383} DEBUG - Received message
of type DagParsingStat
[2021-02-17 14:47:04,103] {dag_processing.py:383} DEBUG - Received message
of type DagParsingStat
[2021-02-17 14:47:04,103] {dag_processing.py:383} DEBUG - Received message
of type DagParsingStat
[2021-02-17 14:47:04,103] {dag_processing.py:383} DEBUG - Received message
of type DagParsingStat
[2021-02-17 14:47:04,103] {dag_processing.py:383} DEBUG - Received message
of type DagParsingStat
[2021-02-17 14:47:04,104] {dag_processing.py:383} DEBUG - Received message
of type DagParsingStat
[2021-02-17 14:47:04,104] {dag_processing.py:383} DEBUG - Received message
of type DagParsingStat
[2021-02-17 14:47:04,104] {dag_processing.py:383} DEBUG - Received message
of type DagParsingStat
[2021-02-17 14:47:04,104] {dag_processing.py:383} DEBUG - Received message
of type DagParsingStat
[2021-02-17 14:47:04,104] {dag_processing.py:383} DEBUG - Received message
of type DagParsingStat
[2021-02-17 14:47:04,105] {dag_processing.py:383} DEBUG - Received message
of type DagParsingStat
[2021-02-17 14:47:04,105] {dag_processing.py:383} DEBUG - Received message
of type DagParsingStat
[2021-02-17 14:47:04,105] {dag_processing.py:383} DEBUG - Received message
of type DagParsingStat
[2021-02-17 14:47:04,105] {dag_processing.py:383} DEBUG - Received message
of type DagParsingStat
[2021-02-17 14:47:04,105] {dag_processing.py:383} DEBUG - Received message
of type DagParsingStat
[2021-02-17 14:47:04,105] {dag_processing.py:383} DEBUG - Received message
of type DagParsingStat
[2021-02-17 14:47:04,106] {dag_processing.py:383} DEBUG - Received message
of type DagParsingStat
[2021-02-17 14:47:04,106] {dag_processing.py:383} DEBUG - Received message
of type DagParsingStat
[2021-02-17 14:47:04,106] {dag_processing.py:383} DEBUG - Received message
of type DagParsingStat
[2021-02-17 14:47:04,106] {dag_processing.py:383} DEBUG - Received message
of type DagParsingStat
[2021-02-17 14:47:04,106] {dag_processing.py:383} DEBUG - Received message
of type DagParsingStat
[2021-02-17 14:47:04,106] {dag_processing.py:383} DEBUG - Received message
of type DagParsingStat
[2021-02-17 14:47:04,107] {dag_processing.py:383} DEBUG - Received message
of type DagParsingStat
[2021-02-17 14:47:04,107] {dag_processing.py:383} DEBUG - Received message
of type DagParsingStat
[2021-02-17 14:47:04,107] {dag_processing.py:383} DEBUG - Received message
of type DagParsingStat
[2021-02-17 14:47:04,107] {dag_processing.py:383} DEBUG - Received message
of type DagParsingStat
[2021-02-17 14:47:04,107] {dag_processing.py:383} DEBUG - Received message
of type DagParsingStat
[2021-02-17 14:47:04,107] {dag_processing.py:383} DEBUG - Received message
of type DagParsingStat
[2021-02-17 14:47:04,107] {dag_processing.py:383} DEBUG - Received message
of type DagParsingStat
[2021-02-17 14:47:04,107] {dag_processing.py:383} DEBUG - Received message
of type DagParsingStat
[2021-02-17 14:47:04,108] {dag_processing.py:383} DEBUG - Received message
of type DagParsingStat
[2021-02-17 14:47:04,108] {dag_processing.py:383} DEBUG - Received message
of type DagParsingStat
[2021-02-17 14:47:04,108] {dag_processing.py:383} DEBUG - Received message
of type DagParsingStat
[2021-02-17 14:47:04,108] {dag_processing.py:383} DEBUG - Received message
of type DagParsingStat
[2021-02-17 14:47:04,108] {dag_processing.py:383} DEBUG - Received message
of type DagParsingStat
[2021-02-17 14:47:04,108] {dag_processing.py:383} DEBUG - Received message
of type DagParsingStat
[2021-02-17 14:47:04,109] {dag_processing.py:383} DEBUG - Received message
of type DagParsingStat
[2021-02-17 14:47:04,109] {dag_processing.py:383} DEBUG - Received message
of type DagParsingStat
[2021-02-17 14:47:04,109] {dag_processing.py:383} DEBUG - Received message
of type DagParsingStat
[2021-02-17 14:47:04,109] {dag_processing.py:383} DEBUG - Received message
of type DagParsingStat
[2021-02-17 14:47:04,109] {dag_processing.py:383} DEBUG - Received message
of type DagParsingStat
[2021-02-17 14:47:04,109] {dag_processing.py:383} DEBUG - Received message
of type DagParsingStat
[2021-02-17 14:47:04,110] {dag_processing.py:383} DEBUG - Received message
of type DagParsingStat
[2021-02-17 14:47:04,110] {dag_processing.py:383} DEBUG - Received message
of type DagParsingStat
[2021-02-17 14:47:04,110] {dag_processing.py:383} DEBUG - Received message
of type DagParsingStat
[2021-02-17 14:47:04,110] {dag_processing.py:383} DEBUG - Received message
of type DagParsingStat
[2021-02-17 14:47:04,110] {dag_processing.py:383} DEBUG - Received message
of type DagParsingStat
[2021-02-17 14:47:04,110] {dag_processing.py:383} DEBUG - Received message
of type DagParsingStat
[2021-02-17 14:47:04,110] {dag_processing.py:383} DEBUG - Received message
of type DagParsingStat
[2021-02-17 14:47:04,111] {dag_processing.py:383} DEBUG - Received message
of type DagParsingStat
[2021-02-17 14:47:04,111] {dag_processing.py:383} DEBUG - Received message
of type DagParsingStat
[2021-02-17 14:47:04,111] {dag_processing.py:383} DEBUG - Received message
of type DagParsingStat
[2021-02-17 14:47:04,111] {dag_processing.py:383} DEBUG - Received message
of type DagParsingStat
[2021-02-17 14:47:04,111] {dag_processing.py:383} DEBUG - Received message
of type DagParsingStat
[2021-02-17 14:47:04,111] {dag_processing.py:383} DEBUG - Received message
of type DagParsingStat
[2021-02-17 14:47:04,111] {dag_processing.py:383} DEBUG - Received message
of type DagParsingStat
[2021-02-17 14:47:04,112] {dag_processing.py:383} DEBUG - Received message
of type DagParsingStat
[2021-02-17 14:47:04,112] {dag_processing.py:383} DEBUG - Received message
of type DagParsingStat
[2021-02-17 14:47:04,112] {dag_processing.py:383} DEBUG - Received message
of type DagParsingStat
[2021-02-17 14:47:04,112] {dag_processing.py:383} DEBUG - Received message
of type DagParsingStat
[2021-02-17 14:47:04,112] {dag_processing.py:383} DEBUG - Received message
of type DagParsingStat
[2021-02-17 14:47:04,112] {dag_processing.py:383} DEBUG - Received message
of type DagParsingStat
[2021-02-17 14:47:04,113] {dag_processing.py:383} DEBUG - Received message
of type DagParsingStat
[2021-02-17 14:47:04,113] {dag_processing.py:383} DEBUG - Received message
of type DagParsingStat
[2021-02-17 14:47:04,113] {dag_processing.py:383} DEBUG - Received message
of type DagParsingStat
[2021-02-17 14:47:04,113] {dag_processing.py:383} DEBUG - Received message
of type DagParsingStat
[2021-02-17 14:47:04,113] {dag_processing.py:383} DEBUG - Received message
of type DagParsingStat
[2021-02-17 14:47:04,113] {dag_processing.py:383} DEBUG - Received message
of type DagParsingStat
[2021-02-17 14:47:04,114] {dag_processing.py:383} DEBUG - Received message
of type DagParsingStat
[2021-02-17 14:47:04,114] {dag_processing.py:383} DEBUG - Received message
of type DagParsingStat
[2021-02-17 14:47:04,114] {dag_processing.py:383} DEBUG - Received message
of type DagParsingStat
[2021-02-17 14:47:04,114] {dag_processing.py:383} DEBUG - Received message
of type DagParsingStat
[2021-02-17 14:47:04,114] {dag_processing.py:383} DEBUG - Received message
of type DagParsingStat
[2021-02-17 14:47:04,115] {dag_processing.py:383} DEBUG - Received message
of type DagParsingStat
[2021-02-17 14:47:04,115] {dag_processing.py:383} DEBUG - Received message
of type DagParsingStat
[2021-02-17 14:47:04,115] {dag_processing.py:383} DEBUG - Received message
of type DagParsingStat
[2021-02-17 14:47:04,115] {dag_processing.py:383} DEBUG - Received message
of type DagParsingStat
[2021-02-17 14:47:04,115] {dag_processing.py:383} DEBUG - Received message
of type DagParsingStat
[2021-02-17 14:47:04,115] {dag_processing.py:383} DEBUG - Received message
of type DagParsingStat
[2021-02-17 14:47:04,116] {dag_processing.py:383} DEBUG - Received message
of type DagParsingStat
[2021-02-17 14:47:04,116] {dag_processing.py:383} DEBUG - Received message
of type DagParsingStat
[2021-02-17 14:47:04,116] {dag_processing.py:383} DEBUG - Received message
of type DagParsingStat
[2021-02-17 14:47:04,116] {dag_processing.py:383} DEBUG - Received message
of type DagParsingStat
[2021-02-17 14:47:04,116] {dag_processing.py:383} DEBUG - Received message
of type DagParsingStat
[2021-02-17 14:47:04,117] {dag_processing.py:383} DEBUG - Received message
of type DagParsingStat
[2021-02-17 14:47:04,117] {dag_processing.py:383} DEBUG - Received message
of type DagParsingStat
[2021-02-17 14:47:04,117] {dag_processing.py:383} DEBUG - Received message
of type DagParsingStat
[2021-02-17 14:47:04,117] {dag_processing.py:383} DEBUG - Received message
of type DagParsingStat
[2021-02-17 14:47:04,117] {dag_processing.py:383} DEBUG - Received message
of type DagParsingStat
[2021-02-17 14:47:04,117] {dag_processing.py:383} DEBUG - Received message
of type DagParsingStat
[2021-02-17 14:47:04,117] {dag_processing.py:383} DEBUG - Received message
of type DagParsingStat
[2021-02-17 14:47:04,118] {dag_processing.py:383} DEBUG - Received message
of type DagParsingStat
[2021-02-17 14:47:04,118] {dag_processing.py:383} DEBUG - Received message
of type DagParsingStat
[2021-02-17 14:47:04,118] {dag_processing.py:383} DEBUG - Received message
of type DagParsingStat
[2021-02-17 14:47:04,118] {dag_processing.py:383} DEBUG - Received message
of type DagParsingStat
[2021-02-17 14:47:04,118] {dag_processing.py:383} DEBUG - Received message
of type DagParsingStat
[2021-02-17 14:47:04,118] {dag_processing.py:383} DEBUG - Received message
of type DagParsingStat
[2021-02-17 14:47:04,118] {dag_processing.py:383} DEBUG - Received message
of type DagParsingStat
[2021-02-17 14:47:04,119] {dag_processing.py:383} DEBUG - Received message
of type DagParsingStat
[2021-02-17 14:47:04,119] {dag_processing.py:383} DEBUG - Received message
of type DagParsingStat
[2021-02-17 14:47:04,119] {dag_processing.py:383} DEBUG - Received message
of type DagParsingStat
[2021-02-17 14:47:04,119] {dag_processing.py:383} DEBUG - Received message
of type DagParsingStat
[2021-02-17 14:47:04,119] {dag_processing.py:383} DEBUG - Received message
of type DagParsingStat
[2021-02-17 14:47:04,119] {dag_processing.py:383} DEBUG - Received message
of type DagParsingStat
[2021-02-17 14:47:04,120] {dag_processing.py:383} DEBUG - Received message
of type DagParsingStat
[2021-02-17 14:47:04,120] {dag_processing.py:383} DEBUG - Received message
of type DagParsingStat
[2021-02-17 14:47:04,120] {dag_processing.py:383} DEBUG - Received message
of type DagParsingStat
[2021-02-17 14:47:04,120] {dag_processing.py:383} DEBUG - Received message
of type DagParsingStat
[2021-02-17 14:47:04,120] {dag_processing.py:383} DEBUG - Received message
of type DagParsingStat
[2021-02-17 14:47:04,120] {dag_processing.py:383} DEBUG - Received message
of type DagParsingStat
[2021-02-17 14:47:04,120] {dag_processing.py:383} DEBUG - Received message
of type DagParsingStat
[2021-02-17 14:47:04,121] {dag_processing.py:383} DEBUG - Received message
of type DagParsingStat
[2021-02-17 14:47:04,121] {dag_processing.py:383} DEBUG - Received message
of type DagParsingStat
[2021-02-17 14:47:04,121] {dag_processing.py:383} DEBUG - Received message
of type DagParsingStat
[2021-02-17 14:47:04,121] {dag_processing.py:383} DEBUG - Received message
of type DagParsingStat
[2021-02-17 14:47:04,121] {dag_processing.py:383} DEBUG - Received message
of type DagParsingStat
[2021-02-17 14:47:04,122] {dag_processing.py:383} DEBUG - Received message
of type DagParsingStat
[2021-02-17 14:47:04,122] {dag_processing.py:383} DEBUG - Received message
of type DagParsingStat
[2021-02-17 14:47:04,122] {dag_processing.py:383} DEBUG - Received message
of type DagParsingStat
[2021-02-17 14:47:04,122] {dag_processing.py:383} DEBUG - Received message
of type DagParsingStat
[2021-02-17 14:47:04,122] {dag_processing.py:383} DEBUG - Received message
of type DagParsingStat
[2021-02-17 14:47:04,122] {dag_processing.py:383} DEBUG - Received message
of type DagParsingStat
[2021-02-17 14:47:04,123] {dag_processing.py:383} DEBUG - Received message
of type DagParsingStat
[2021-02-17 14:47:04,123] {dag_processing.py:383} DEBUG - Received message
of type DagParsingStat
[2021-02-17 14:47:04,123] {dag_processing.py:383} DEBUG - Received message
of type DagParsingStat
[2021-02-17 14:47:04,123] {dag_processing.py:383} DEBUG - Received message
of type DagParsingStat
[2021-02-17 14:47:04,123] {dag_processing.py:383} DEBUG - Received message
of type DagParsingStat
[2021-02-17 14:47:04,123] {dag_processing.py:383} DEBUG - Received message
of type DagParsingStat
[2021-02-17 14:47:04,124] {dag_processing.py:383} DEBUG - Received message
of type DagParsingStat
[2021-02-17 14:47:04,124] {dag_processing.py:383} DEBUG - Received message
of type DagParsingStat
[2021-02-17 14:47:04,124] {dag_processing.py:383} DEBUG - Received message
of type DagParsingStat
[2021-02-17 14:47:04,124] {dag_processing.py:383} DEBUG - Received message
of type DagParsingStat
[2021-02-17 14:47:04,124] {dag_processing.py:383} DEBUG - Received message
of type DagParsingStat
[2021-02-17 14:47:04,124] {dag_processing.py:383} DEBUG - Received message
of type DagParsingStat
[2021-02-17 14:47:04,125] {dag_processing.py:383} DEBUG - Received message
of type DagParsingStat
[2021-02-17 14:47:04,125] {dag_processing.py:383} DEBUG - Received message
of type DagParsingStat
[2021-02-17 14:47:04,125] {dag_processing.py:383} DEBUG - Received message
of type DagParsingStat
[2021-02-17 14:47:04,125] {dag_processing.py:383} DEBUG - Received message
of type DagParsingStat
[2021-02-17 14:47:04,125] {dag_processing.py:383} DEBUG - Received message
of type DagParsingStat
[2021-02-17 14:47:04,125] {dag_processing.py:383} DEBUG - Received message
of type DagParsingStat
[2021-02-17 14:47:04,125] {dag_processing.py:383} DEBUG - Received message
of type DagParsingStat
[2021-02-17 14:47:04,126] {dag_processing.py:383} DEBUG - Received message
of type DagParsingStat
[2021-02-17 14:47:04,126] {dag_processing.py:383} DEBUG - Received message
of type DagParsingStat
[2021-02-17 14:47:04,126] {dag_processing.py:383} DEBUG - Received message
of type DagParsingStat
[2021-02-17 14:47:04,126] {dag_processing.py:383} DEBUG - Received message
of type DagParsingStat
[2021-02-17 14:47:04,126] {dag_processing.py:383} DEBUG - Received message
of type DagParsingStat
[2021-02-17 14:47:04,126] {dag_processing.py:383} DEBUG - Received message
of type DagParsingStat
[2021-02-17 14:47:04,127] {dag_processing.py:383} DEBUG - Received message
of type DagParsingStat
[2021-02-17 14:47:04,127] {dag_processing.py:383} DEBUG - Received message
of type DagParsingStat
[2021-02-17 14:47:04,127] {dag_processing.py:383} DEBUG - Received message
of type DagParsingStat
[2021-02-17 14:47:04,127] {dag_processing.py:383} DEBUG - Received message
of type DagParsingStat
[2021-02-17 14:47:04,127] {dag_processing.py:383} DEBUG - Received message
of type DagParsingStat
[2021-02-17 14:47:04,127] {dag_processing.py:383} DEBUG - Received message
of type DagParsingStat
[2021-02-17 14:47:04,128] {dag_processing.py:383} DEBUG - Received message
of type DagParsingStat
[2021-02-17 14:47:04,128] {dag_processing.py:383} DEBUG - Received message
of type DagParsingStat
[2021-02-17 14:47:04,128] {dag_processing.py:383} DEBUG - Received message
of type DagParsingStat
[2021-02-17 14:47:04,128] {dag_processing.py:383} DEBUG - Received message
of type DagParsingStat
[2021-02-17 14:47:04,128] {dag_processing.py:383} DEBUG - Received message
of type DagParsingStat
[2021-02-17 14:47:04,128] {dag_processing.py:383} DEBUG - Received message
of type DagParsingStat
[2021-02-17 14:47:04,129] {dag_processing.py:383} DEBUG - Received message
of type DagParsingStat
[2021-02-17 14:47:04,129] {dag_processing.py:383} DEBUG - Received message
of type DagParsingStat
[2021-02-17 14:47:04,129] {dag_processing.py:383} DEBUG - Received message
of type DagParsingStat
[2021-02-17 14:47:04,129] {dag_processing.py:383} DEBUG - Received message
of type DagParsingStat
[2021-02-17 14:47:04,130] {dag_processing.py:383} DEBUG - Received message
of type DagParsingStat
[2021-02-17 14:47:04,130] {dag_processing.py:383} DEBUG - Received message
of type DagParsingStat
[2021-02-17 14:47:04,130] {dag_processing.py:383} DEBUG - Received message
of type DagParsingStat
[2021-02-17 14:47:04,130] {dag_processing.py:383} DEBUG - Received message
of type DagParsingStat
[2021-02-17 14:47:04,130] {dag_processing.py:383} DEBUG - Received message
of type DagParsingStat
[2021-02-17 14:47:04,130] {dag_processing.py:383} DEBUG - Received message
of type DagParsingStat
[2021-02-17 14:47:04,131] {dag_processing.py:383} DEBUG - Received message
of type DagParsingStat
[2021-02-17 14:47:04,131] {dag_processing.py:383} DEBUG - Received message
of type DagParsingStat
[2021-02-17 14:47:04,131] {dag_processing.py:383} DEBUG - Received message
of type DagParsingStat
[2021-02-17 14:47:04,131] {dag_processing.py:383} DEBUG - Received message
of type DagParsingStat
[2021-02-17 14:47:04,131] {dag_processing.py:383} DEBUG - Received message
of type DagParsingStat
[2021-02-17 14:47:04,131] {dag_processing.py:383} DEBUG - Received message
of type DagParsingStat
[2021-02-17 14:47:04,132] {dag_processing.py:383} DEBUG - Received message
of type DagParsingStat
[2021-02-17 14:47:04,132] {dag_processing.py:383} DEBUG - Received message
of type DagParsingStat
[2021-02-17 14:47:04,132] {dag_processing.py:383} DEBUG - Received message
of type DagParsingStat
[2021-02-17 14:47:04,132] {dag_processing.py:383} DEBUG - Received message
of type DagParsingStat
[2021-02-17 14:47:04,132] {dag_processing.py:383} DEBUG - Received message
of type DagParsingStat
[2021-02-17 14:47:04,133] {dag_processing.py:383} DEBUG - Received message
of type DagParsingStat
[2021-02-17 14:47:04,134] {scheduler_job.py:309} DEBUG - Waiting for
<ForkProcess name='DagFileProcessor676-Process' pid=4773 parent=34 stopped
exitcode=0>
[2021-02-17 14:47:04,137] {dag_processing.py:383} DEBUG - Received message
of type DagParsingStat
[2021-02-17 14:47:04,137] {dag_processing.py:383} DEBUG - Received message
of type DagParsingStat
[2021-02-17 14:47:04,137] {dag_processing.py:383} DEBUG - Received message
of type DagParsingStat
[2021-02-17 14:47:04,137] {dag_processing.py:383} DEBUG - Received message
of type DagParsingStat
[2021-02-17 14:47:04,137] {dag_processing.py:383} DEBUG - Received message
of type DagParsingStat
[2021-02-17 14:47:04,137] {dag_processing.py:383} DEBUG - Received message
of type DagParsingStat
[2021-02-17 14:47:04,138] {dag_processing.py:383} DEBUG - Received message
of type DagParsingStat
[2021-02-17 14:47:04,138] {dag_processing.py:383} DEBUG - Received message
of type DagParsingStat
[2021-02-17 14:47:04,138] {dag_processing.py:383} DEBUG - Received message
of type DagParsingStat
[2021-02-17 14:47:04,138] {dag_processing.py:383} DEBUG - Received message
of type DagParsingStat
[2021-02-17 14:47:04,138] {dag_processing.py:383} DEBUG - Received message
of type DagParsingStat
[2021-02-17 14:47:04,139] {dag_processing.py:383} DEBUG - Received message
of type DagParsingStat
[2021-02-17 14:47:04,139] {dag_processing.py:383} DEBUG - Received message
of type DagParsingStat
[2021-02-17 14:47:04,139] {dag_processing.py:383} DEBUG - Received message
of type DagParsingStat
[2021-02-17 14:47:04,139] {dag_processing.py:383} DEBUG - Received message
of type DagParsingStat
[2021-02-17 14:47:04,139] {dag_processing.py:383} DEBUG - Received message
of type DagParsingStat
[2021-02-17 14:47:04,139] {dag_processing.py:383} DEBUG - Received message
of type DagParsingStat
[2021-02-17 14:47:04,139] {dag_processing.py:383} DEBUG - Received message
of type DagParsingStat
[2021-02-17 14:47:04,140] {dag_processing.py:383} DEBUG - Received message
of type DagParsingStat
[2021-02-17 14:47:04,140] {dag_processing.py:383} DEBUG - Received message
of type DagParsingStat
[2021-02-17 14:47:04,140] {dag_processing.py:383} DEBUG - Received message
of type DagParsingStat
[2021-02-17 14:47:04,140] {dag_processing.py:383} DEBUG - Received message
of type DagParsingStat
[2021-02-17 14:47:04,140] {dag_processing.py:383} DEBUG - Received message
of type DagParsingStat
[2021-02-17 14:47:04,140] {dag_processing.py:383} DEBUG - Received message
of type DagParsingStat
[2021-02-17 14:47:04,140] {dag_processing.py:383} DEBUG - Received message
of type DagParsingStat
[2021-02-17 14:47:04,141] {dag_processing.py:383} DEBUG - Received message
of type DagParsingStat
[2021-02-17 14:47:04,141] {dag_processing.py:383} DEBUG - Received message
of type DagParsingStat
[2021-02-17 14:47:04,141] {dag_processing.py:383} DEBUG - Received message
of type DagParsingStat
[2021-02-17 14:47:04,141] {dag_processing.py:383} DEBUG - Received message
of type DagParsingStat
[2021-02-17 14:47:04,141] {dag_processing.py:383} DEBUG - Received message
of type DagParsingStat
[2021-02-17 14:47:04,141] {dag_processing.py:383} DEBUG - Received message
of type DagParsingStat
[2021-02-17 14:47:04,141] {dag_processing.py:383} DEBUG - Received message
of type DagParsingStat
[2021-02-17 14:47:04,142] {dag_processing.py:383} DEBUG - Received message
of type DagParsingStat
[2021-02-17 14:47:04,142] {dag_processing.py:383} DEBUG - Received message
of type DagParsingStat
[2021-02-17 14:47:04,142] {dag_processing.py:383} DEBUG - Received message
of type DagParsingStat
[2021-02-17 14:47:04,142] {dag_processing.py:383} DEBUG - Received message
of type DagParsingStat
[2021-02-17 14:47:04,142] {dag_processing.py:383} DEBUG - Received message
of type DagParsingStat
[2021-02-17 14:47:04,142] {dag_processing.py:383} DEBUG - Received message
of type DagParsingStat
[2021-02-17 14:47:04,142] {dag_processing.py:383} DEBUG - Received message
of type DagParsingStat
[2021-02-17 14:47:04,142] {serialized_dag.py:193} DEBUG - Deleting
Serialized DAGs (for which DAG files are deleted) from serialized_dag table
[2021-02-17 14:47:04,143] {dag_processing.py:383} DEBUG - Received message
of type DagParsingStat
[2021-02-17 14:47:04,143] {dag_processing.py:383} DEBUG - Received message
of type DagParsingStat
[2021-02-17 14:47:04,143] {dag_processing.py:383} DEBUG - Received message
of type DagParsingStat
[2021-02-17 14:47:04,143] {dag_processing.py:383} DEBUG - Received message
of type DagParsingStat
[2021-02-17 14:47:04,143] {dag_processing.py:383} DEBUG - Received message
of type DagParsingStat
[2021-02-17 14:47:04,143] {dag_processing.py:383} DEBUG - Received message
of type DagParsingStat
[2021-02-17 14:47:04,144] {dag_processing.py:383} DEBUG - Received message
of type DagParsingStat
[2021-02-17 14:47:04,144] {dag_processing.py:383} DEBUG - Received message
of type DagParsingStat
[2021-02-17 14:47:04,144] {dag_processing.py:383} DEBUG - Received message
of type DagParsingStat
[2021-02-17 14:47:04,144] {dag_processing.py:383} DEBUG - Received message
of type DagParsingStat
[2021-02-17 14:47:04,144] {dag_processing.py:383} DEBUG - Received message
of type DagParsingStat
[2021-02-17 14:47:04,145] {dag_processing.py:383} DEBUG - Received message
of type DagParsingStat
[2021-02-17 14:47:04,145] {dag_processing.py:383} DEBUG - Received message
of type DagParsingStat
[2021-02-17 14:47:04,145] {dag_processing.py:383} DEBUG - Received message
of type DagParsingStat
[2021-02-17 14:47:04,145] {dag_processing.py:383} DEBUG - Received message
of type DagParsingStat
[2021-02-17 14:47:04,146] {dag_processing.py:383} DEBUG - Received message
of type DagParsingStat
[2021-02-17 14:47:04,146] {dag_processing.py:383} DEBUG - Received message
of type DagParsingStat
[2021-02-17 14:47:04,146] {dag_processing.py:383} DEBUG - Received message
of type DagParsingStat
[2021-02-17 14:47:04,146] {dag_processing.py:383} DEBUG - Received message
of type DagParsingStat
[2021-02-17 14:47:04,146] {dag_processing.py:383} DEBUG - Received message
of type DagParsingStat
[2021-02-17 14:47:04,146] {dag_processing.py:383} DEBUG - Received message
of type DagParsingStat
[2021-02-17 14:47:04,147] {dag_processing.py:383} DEBUG - Received message
of type DagParsingStat
[2021-02-17 14:47:04,147] {dag_processing.py:383} DEBUG - Received message
of type DagParsingStat
[2021-02-17 14:47:04,147] {dag_processing.py:383} DEBUG - Received message
of type DagParsingStat
[2021-02-17 14:47:04,147] {dag_processing.py:383} DEBUG - Received message
of type DagParsingStat
[2021-02-17 14:47:04,147] {dag_processing.py:383} DEBUG - Received message
of type DagParsingStat
[2021-02-17 14:47:04,147] {dag.py:2199} DEBUG - Deactivating DAGs (for which
DAG files are deleted) from dag table
[2021-02-17 14:47:04,147] {dag_processing.py:383} DEBUG - Received message
of type DagParsingStat
[2021-02-17 14:47:04,148] {dag_processing.py:383} DEBUG - Received message
of type DagParsingStat
[2021-02-17 14:47:04,148] {dag_processing.py:383} DEBUG - Received message
of type DagParsingStat
[2021-02-17 14:47:04,148] {dag_processing.py:383} DEBUG - Received message
of type DagParsingStat
[2021-02-17 14:47:04,148] {dag_processing.py:383} DEBUG - Received message
of type DagParsingStat
[2021-02-17 14:47:04,148] {dag_processing.py:383} DEBUG - Received message
of type DagParsingStat
[2021-02-17 14:47:04,152] {dagcode.py:135} DEBUG - Deleting code from
dag_code table
[2021-02-17 14:47:04,174] {base_job.py:219} DEBUG - [heartbeat]
```
This might be because I have many terminated worker pods that are not
deleted? They come from either a deleted DAG or previous airflow 1.10.13.
I cleared my namespace of pods and now I will rerun with liveness probe up.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
