[
https://issues.apache.org/jira/browse/AIRFLOW-166?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Yuri Bendana updated AIRFLOW-166:
---------------------------------
Description:
I just upgraded to 1.7.1.2 from 1.6 and I'm having a problem starting the
webserver using the systemd script. This used to work fine. The issue is that
it starts and then just hangs, no error is reported and it finally times out
after about a minute. I tried starting it from the command line and it works
fine without timing out. I also ran it in daemon mode with -D and again it
seems to be fine. Any thoughts on how to debug this?
Here's the log output:
{code}
May 23 16:27:50 ybendana-linux systemd: Starting Airflow webserver daemon...
May 23 16:27:51 ybendana-linux airflow: [2016-05-23 16:27:51,444]
{__init__.py:36} INFO - Using executor LocalExecutor
May 23 16:27:53 ybendana-linux airflow: ____________ _____________
May 23 16:27:53 ybendana-linux airflow: ____ |__( )_________ __/__
/________ __
May 23 16:27:53 ybendana-linux airflow: ____ /| |_ /__ ___/_ /_ __ /_ __
\_ | /| / /
May 23 16:27:53 ybendana-linux airflow: ___ ___ | / _ / _ __/ _ / / /_/
/_ |/ |/ /
May 23 16:27:53 ybendana-linux airflow: _/_/ |_/_/ /_/ /_/ /_/
\____/____/|__/
May 23 16:27:53 ybendana-linux airflow: [2016-05-23 16:27:53,446]
{models.py:154} INFO - Filling up the DagBag from /opt/airflow/dags
May 23 16:27:55 ybendana-linux airflow: [2016-05-23 16:27:55 +0000] [15960]
[INFO] Starting gunicorn 19.3.0
May 23 16:27:55 ybendana-linux airflow: [2016-05-23 16:27:55 +0000] [15960]
[INFO] Listening at: http://0.0.0.0:8080 (15960)
May 23 16:27:55 ybendana-linux airflow: [2016-05-23 16:27:55 +0000] [15960]
[INFO] Using worker: sync
May 23 16:27:55 ybendana-linux airflow: [2016-05-23 16:27:55 +0000] [16067]
[INFO] Booting worker with pid: 16067
May 23 16:27:55 ybendana-linux airflow: [2016-05-23 16:27:55 +0000] [16069]
[INFO] Booting worker with pid: 16069
May 23 16:27:55 ybendana-linux airflow: [2016-05-23 16:27:55 +0000] [16070]
[INFO] Booting worker with pid: 16070
May 23 16:27:55 ybendana-linux airflow: [2016-05-23 16:27:55 +0000] [16071]
[INFO] Booting worker with pid: 16071
May 23 16:27:55 ybendana-linux airflow: [2016-05-23 16:27:55,876]
{__init__.py:36} INFO - Using executor LocalExecutor
May 23 16:27:55 ybendana-linux airflow: [2016-05-23 16:27:55,950]
{__init__.py:36} INFO - Using executor LocalExecutor
May 23 16:27:55 ybendana-linux airflow: [2016-05-23 16:27:55,972]
{__init__.py:36} INFO - Using executor LocalExecutor
May 23 16:27:55 ybendana-linux airflow: [2016-05-23 16:27:55,997]
{__init__.py:36} INFO - Using executor LocalExecutor
May 23 16:27:57 ybendana-linux airflow: [2016-05-23 16:27:57,885]
{models.py:154} INFO - Filling up the DagBag from /opt/airflow/dags
May 23 16:27:57 ybendana-linux airflow: [2016-05-23 16:27:57,951]
{models.py:154} INFO - Filling up the DagBag from /opt/airflow/dags
May 23 16:27:57 ybendana-linux airflow: [2016-05-23 16:27:57,983]
{models.py:154} INFO - Filling up the DagBag from /opt/airflow/dags
May 23 16:27:58 ybendana-linux airflow: [2016-05-23 16:27:58,014]
{models.py:154} INFO - Filling up the DagBag from /opt/airflow/dags
May 23 16:29:20 ybendana-linux systemd: airflow-webserver.service start
operation timed out. Terminating.
May 23 16:29:20 ybendana-linux airflow: [2016-05-23 16:29:20 +0000] [16070]
[INFO] Worker exiting (pid: 16070)
May 23 16:29:20 ybendana-linux airflow: [2016-05-23 16:29:20 +0000] [15960]
[INFO] Handling signal: term
May 23 16:29:20 ybendana-linux airflow: [2016-05-23 16:29:20 +0000] [16071]
[INFO] Worker exiting (pid: 16071)
May 23 16:29:20 ybendana-linux airflow: [2016-05-23 16:29:20 +0000] [16069]
[INFO] Worker exiting (pid: 16069)
May 23 16:29:20 ybendana-linux airflow: [2016-05-23 16:29:20 +0000] [16067]
[INFO] Worker exiting (pid: 16067)
May 23 16:29:21 ybendana-linux airflow: [2016-05-23 16:29:21 +0000] [15960]
[INFO] Shutting down: Master
May 23 16:29:21 ybendana-linux systemd: Failed to start Airflow webserver
daemon.
May 23 16:29:21 ybendana-linux systemd: Unit airflow-webserver.service entered
failed state.
May 23 16:29:21 ybendana-linux systemd: airflow-webserver.service failed.
{code}
was:
I just upgraded to 1.7.1.2 from 1.6 and I'm having a problem starting the
webserver using the systemd script. This used to work fine. The issue is that
it starts and then just hangs, no error is reported and it finally times out
after about a minute. I tried starting it from the command line and it works
fine without timing out. I also ran it in daemon mode with -D and again it
seems to be fine. Any thoughts on how to debug this?
Here's the log output:
May 23 16:27:50 ybendana-linux systemd: Starting Airflow webserver daemon...
May 23 16:27:51 ybendana-linux airflow: [2016-05-23 16:27:51,444]
{__init__.py:36} INFO - Using executor LocalExecutor
May 23 16:27:53 ybendana-linux airflow: ____________ _____________
May 23 16:27:53 ybendana-linux airflow: ____ |__( )_________ __/__
/________ __
May 23 16:27:53 ybendana-linux airflow: ____ /| |_ /__ ___/_ /_ __ /_ __
\_ | /| / /
May 23 16:27:53 ybendana-linux airflow: ___ ___ | / _ / _ __/ _ / / /_/
/_ |/ |/ /
May 23 16:27:53 ybendana-linux airflow: _/_/ |_/_/ /_/ /_/ /_/
\____/____/|__/
May 23 16:27:53 ybendana-linux airflow: [2016-05-23 16:27:53,446]
{models.py:154} INFO - Filling up the DagBag from /opt/airflow/dags
May 23 16:27:55 ybendana-linux airflow: [2016-05-23 16:27:55 +0000] [15960]
[INFO] Starting gunicorn 19.3.0
May 23 16:27:55 ybendana-linux airflow: [2016-05-23 16:27:55 +0000] [15960]
[INFO] Listening at: http://0.0.0.0:8080 (15960)
May 23 16:27:55 ybendana-linux airflow: [2016-05-23 16:27:55 +0000] [15960]
[INFO] Using worker: sync
May 23 16:27:55 ybendana-linux airflow: [2016-05-23 16:27:55 +0000] [16067]
[INFO] Booting worker with pid: 16067
May 23 16:27:55 ybendana-linux airflow: [2016-05-23 16:27:55 +0000] [16069]
[INFO] Booting worker with pid: 16069
May 23 16:27:55 ybendana-linux airflow: [2016-05-23 16:27:55 +0000] [16070]
[INFO] Booting worker with pid: 16070
May 23 16:27:55 ybendana-linux airflow: [2016-05-23 16:27:55 +0000] [16071]
[INFO] Booting worker with pid: 16071
May 23 16:27:55 ybendana-linux airflow: [2016-05-23 16:27:55,876]
{__init__.py:36} INFO - Using executor LocalExecutor
May 23 16:27:55 ybendana-linux airflow: [2016-05-23 16:27:55,950]
{__init__.py:36} INFO - Using executor LocalExecutor
May 23 16:27:55 ybendana-linux airflow: [2016-05-23 16:27:55,972]
{__init__.py:36} INFO - Using executor LocalExecutor
May 23 16:27:55 ybendana-linux airflow: [2016-05-23 16:27:55,997]
{__init__.py:36} INFO - Using executor LocalExecutor
May 23 16:27:57 ybendana-linux airflow: [2016-05-23 16:27:57,885]
{models.py:154} INFO - Filling up the DagBag from /opt/airflow/dags
May 23 16:27:57 ybendana-linux airflow: [2016-05-23 16:27:57,951]
{models.py:154} INFO - Filling up the DagBag from /opt/airflow/dags
May 23 16:27:57 ybendana-linux airflow: [2016-05-23 16:27:57,983]
{models.py:154} INFO - Filling up the DagBag from /opt/airflow/dags
May 23 16:27:58 ybendana-linux airflow: [2016-05-23 16:27:58,014]
{models.py:154} INFO - Filling up the DagBag from /opt/airflow/dags
May 23 16:29:20 ybendana-linux systemd: airflow-webserver.service start
operation timed out. Terminating.
May 23 16:29:20 ybendana-linux airflow: [2016-05-23 16:29:20 +0000] [16070]
[INFO] Worker exiting (pid: 16070)
May 23 16:29:20 ybendana-linux airflow: [2016-05-23 16:29:20 +0000] [15960]
[INFO] Handling signal: term
May 23 16:29:20 ybendana-linux airflow: [2016-05-23 16:29:20 +0000] [16071]
[INFO] Worker exiting (pid: 16071)
May 23 16:29:20 ybendana-linux airflow: [2016-05-23 16:29:20 +0000] [16069]
[INFO] Worker exiting (pid: 16069)
May 23 16:29:20 ybendana-linux airflow: [2016-05-23 16:29:20 +0000] [16067]
[INFO] Worker exiting (pid: 16067)
May 23 16:29:21 ybendana-linux airflow: [2016-05-23 16:29:21 +0000] [15960]
[INFO] Shutting down: Master
May 23 16:29:21 ybendana-linux systemd: Failed to start Airflow webserver
daemon.
May 23 16:29:21 ybendana-linux systemd: Unit airflow-webserver.service entered
failed state.
May 23 16:29:21 ybendana-linux systemd: airflow-webserver.service failed.
> Unable to start webserver using systemd script
> ----------------------------------------------
>
> Key: AIRFLOW-166
> URL: https://issues.apache.org/jira/browse/AIRFLOW-166
> Project: Apache Airflow
> Issue Type: Bug
> Affects Versions: Airflow 1.7.1
> Environment: CentOS 7
> Reporter: Yuri Bendana
>
> I just upgraded to 1.7.1.2 from 1.6 and I'm having a problem starting the
> webserver using the systemd script. This used to work fine. The issue is
> that it starts and then just hangs, no error is reported and it finally times
> out after about a minute. I tried starting it from the command line and it
> works fine without timing out. I also ran it in daemon mode with -D and
> again it seems to be fine. Any thoughts on how to debug this?
> Here's the log output:
> {code}
> May 23 16:27:50 ybendana-linux systemd: Starting Airflow webserver daemon...
> May 23 16:27:51 ybendana-linux airflow: [2016-05-23 16:27:51,444]
> {__init__.py:36} INFO - Using executor LocalExecutor
> May 23 16:27:53 ybendana-linux airflow: ____________ _____________
> May 23 16:27:53 ybendana-linux airflow: ____ |__( )_________ __/__
> /________ __
> May 23 16:27:53 ybendana-linux airflow: ____ /| |_ /__ ___/_ /_ __ /_
> __ \_ | /| / /
> May 23 16:27:53 ybendana-linux airflow: ___ ___ | / _ / _ __/ _ / /
> /_/ /_ |/ |/ /
> May 23 16:27:53 ybendana-linux airflow: _/_/ |_/_/ /_/ /_/ /_/
> \____/____/|__/
> May 23 16:27:53 ybendana-linux airflow: [2016-05-23 16:27:53,446]
> {models.py:154} INFO - Filling up the DagBag from /opt/airflow/dags
> May 23 16:27:55 ybendana-linux airflow: [2016-05-23 16:27:55 +0000] [15960]
> [INFO] Starting gunicorn 19.3.0
> May 23 16:27:55 ybendana-linux airflow: [2016-05-23 16:27:55 +0000] [15960]
> [INFO] Listening at: http://0.0.0.0:8080 (15960)
> May 23 16:27:55 ybendana-linux airflow: [2016-05-23 16:27:55 +0000] [15960]
> [INFO] Using worker: sync
> May 23 16:27:55 ybendana-linux airflow: [2016-05-23 16:27:55 +0000] [16067]
> [INFO] Booting worker with pid: 16067
> May 23 16:27:55 ybendana-linux airflow: [2016-05-23 16:27:55 +0000] [16069]
> [INFO] Booting worker with pid: 16069
> May 23 16:27:55 ybendana-linux airflow: [2016-05-23 16:27:55 +0000] [16070]
> [INFO] Booting worker with pid: 16070
> May 23 16:27:55 ybendana-linux airflow: [2016-05-23 16:27:55 +0000] [16071]
> [INFO] Booting worker with pid: 16071
> May 23 16:27:55 ybendana-linux airflow: [2016-05-23 16:27:55,876]
> {__init__.py:36} INFO - Using executor LocalExecutor
> May 23 16:27:55 ybendana-linux airflow: [2016-05-23 16:27:55,950]
> {__init__.py:36} INFO - Using executor LocalExecutor
> May 23 16:27:55 ybendana-linux airflow: [2016-05-23 16:27:55,972]
> {__init__.py:36} INFO - Using executor LocalExecutor
> May 23 16:27:55 ybendana-linux airflow: [2016-05-23 16:27:55,997]
> {__init__.py:36} INFO - Using executor LocalExecutor
> May 23 16:27:57 ybendana-linux airflow: [2016-05-23 16:27:57,885]
> {models.py:154} INFO - Filling up the DagBag from /opt/airflow/dags
> May 23 16:27:57 ybendana-linux airflow: [2016-05-23 16:27:57,951]
> {models.py:154} INFO - Filling up the DagBag from /opt/airflow/dags
> May 23 16:27:57 ybendana-linux airflow: [2016-05-23 16:27:57,983]
> {models.py:154} INFO - Filling up the DagBag from /opt/airflow/dags
> May 23 16:27:58 ybendana-linux airflow: [2016-05-23 16:27:58,014]
> {models.py:154} INFO - Filling up the DagBag from /opt/airflow/dags
> May 23 16:29:20 ybendana-linux systemd: airflow-webserver.service start
> operation timed out. Terminating.
> May 23 16:29:20 ybendana-linux airflow: [2016-05-23 16:29:20 +0000] [16070]
> [INFO] Worker exiting (pid: 16070)
> May 23 16:29:20 ybendana-linux airflow: [2016-05-23 16:29:20 +0000] [15960]
> [INFO] Handling signal: term
> May 23 16:29:20 ybendana-linux airflow: [2016-05-23 16:29:20 +0000] [16071]
> [INFO] Worker exiting (pid: 16071)
> May 23 16:29:20 ybendana-linux airflow: [2016-05-23 16:29:20 +0000] [16069]
> [INFO] Worker exiting (pid: 16069)
> May 23 16:29:20 ybendana-linux airflow: [2016-05-23 16:29:20 +0000] [16067]
> [INFO] Worker exiting (pid: 16067)
> May 23 16:29:21 ybendana-linux airflow: [2016-05-23 16:29:21 +0000] [15960]
> [INFO] Shutting down: Master
> May 23 16:29:21 ybendana-linux systemd: Failed to start Airflow webserver
> daemon.
> May 23 16:29:21 ybendana-linux systemd: Unit airflow-webserver.service
> entered failed state.
> May 23 16:29:21 ybendana-linux systemd: airflow-webserver.service failed.
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
