[jira] [Updated] (AIRFLOW-2615) Webserver parent not using cached app
[ https://issues.apache.org/jira/browse/AIRFLOW-2615?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bolke de Bruin updated AIRFLOW-2615: Affects Version/s: 1.10.0 > Webserver parent not using cached app > - > > Key: AIRFLOW-2615 > URL: https://issues.apache.org/jira/browse/AIRFLOW-2615 > Project: Apache Airflow > Issue Type: Bug >Affects Versions: 1.10.0 >Reporter: Kevin Yang >Assignee: Kevin Yang >Priority: Major > Fix For: 2.0.0 > > > From what I can tell, the app cached > [here|https://github.com/apache/incubator-airflow/blob/master/airflow/bin/cli.py#L790] > attempt to cache the app for later use-likely to be for the expensive > DagBag() creation. Before I dive into the webserver parsing everything in one > process problem, I was hoping this cached app would save me sometime. However > it seems to me that every subprocess spun up by gunicorn is trying to create > the DagBag() right after they've been created--make sense to me since we > didn't share the cached app to the subprocess( doubt we can). If what I > observed is true, why do we cache the app at all in the parent process? -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (AIRFLOW-2615) Webserver parent not using cached app
[ https://issues.apache.org/jira/browse/AIRFLOW-2615?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bolke de Bruin updated AIRFLOW-2615: Fix Version/s: (was: 1.10.0) 2.0.0 > Webserver parent not using cached app > - > > Key: AIRFLOW-2615 > URL: https://issues.apache.org/jira/browse/AIRFLOW-2615 > Project: Apache Airflow > Issue Type: Bug >Reporter: Kevin Yang >Assignee: Kevin Yang >Priority: Major > Fix For: 2.0.0 > > > From what I can tell, the app cached > [here|https://github.com/apache/incubator-airflow/blob/master/airflow/bin/cli.py#L790] > attempt to cache the app for later use-likely to be for the expensive > DagBag() creation. Before I dive into the webserver parsing everything in one > process problem, I was hoping this cached app would save me sometime. However > it seems to me that every subprocess spun up by gunicorn is trying to create > the DagBag() right after they've been created--make sense to me since we > didn't share the cached app to the subprocess( doubt we can). If what I > observed is true, why do we cache the app at all in the parent process? -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (AIRFLOW-2615) Webserver parent not using cached app
[ https://issues.apache.org/jira/browse/AIRFLOW-2615?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kevin Yang updated AIRFLOW-2615: Summary: Webserver parent not using cached app (was: Webserver not using cached app) > Webserver parent not using cached app > - > > Key: AIRFLOW-2615 > URL: https://issues.apache.org/jira/browse/AIRFLOW-2615 > Project: Apache Airflow > Issue Type: Bug >Reporter: Kevin Yang >Priority: Major > > From what I can tell, the app cached > [here|https://github.com/apache/incubator-airflow/blob/master/airflow/bin/cli.py#L790] > attempt to cache the app for later use-likely to be for the expensive > DagBag() creation. Before I dive into the webserver parsing everything in one > process problem, I was hoping this cached app would save me sometime. However > it seems to me that every subprocess spun up by gunicorn is trying to create > the DagBag() right after they've been created--make sense to me since we > didn't share the cached app to the subprocess( doubt we can). If what I > observed is true, why do we cache the app at all? -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (AIRFLOW-2615) Webserver parent not using cached app
[ https://issues.apache.org/jira/browse/AIRFLOW-2615?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kevin Yang updated AIRFLOW-2615: Description: From what I can tell, the app cached [here|https://github.com/apache/incubator-airflow/blob/master/airflow/bin/cli.py#L790] attempt to cache the app for later use-likely to be for the expensive DagBag() creation. Before I dive into the webserver parsing everything in one process problem, I was hoping this cached app would save me sometime. However it seems to me that every subprocess spun up by gunicorn is trying to create the DagBag() right after they've been created--make sense to me since we didn't share the cached app to the subprocess( doubt we can). If what I observed is true, why do we cache the app at all in the parent process? (was: From what I can tell, the app cached [here|https://github.com/apache/incubator-airflow/blob/master/airflow/bin/cli.py#L790] attempt to cache the app for later use-likely to be for the expensive DagBag() creation. Before I dive into the webserver parsing everything in one process problem, I was hoping this cached app would save me sometime. However it seems to me that every subprocess spun up by gunicorn is trying to create the DagBag() right after they've been created--make sense to me since we didn't share the cached app to the subprocess( doubt we can). If what I observed is true, why do we cache the app at all?) > Webserver parent not using cached app > - > > Key: AIRFLOW-2615 > URL: https://issues.apache.org/jira/browse/AIRFLOW-2615 > Project: Apache Airflow > Issue Type: Bug >Reporter: Kevin Yang >Priority: Major > > From what I can tell, the app cached > [here|https://github.com/apache/incubator-airflow/blob/master/airflow/bin/cli.py#L790] > attempt to cache the app for later use-likely to be for the expensive > DagBag() creation. Before I dive into the webserver parsing everything in one > process problem, I was hoping this cached app would save me sometime. However > it seems to me that every subprocess spun up by gunicorn is trying to create > the DagBag() right after they've been created--make sense to me since we > didn't share the cached app to the subprocess( doubt we can). If what I > observed is true, why do we cache the app at all in the parent process? -- This message was sent by Atlassian JIRA (v7.6.3#76005)
