[jira] [Created] (AIRFLOW-2190) base_url with a subpath generates TypeError

John Arnold created AIRFLOW-2190: Summary: base_url with a subpath generates TypeError Key: AIRFLOW-2190 URL: https://issues.apache.org/jira/browse/AIRFLOW-2190 Project: Apache Airflow Issue

[jira] [Commented] (AIRFLOW-2190) base_url with a subpath generates TypeError

[ https://issues.apache.org/jira/browse/AIRFLOW-2190?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16390180#comment-16390180 ] John Arnold commented on AIRFLOW-2190: -- This code is bugged: [https://github.com/ap

[jira] [Commented] (AIRFLOW-2190) base_url with a subpath generates TypeError

[ https://issues.apache.org/jira/browse/AIRFLOW-2190?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16390225#comment-16390225 ] John Arnold commented on AIRFLOW-2190: -- I got the subpath working... the docs are w

[jira] [Comment Edited] (AIRFLOW-2190) base_url with a subpath generates TypeError

[ https://issues.apache.org/jira/browse/AIRFLOW-2190?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16390225#comment-16390225 ] John Arnold edited comment on AIRFLOW-2190 at 3/7/18 9:04 PM:

[jira] [Commented] (AIRFLOW-2190) base_url with a subpath generates TypeError

[ https://issues.apache.org/jira/browse/AIRFLOW-2190?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16390244#comment-16390244 ] John Arnold commented on AIRFLOW-2190: -- It seems like some of the redirects don't w

[jira] [Created] (AIRFLOW-2191) Scheduler logging is too verbose on INFO

John Arnold created AIRFLOW-2191: Summary: Scheduler logging is too verbose on INFO Key: AIRFLOW-2191 URL: https://issues.apache.org/jira/browse/AIRFLOW-2191 Project: Apache Airflow Issue Typ

[jira] [Commented] (AIRFLOW-2191) Scheduler logging is too verbose on INFO

[ https://issues.apache.org/jira/browse/AIRFLOW-2191?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16390569#comment-16390569 ] John Arnold commented on AIRFLOW-2191: -- https://github.com/apache/incubator-airflow

[jira] [Commented] (AIRFLOW-1235) Odd behaviour when all gunicorn workers die

[ https://issues.apache.org/jira/browse/AIRFLOW-1235?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16392021#comment-16392021 ] John Arnold commented on AIRFLOW-1235: -- I have this same problem – workers fail due

[jira] [Created] (AIRFLOW-2201) Scheduler processes dags every second

John Arnold created AIRFLOW-2201: Summary: Scheduler processes dags every second Key: AIRFLOW-2201 URL: https://issues.apache.org/jira/browse/AIRFLOW-2201 Project: Apache Airflow Issue Type:

[jira] [Updated] (AIRFLOW-2201) Scheduler processes dags every second

[ https://issues.apache.org/jira/browse/AIRFLOW-2201?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] John Arnold updated AIRFLOW-2201: - Priority: Critical (was: Major) > Scheduler processes dags every second > -

[jira] [Closed] (AIRFLOW-2191) Scheduler logging is too verbose on INFO

[ https://issues.apache.org/jira/browse/AIRFLOW-2191?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] John Arnold closed AIRFLOW-2191. > Scheduler logging is too verbose on INFO > > >

[jira] [Commented] (AIRFLOW-1235) Odd behaviour when all gunicorn workers die

[ https://issues.apache.org/jira/browse/AIRFLOW-1235?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16397679#comment-16397679 ] John Arnold commented on AIRFLOW-1235: -- Does anyone know how to fix this so that ei

[jira] [Created] (AIRFLOW-2215) celery task launches subprocess without environment vars

John Arnold created AIRFLOW-2215: Summary: celery task launches subprocess without environment vars Key: AIRFLOW-2215 URL: https://issues.apache.org/jira/browse/AIRFLOW-2215 Project: Apache Airflow

[jira] [Updated] (AIRFLOW-2215) celery task launches subprocess without environment vars

[ https://issues.apache.org/jira/browse/AIRFLOW-2215?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] John Arnold updated AIRFLOW-2215: - Description: The celery task that is ran by CeleryExecutor does not preserve the environment va

[jira] [Commented] (AIRFLOW-1235) Odd behaviour when all gunicorn workers die

[ https://issues.apache.org/jira/browse/AIRFLOW-1235?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16405394#comment-16405394 ] John Arnold commented on AIRFLOW-1235: -- external check and restart is not a great u

[jira] [Commented] (AIRFLOW-1235) Odd behaviour when all gunicorn workers die

[ https://issues.apache.org/jira/browse/AIRFLOW-1235?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16405547#comment-16405547 ] John Arnold commented on AIRFLOW-1235: -- For reference, here's what it looks linke i

[jira] [Created] (AIRFLOW-2244) Exception importing Azure (wasb) log handler

John Arnold created AIRFLOW-2244: Summary: Exception importing Azure (wasb) log handler Key: AIRFLOW-2244 URL: https://issues.apache.org/jira/browse/AIRFLOW-2244 Project: Apache Airflow Issue

[jira] [Commented] (AIRFLOW-1775) Remote file handler for logging

[ https://issues.apache.org/jira/browse/AIRFLOW-1775?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16410486#comment-16410486 ] John Arnold commented on AIRFLOW-1775: -- in airflow/models.py: if 'mysql' in settin

[jira] [Issue Comment Deleted] (AIRFLOW-1775) Remote file handler for logging

[ https://issues.apache.org/jira/browse/AIRFLOW-1775?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] John Arnold updated AIRFLOW-1775: - Comment: was deleted (was: in airflow/models.py: if 'mysql' in settings.SQL_ALCHEMY_CONN: Long

[jira] [Commented] (AIRFLOW-2244) Exception importing Azure (wasb) log handler

[ https://issues.apache.org/jira/browse/AIRFLOW-2244?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16410636#comment-16410636 ] John Arnold commented on AIRFLOW-2244: -- in airflow/models.py: if 'mysql' in settin

[jira] [Commented] (AIRFLOW-2244) Exception importing Azure (wasb) log handler

[ https://issues.apache.org/jira/browse/AIRFLOW-2244?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16412325#comment-16412325 ] John Arnold commented on AIRFLOW-2244: -- It doesn't appear that this LongText variab

[jira] [Created] (AIRFLOW-2275) Scheduler errors for running tasks

John Arnold created AIRFLOW-2275: Summary: Scheduler errors for running tasks Key: AIRFLOW-2275 URL: https://issues.apache.org/jira/browse/AIRFLOW-2275 Project: Apache Airflow Issue Type: Bug

[jira] [Closed] (AIRFLOW-2244) Exception importing Azure (wasb) log handler

[ https://issues.apache.org/jira/browse/AIRFLOW-2244?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] John Arnold closed AIRFLOW-2244. > Exception importing Azure (wasb) log handler > > >

[jira] [Commented] (AIRFLOW-2162) Run DAG as user other than airflow does NOT have access to AIRFLOW_ environment variables

[ https://issues.apache.org/jira/browse/AIRFLOW-2162?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16423281#comment-16423281 ] John Arnold commented on AIRFLOW-2162: -- I've been submitting PRs to make sure the e

[jira] [Commented] (AIRFLOW-2162) Run DAG as user other than airflow does NOT have access to AIRFLOW_ environment variables

[ https://issues.apache.org/jira/browse/AIRFLOW-2162?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16424624#comment-16424624 ] John Arnold commented on AIRFLOW-2162: -- Here, I PR'd it for you.  https://github.co

[jira] [Assigned] (AIRFLOW-2162) Run DAG as user other than airflow does NOT have access to AIRFLOW_ environment variables

[ https://issues.apache.org/jira/browse/AIRFLOW-2162?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] John Arnold reassigned AIRFLOW-2162: Assignee: John Arnold > Run DAG as user other than airflow does NOT have access to AIRFLO

[jira] [Commented] (AIRFLOW-2278) Add possibility to edit DAGs in webapp

[ https://issues.apache.org/jira/browse/AIRFLOW-2278?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16424627#comment-16424627 ] John Arnold commented on AIRFLOW-2278: -- How would this work when using remote execu

[jira] [Commented] (AIRFLOW-2156) Parallelize Celery Executor

[ https://issues.apache.org/jira/browse/AIRFLOW-2156?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16431308#comment-16431308 ] John Arnold commented on AIRFLOW-2156: -- This seems like a pretty easy candidate for

[jira] [Commented] (AIRFLOW-687) Gracefully halt workers through CLI

[ https://issues.apache.org/jira/browse/AIRFLOW-687?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16431330#comment-16431330 ] John Arnold commented on AIRFLOW-687: - commenting on old Jira for posterity – my solu

[jira] [Created] (AIRFLOW-2307) Airflow should track Celery task ID

John Arnold created AIRFLOW-2307: Summary: Airflow should track Celery task ID Key: AIRFLOW-2307 URL: https://issues.apache.org/jira/browse/AIRFLOW-2307 Project: Apache Airflow Issue Type: Im

[jira] [Created] (AIRFLOW-2309) TypeError exception when calculating duration

John Arnold created AIRFLOW-2309: Summary: TypeError exception when calculating duration Key: AIRFLOW-2309 URL: https://issues.apache.org/jira/browse/AIRFLOW-2309 Project: Apache Airflow Issu

[jira] [Created] (AIRFLOW-2338) Airflow requires Hive config

John Arnold created AIRFLOW-2338: Summary: Airflow requires Hive config Key: AIRFLOW-2338 URL: https://issues.apache.org/jira/browse/AIRFLOW-2338 Project: Apache Airflow Issue Type: Bug

[jira] [Created] (AIRFLOW-2339) setup.py version doesn't use git info

John Arnold created AIRFLOW-2339: Summary: setup.py version doesn't use git info Key: AIRFLOW-2339 URL: https://issues.apache.org/jira/browse/AIRFLOW-2339 Project: Apache Airflow Issue Type:

[jira] [Created] (AIRFLOW-2340) SQLalchemy pessimistic connection handling not working

John Arnold created AIRFLOW-2340: Summary: SQLalchemy pessimistic connection handling not working Key: AIRFLOW-2340 URL: https://issues.apache.org/jira/browse/AIRFLOW-2340 Project: Apache Airflow

[jira] [Updated] (AIRFLOW-2340) SQLalchemy pessimistic connection handling not working

[ https://issues.apache.org/jira/browse/AIRFLOW-2340?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] John Arnold updated AIRFLOW-2340: - Attachment: webserver.txt > SQLalchemy pessimistic connection handling not working > ---

[jira] [Commented] (AIRFLOW-2340) SQLalchemy pessimistic connection handling not working

[ https://issues.apache.org/jira/browse/AIRFLOW-2340?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16443161#comment-16443161 ] John Arnold commented on AIRFLOW-2340: -- Now getting similar errors for the webserve

[jira] [Commented] (AIRFLOW-2319) Table "dag_run" has (bad) second index on (dag_id, execution_date)

[ https://issues.apache.org/jira/browse/AIRFLOW-2319?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16444653#comment-16444653 ] John Arnold commented on AIRFLOW-2319: -- IMO, the index is probably needed for perfo

[jira] [Commented] (AIRFLOW-2319) Table "dag_run" has (bad) second index on (dag_id, execution_date)

[ https://issues.apache.org/jira/browse/AIRFLOW-2319?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16444711#comment-16444711 ] John Arnold commented on AIRFLOW-2319: -- Also, if you truly need "duplicate" dag run

[jira] [Comment Edited] (AIRFLOW-2319) Table "dag_run" has (bad) second index on (dag_id, execution_date)

[ https://issues.apache.org/jira/browse/AIRFLOW-2319?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16444711#comment-16444711 ] John Arnold edited comment on AIRFLOW-2319 at 4/19/18 8:14 PM: ---

[jira] [Comment Edited] (AIRFLOW-2319) Table "dag_run" has (bad) second index on (dag_id, execution_date)

[ https://issues.apache.org/jira/browse/AIRFLOW-2319?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16444711#comment-16444711 ] John Arnold edited comment on AIRFLOW-2319 at 4/19/18 8:14 PM: ---

[jira] [Created] (AIRFLOW-2367) High POSTGRES DB CPU utilization

John Arnold created AIRFLOW-2367: Summary: High POSTGRES DB CPU utilization Key: AIRFLOW-2367 URL: https://issues.apache.org/jira/browse/AIRFLOW-2367 Project: Apache Airflow Issue Type: Bug

[jira] [Updated] (AIRFLOW-2366) New Webserver UI with Role-Based Access Control not working

[ https://issues.apache.org/jira/browse/AIRFLOW-2366?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] John Arnold updated AIRFLOW-2366: - Attachment: cpu.png > New Webserver UI with Role-Based Access Control not working >

[jira] [Updated] (AIRFLOW-2367) High POSTGRES DB CPU utilization

[ https://issues.apache.org/jira/browse/AIRFLOW-2367?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] John Arnold updated AIRFLOW-2367: - Attachment: cpu.png > High POSTGRES DB CPU utilization > > >

[jira] [Updated] (AIRFLOW-2366) New Webserver UI with Role-Based Access Control not working

[ https://issues.apache.org/jira/browse/AIRFLOW-2366?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] John Arnold updated AIRFLOW-2366: - Attachment: (was: cpu.png) > New Webserver UI with Role-Based Access Control not working > -

[jira] [Updated] (AIRFLOW-2367) High POSTGRES DB CPU utilization

[ https://issues.apache.org/jira/browse/AIRFLOW-2367?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] John Arnold updated AIRFLOW-2367: - Attachment: postgres.png > High POSTGRES DB CPU utilization > >

[jira] [Commented] (AIRFLOW-2367) High POSTGRES DB CPU utilization

[ https://issues.apache.org/jira/browse/AIRFLOW-2367?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16448953#comment-16448953 ] John Arnold commented on AIRFLOW-2367: -- [~bolke]  Any suggestions on what metrics o

[jira] [Comment Edited] (AIRFLOW-2367) High POSTGRES DB CPU utilization

[ https://issues.apache.org/jira/browse/AIRFLOW-2367?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16448953#comment-16448953 ] John Arnold edited comment on AIRFLOW-2367 at 4/23/18 10:18 PM: --

[jira] [Commented] (AIRFLOW-2367) High POSTGRES DB CPU utilization

[ https://issues.apache.org/jira/browse/AIRFLOW-2367?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16448969#comment-16448969 ] John Arnold commented on AIRFLOW-2367: -- This query is taking the most time: SELECT

[jira] [Commented] (AIRFLOW-2367) High POSTGRES DB CPU utilization

[ https://issues.apache.org/jira/browse/AIRFLOW-2367?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16448974#comment-16448974 ] John Arnold commented on AIRFLOW-2367: -- This insert is taking the second-most time

[jira] [Comment Edited] (AIRFLOW-2367) High POSTGRES DB CPU utilization

[ https://issues.apache.org/jira/browse/AIRFLOW-2367?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16448969#comment-16448969 ] John Arnold edited comment on AIRFLOW-2367 at 4/23/18 10:30 PM: --

[jira] [Comment Edited] (AIRFLOW-2367) High POSTGRES DB CPU utilization

[ https://issues.apache.org/jira/browse/AIRFLOW-2367?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16448974#comment-16448974 ] John Arnold edited comment on AIRFLOW-2367 at 4/23/18 10:31 PM: --

[jira] [Commented] (AIRFLOW-2367) High POSTGRES DB CPU utilization

[ https://issues.apache.org/jira/browse/AIRFLOW-2367?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16448978#comment-16448978 ] John Arnold commented on AIRFLOW-2367: -- I believe the "top talker" query is from th

[jira] [Comment Edited] (AIRFLOW-2367) High POSTGRES DB CPU utilization

[ https://issues.apache.org/jira/browse/AIRFLOW-2367?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16448978#comment-16448978 ] John Arnold edited comment on AIRFLOW-2367 at 4/23/18 10:34 PM: --

[jira] [Commented] (AIRFLOW-2367) High POSTGRES DB CPU utilization

[ https://issues.apache.org/jira/browse/AIRFLOW-2367?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16448980#comment-16448980 ] John Arnold commented on AIRFLOW-2367: -- Manual repro of the top query only takes 25

[jira] [Commented] (AIRFLOW-2393) UI tree view struggles with large dags (60 tasks)x25 dag histories

[ https://issues.apache.org/jira/browse/AIRFLOW-2393?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16458921#comment-16458921 ] John Arnold commented on AIRFLOW-2393: -- yeah it gets slow for us too.  ~100 paralle

[jira] [Commented] (AIRFLOW-2190) base_url with a subpath generates TypeError

[ https://issues.apache.org/jira/browse/AIRFLOW-2190?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16538977#comment-16538977 ] John Arnold commented on AIRFLOW-2190: -- [~elgalu] Glad you got it working.  I thin