[jira] [Created] (AIRFLOW-6468) fix install on python 3.7

2020-01-05 Thread t oo (Jira)
t oo created AIRFLOW-6468: - Summary: fix install on python 3.7 Key: AIRFLOW-6468 URL: https://issues.apache.org/jira/browse/AIRFLOW-6468 Project: Apache Airflow Issue Type: New Feature Comp

[jira] [Closed] (AIRFLOW-6455) refactor wildcard SQLs

2020-01-04 Thread t oo (Jira)
[ https://issues.apache.org/jira/browse/AIRFLOW-6455?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] t oo closed AIRFLOW-6455. - Resolution: Duplicate > refactor wildcard SQLs > -- > > Key: AIRFLOW-6455 >

[jira] [Updated] (AIRFLOW-6415) get_task_instances() - don't look at 30 days of exec_dates

2020-01-04 Thread t oo (Jira)
[ https://issues.apache.org/jira/browse/AIRFLOW-6415?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] t oo updated AIRFLOW-6415: -- Priority: Minor (was: Major) > get_task_instances() - don't look at 30 days of exec_dates >

[jira] [Updated] (AIRFLOW-6389) add config for 'allow_multi_scheduler_instances' default True

2020-01-04 Thread t oo (Jira)
[ https://issues.apache.org/jira/browse/AIRFLOW-6389?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] t oo updated AIRFLOW-6389: -- Priority: Minor (was: Major) > add config for 'allow_multi_scheduler_instances' default True > -

[jira] [Updated] (AIRFLOW-6455) refactor wildcard SQLs

2020-01-04 Thread t oo (Jira)
[ https://issues.apache.org/jira/browse/AIRFLOW-6455?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] t oo updated AIRFLOW-6455: -- Priority: Trivial (was: Major) > refactor wildcard SQLs > -- > > Key: AI

[jira] [Created] (AIRFLOW-6455) refactor wildcard SQLs

2020-01-03 Thread t oo (Jira)
t oo created AIRFLOW-6455: - Summary: refactor wildcard SQLs Key: AIRFLOW-6455 URL: https://issues.apache.org/jira/browse/AIRFLOW-6455 Project: Apache Airflow Issue Type: Improvement Compone

[jira] [Created] (AIRFLOW-6454) add test for time taken by scheduler to run dag of diff num of tasks (2 vs 20 vs 200 vs 2000 vs 20000 simple 1 line print tasks)

2020-01-03 Thread t oo (Jira)
t oo created AIRFLOW-6454: - Summary: add test for time taken by scheduler to run dag of diff num of tasks (2 vs 20 vs 200 vs 2000 vs 2 simple 1 line print tasks) Key: AIRFLOW-6454 URL: https://issues.apache.org/jira/b

[jira] [Created] (AIRFLOW-6453) add test for time taken by dag_processor_manager to parse dag of diff num of tasks (2 vs 20 vs 200 vs 2000 vs 20000 tasks)

2020-01-03 Thread t oo (Jira)
t oo created AIRFLOW-6453: - Summary: add test for time taken by dag_processor_manager to parse dag of diff num of tasks (2 vs 20 vs 200 vs 2000 vs 2 tasks) Key: AIRFLOW-6453 URL: https://issues.apache.org/jira/browse/

[jira] [Created] (AIRFLOW-6452) scheduler_job.py - remove excess sleep/log/duration calls

2020-01-03 Thread t oo (Jira)
t oo created AIRFLOW-6452: - Summary: scheduler_job.py - remove excess sleep/log/duration calls Key: AIRFLOW-6452 URL: https://issues.apache.org/jira/browse/AIRFLOW-6452 Project: Apache Airflow Issue

[jira] [Created] (AIRFLOW-6451) self._print_stat() in dag_processing.py should be skippable by config option

2020-01-03 Thread t oo (Jira)
t oo created AIRFLOW-6451: - Summary: self._print_stat() in dag_processing.py should be skippable by config option Key: AIRFLOW-6451 URL: https://issues.apache.org/jira/browse/AIRFLOW-6451 Project: Apache Airf

[jira] [Created] (AIRFLOW-6450) Move conf.get calls to static classes, assign repeated utcnow() calls to variable, reduce Stats.() / log.debug overhead

2020-01-03 Thread t oo (Jira)
t oo created AIRFLOW-6450: - Summary: Move conf.get calls to static classes, assign repeated utcnow() calls to variable, reduce Stats.() / log.debug overhead Key: AIRFLOW-6450 URL: https://issues.apache.org/jira/browse/AIR

[jira] [Work started] (AIRFLOW-6437) sql filters - remove in (NULL)

2020-01-03 Thread t oo (Jira)
[ https://issues.apache.org/jira/browse/AIRFLOW-6437?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Work on AIRFLOW-6437 started by t oo. - > sql filters - remove in (NULL) > -- > > Key: A

[jira] [Updated] (AIRFLOW-6443) Pool name - increase length > 50, make cli give error if too large

2020-01-03 Thread t oo (Jira)
[ https://issues.apache.org/jira/browse/AIRFLOW-6443?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] t oo updated AIRFLOW-6443: -- Description: create some pool names (using cli) with 70 or 80 character length   1. UI does not allow creat

[jira] [Updated] (AIRFLOW-6443) Pool name - increase length > 50, make cli give error if too large

2020-01-03 Thread t oo (Jira)
[ https://issues.apache.org/jira/browse/AIRFLOW-6443?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] t oo updated AIRFLOW-6443: -- Component/s: (was: scheduler) models database cli > Po

[jira] [Created] (AIRFLOW-6443) Pool name - increase length > 50, make cli give error if too large

2020-01-03 Thread t oo (Jira)
t oo created AIRFLOW-6443: - Summary: Pool name - increase length > 50, make cli give error if too large Key: AIRFLOW-6443 URL: https://issues.apache.org/jira/browse/AIRFLOW-6443 Project: Apache Airflow

[jira] [Assigned] (AIRFLOW-6443) Pool name - increase length > 50, make cli give error if too large

2020-01-03 Thread t oo (Jira)
[ https://issues.apache.org/jira/browse/AIRFLOW-6443?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] t oo reassigned AIRFLOW-6443: - Assignee: (was: Ash Berlin-Taylor) > Pool name - increase length > 50, make cli give error if too

[jira] [Updated] (AIRFLOW-6443) Pool name - increase length > 50, make cli give error if too large

2020-01-03 Thread t oo (Jira)
[ https://issues.apache.org/jira/browse/AIRFLOW-6443?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] t oo updated AIRFLOW-6443: -- Fix Version/s: (was: 1.10.4) > Pool name - increase length > 50, make cli give error if too large > -

[jira] [Created] (AIRFLOW-6437) sql filters - remove in (NULL)

2020-01-03 Thread t oo (Jira)
t oo created AIRFLOW-6437: - Summary: sql filters - remove in (NULL) Key: AIRFLOW-6437 URL: https://issues.apache.org/jira/browse/AIRFLOW-6437 Project: Apache Airflow Issue Type: Improvement

[jira] [Work started] (AIRFLOW-6433) reduce conf.get lookups in scheduler_job.py loops

2020-01-02 Thread t oo (Jira)
[ https://issues.apache.org/jira/browse/AIRFLOW-6433?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Work on AIRFLOW-6433 started by t oo. - > reduce conf.get lookups in scheduler_job.py loops > --

[jira] [Updated] (AIRFLOW-6433) reduce conf.get lookups in scheduler_job.py loops

2020-01-02 Thread t oo (Jira)
[ https://issues.apache.org/jira/browse/AIRFLOW-6433?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] t oo updated AIRFLOW-6433: -- Description: trivial perf benefit available > reduce conf.get lookups in scheduler_job.py loops > ---

[jira] [Created] (AIRFLOW-6433) reduce conf.get lookups in scheduler_job.py loops

2020-01-02 Thread t oo (Jira)
t oo created AIRFLOW-6433: - Summary: reduce conf.get lookups in scheduler_job.py loops Key: AIRFLOW-6433 URL: https://issues.apache.org/jira/browse/AIRFLOW-6433 Project: Apache Airflow Issue Type: Im

[jira] [Commented] (AIRFLOW-4762) Test against Python 3.6, 3.7 and 3.8b1

2020-01-02 Thread t oo (Jira)
[ https://issues.apache.org/jira/browse/AIRFLOW-4762?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17007083#comment-17007083 ] t oo commented on AIRFLOW-4762: --- I think tests are already run on 3.6 and 3.7 but not 3.8

[jira] [Closed] (AIRFLOW-6349) security - api should deny access by default

2020-01-02 Thread t oo (Jira)
[ https://issues.apache.org/jira/browse/AIRFLOW-6349?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] t oo closed AIRFLOW-6349. - Resolution: Duplicate > security - api should deny access by default >

[jira] [Created] (AIRFLOW-6420) ui - show summary of task/dagrun stats across ALL dags

2020-01-01 Thread t oo (Jira)
t oo created AIRFLOW-6420: - Summary: ui - show summary of task/dagrun stats across ALL dags Key: AIRFLOW-6420 URL: https://issues.apache.org/jira/browse/AIRFLOW-6420 Project: Apache Airflow Issue Typ

[jira] [Created] (AIRFLOW-6419) dag_processor_manager/webserver logs should be created under date folder

2020-01-01 Thread t oo (Jira)
t oo created AIRFLOW-6419: - Summary: dag_processor_manager/webserver logs should be created under date folder Key: AIRFLOW-6419 URL: https://issues.apache.org/jira/browse/AIRFLOW-6419 Project: Apache Airflow

[jira] [Work started] (AIRFLOW-4502) new cli command - task_states_for_dag_run

2020-01-01 Thread t oo (Jira)
[ https://issues.apache.org/jira/browse/AIRFLOW-4502?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Work on AIRFLOW-4502 started by t oo. - > new cli command - task_states_for_dag_run > - > >

[jira] [Commented] (AIRFLOW-5523) DAGs never get cleaned up from webservers if DAG file is removed from the scheduler

2020-01-01 Thread t oo (Jira)
[ https://issues.apache.org/jira/browse/AIRFLOW-5523?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17006379#comment-17006379 ] t oo commented on AIRFLOW-5523: --- my thoughts: autodelete dags that are no longer in any d

[jira] [Closed] (AIRFLOW-5355) 1.10.4 upgrade issues - No module named kubernetes (but i'm using localexecutor)

2019-12-31 Thread t oo (Jira)
[ https://issues.apache.org/jira/browse/AIRFLOW-5355?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] t oo closed AIRFLOW-5355. - Resolution: Fixed > 1.10.4 upgrade issues - No module named kubernetes (but i'm using > localexecutor) > -

[jira] [Closed] (AIRFLOW-5406) Allow spark_submit_hook to run without pip install kubernetes/airflow[kubernetes]

2019-12-31 Thread t oo (Jira)
[ https://issues.apache.org/jira/browse/AIRFLOW-5406?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] t oo closed AIRFLOW-5406. - Resolution: Fixed > Allow spark_submit_hook to run without pip install > kubernetes/airflow[kubernetes] >

[jira] [Created] (AIRFLOW-6415) get_task_instances() - don't look at 30 days of exec_dates

2019-12-31 Thread t oo (Jira)
t oo created AIRFLOW-6415: - Summary: get_task_instances() - don't look at 30 days of exec_dates Key: AIRFLOW-6415 URL: https://issues.apache.org/jira/browse/AIRFLOW-6415 Project: Apache Airflow Issue

[jira] [Commented] (AIRFLOW-4464) Fix case-insensitive id columns in mysql

2019-12-30 Thread t oo (Jira)
[ https://issues.apache.org/jira/browse/AIRFLOW-4464?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17005764#comment-17005764 ] t oo commented on AIRFLOW-4464: --- ran into this. https://github.com/apache/airflow/bl

[jira] [Updated] (AIRFLOW-6369) clear command needs a 'conf' option

2019-12-29 Thread t oo (Jira)
[ https://issues.apache.org/jira/browse/AIRFLOW-6369?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] t oo updated AIRFLOW-6369: -- Description: key-value pairs of conf can be passed into trigger_dag command ie --conf '{"ric":"amzn"}' clear

[jira] [Work started] (AIRFLOW-6387) print details of success/skipped task

2019-12-29 Thread t oo (Jira)
[ https://issues.apache.org/jira/browse/AIRFLOW-6387?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Work on AIRFLOW-6387 started by t oo. - > print details of success/skipped task > - > >

[jira] [Updated] (AIRFLOW-6387) print details of success/skipped task

2019-12-29 Thread t oo (Jira)
[ https://issues.apache.org/jira/browse/AIRFLOW-6387?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] t oo updated AIRFLOW-6387: -- Description: print success/skipped (with dag id, start/end times, task id), similar to airflow-6358. Useful

[jira] [Updated] (AIRFLOW-6387) print details of success/skipped task

2019-12-29 Thread t oo (Jira)
[ https://issues.apache.org/jira/browse/AIRFLOW-6387?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] t oo updated AIRFLOW-6387: -- Description: print success/skipped, similar to airflow-6358. Useful because when running 'clear' command all

[jira] [Updated] (AIRFLOW-4749) airflow connections cli - --conn_password does not allow passwords starting with dash

2019-12-29 Thread t oo (Jira)
[ https://issues.apache.org/jira/browse/AIRFLOW-4749?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] t oo updated AIRFLOW-4749: -- Description: h3. [connections|https://airflow.apache.org/cli.html#id30] List/Add/Delete connections     If i

[jira] [Work started] (AIRFLOW-4445) [security] mushroom cloud errors too verbose

2019-12-29 Thread t oo (Jira)
[ https://issues.apache.org/jira/browse/AIRFLOW-4445?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Work on AIRFLOW-4445 started by t oo. - > [security] mushroom cloud errors too verbose > ---

[jira] [Work started] (AIRFLOW-6356) make cli only print output relevant to dag argument passed in rather than from any random dag in bag

2019-12-29 Thread t oo (Jira)
[ https://issues.apache.org/jira/browse/AIRFLOW-6356?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Work on AIRFLOW-6356 started by t oo. - > make cli only print output relevant to dag argument passed in rather than > from any rand

[jira] [Updated] (AIRFLOW-6389) add config for 'allow_multi_scheduler_instances' default True

2019-12-29 Thread t oo (Jira)
[ https://issues.apache.org/jira/browse/AIRFLOW-6389?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] t oo updated AIRFLOW-6389: -- Description: right now common deployment pattern with blue/green build is: 1. on EC2 1, start scheduler 2. As

[jira] [Updated] (AIRFLOW-6389) add config for 'allow_multi_scheduler_instances' default True

2019-12-29 Thread t oo (Jira)
[ https://issues.apache.org/jira/browse/AIRFLOW-6389?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] t oo updated AIRFLOW-6389: -- Description: right now common deployment pattern with blue/green build is: 1. on EC2 1, start scheduler 2. As

[jira] [Created] (AIRFLOW-6389) add config for 'allow_multi_scheduler_instances' default True

2019-12-29 Thread t oo (Jira)
t oo created AIRFLOW-6389: - Summary: add config for 'allow_multi_scheduler_instances' default True Key: AIRFLOW-6389 URL: https://issues.apache.org/jira/browse/AIRFLOW-6389 Project: Apache Airflow I

[jira] [Assigned] (AIRFLOW-6388) SparkSubmitOperator polling should not 'consume' a slot

2019-12-29 Thread t oo (Jira)
[ https://issues.apache.org/jira/browse/AIRFLOW-6388?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] t oo reassigned AIRFLOW-6388: - Assignee: t oo > SparkSubmitOperator polling should not 'consume' a slot > ---

[jira] [Assigned] (AIRFLOW-6388) SparkSubmitOperator polling should not 'consume' a slot

2019-12-29 Thread t oo (Jira)
[ https://issues.apache.org/jira/browse/AIRFLOW-6388?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] t oo reassigned AIRFLOW-6388: - Assignee: (was: t oo) > SparkSubmitOperator polling should not 'consume' a slot >

[jira] [Updated] (AIRFLOW-6388) SparkSubmitOperator polling should not 'consume' a slot

2019-12-29 Thread t oo (Jira)
[ https://issues.apache.org/jira/browse/AIRFLOW-6388?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] t oo updated AIRFLOW-6388: -- Description: Spark jobs can often take many minutes (or even hours) to complete. The spark submit operator s

[jira] [Updated] (AIRFLOW-6388) SparkSubmitOperator polling should not 'consume' a slot

2019-12-29 Thread t oo (Jira)
[ https://issues.apache.org/jira/browse/AIRFLOW-6388?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] t oo updated AIRFLOW-6388: -- Description: Spark jobs can often take many minutes (or even hours) to complete. The spark submit operator s

[jira] [Created] (AIRFLOW-6388) SparkSubmitOperator polling should not 'consume' a slot

2019-12-29 Thread t oo (Jira)
t oo created AIRFLOW-6388: - Summary: SparkSubmitOperator polling should not 'consume' a slot Key: AIRFLOW-6388 URL: https://issues.apache.org/jira/browse/AIRFLOW-6388 Project: Apache Airflow Issue Ty

[jira] [Updated] (AIRFLOW-6387) print details of success/skipped task

2019-12-29 Thread t oo (Jira)
[ https://issues.apache.org/jira/browse/AIRFLOW-6387?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] t oo updated AIRFLOW-6387: -- Description: print success/skipped, similar to airflow-6358 (was: task instance.py   BEFORE self.state =

[jira] [Created] (AIRFLOW-6387) print details of success/skipped task

2019-12-29 Thread t oo (Jira)
t oo created AIRFLOW-6387: - Summary: print details of success/skipped task Key: AIRFLOW-6387 URL: https://issues.apache.org/jira/browse/AIRFLOW-6387 Project: Apache Airflow Issue Type: Improvement

[jira] [Work started] (AIRFLOW-4428) Externally triggered DagRun marked as 'success' state but all tasks for it are 'None' state

2019-12-29 Thread t oo (Jira)
[ https://issues.apache.org/jira/browse/AIRFLOW-4428?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Work on AIRFLOW-4428 started by t oo. - > Externally triggered DagRun marked as 'success' state but all tasks for it > are 'None' s

[jira] [Work started] (AIRFLOW-6360) config option to skip task_stats from getting completed dagruns/tis

2019-12-28 Thread t oo (Jira)
[ https://issues.apache.org/jira/browse/AIRFLOW-6360?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Work on AIRFLOW-6360 started by t oo. - > config option to skip task_stats from getting completed dagruns/tis >

[jira] [Updated] (AIRFLOW-5406) Allow spark_submit_hook to run without pip install kubernetes/airflow[kubernetes]

2019-12-28 Thread t oo (Jira)
[ https://issues.apache.org/jira/browse/AIRFLOW-5406?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] t oo updated AIRFLOW-5406: -- Description: Allow spark_submit_hook to run without pip install kubernetes/airflow[kubernetes] k8s dependen

[jira] [Reopened] (AIRFLOW-5406) Allow spark_submit_hook to run without pip install kubernetes/airflow[kubernetes]

2019-12-28 Thread t oo (Jira)
[ https://issues.apache.org/jira/browse/AIRFLOW-5406?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] t oo reopened AIRFLOW-5406: --- extra file (airflow/settings.py) in 1.10.x branch to change that is not in master branch > Allow spark_submit

[jira] [Updated] (AIRFLOW-6356) make cli only print output relevant to dag argument passed in rather than from any random dag in bag

2019-12-27 Thread t oo (Jira)
[ https://issues.apache.org/jira/browse/AIRFLOW-6356?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] t oo updated AIRFLOW-6356: -- Description: I have 5 different .py that each generate dynamic dags. One of the .py had a bug where it was p

[jira] [Updated] (AIRFLOW-6369) clear command needs a 'conf' option

2019-12-27 Thread t oo (Jira)
[ https://issues.apache.org/jira/browse/AIRFLOW-6369?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] t oo updated AIRFLOW-6369: -- Description: key-value pairs of conf can be passed into trigger_dag command ie --conf '{"ric":"amzn"}' clear

[jira] [Created] (AIRFLOW-6369) clear command needs a 'conf' option

2019-12-27 Thread t oo (Jira)
t oo created AIRFLOW-6369: - Summary: clear command needs a 'conf' option Key: AIRFLOW-6369 URL: https://issues.apache.org/jira/browse/AIRFLOW-6369 Project: Apache Airflow Issue Type: Improvement

[jira] [Assigned] (AIRFLOW-5385) SparkSubmit status spend lot of time

2019-12-27 Thread t oo (Jira)
[ https://issues.apache.org/jira/browse/AIRFLOW-5385?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] t oo reassigned AIRFLOW-5385: - Assignee: t oo > SparkSubmit status spend lot of time > > >

[jira] [Updated] (AIRFLOW-5406) Allow spark_submit_hook to run without pip install kubernetes/airflow[kubernetes]

2019-12-27 Thread t oo (Jira)
[ https://issues.apache.org/jira/browse/AIRFLOW-5406?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] t oo updated AIRFLOW-5406: -- Issue Type: Bug (was: Improvement) > Allow spark_submit_hook to run without pip install > kubernetes/airflo

[jira] [Work started] (AIRFLOW-5406) Allow spark_submit_hook to run without pip install kubernetes/airflow[kubernetes]

2019-12-27 Thread t oo (Jira)
[ https://issues.apache.org/jira/browse/AIRFLOW-5406?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Work on AIRFLOW-5406 started by t oo. - > Allow spark_submit_hook to run without pip install > kubernetes/airflow[kubernetes] > ---

[jira] [Updated] (AIRFLOW-5406) Allow spark_submit_hook to run without pip install kubernetes/airflow[kubernetes]

2019-12-26 Thread t oo (Jira)
[ https://issues.apache.org/jira/browse/AIRFLOW-5406?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] t oo updated AIRFLOW-5406: -- Priority: Critical (was: Minor) > Allow spark_submit_hook to run without pip install > kubernetes/airflow[k

[jira] [Updated] (AIRFLOW-5406) Allow spark_submit_hook to run without pip install kubernetes/airflow[kubernetes]

2019-12-26 Thread t oo (Jira)
[ https://issues.apache.org/jira/browse/AIRFLOW-5406?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] t oo updated AIRFLOW-5406: -- Priority: Blocker (was: Critical) > Allow spark_submit_hook to run without pip install > kubernetes/airflow

[jira] [Work started] (AIRFLOW-6229) SparkSubmitOperator polls forever if status json can't find driverState tag

2019-12-26 Thread t oo (Jira)
[ https://issues.apache.org/jira/browse/AIRFLOW-6229?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Work on AIRFLOW-6229 started by t oo. - > SparkSubmitOperator polls forever if status json can't find driverState tag >

[jira] [Commented] (AIRFLOW-4749) airflow connections cli - --conn_password does not allow passwords starting with dash

2019-12-26 Thread t oo (Jira)
[ https://issues.apache.org/jira/browse/AIRFLOW-4749?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17003805#comment-17003805 ] t oo commented on AIRFLOW-4749: --- the password is dynamically generated, caller should not

[jira] [Closed] (AIRFLOW-5273) Config flag to limit webUI to show only last 35 days of taskinstances/dagruns

2019-12-26 Thread t oo (Jira)
[ https://issues.apache.org/jira/browse/AIRFLOW-5273?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] t oo closed AIRFLOW-5273. - Resolution: Invalid > Config flag to limit webUI to show only last 35 days of taskinstances/dagruns > -

[jira] [Work started] (AIRFLOW-6350) security - spark submit operator logging+exceptions should mask passwords

2019-12-26 Thread t oo (Jira)
[ https://issues.apache.org/jira/browse/AIRFLOW-6350?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Work on AIRFLOW-6350 started by t oo. - > security - spark submit operator logging+exceptions should mask passwords > -

[jira] [Work started] (AIRFLOW-6335) dag_processor_manager timeout logs should be ERROR not INFO/WARN

2019-12-26 Thread t oo (Jira)
[ https://issues.apache.org/jira/browse/AIRFLOW-6335?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Work on AIRFLOW-6335 started by t oo. - > dag_processor_manager timeout logs should be ERROR not INFO/WARN > ---

[jira] [Work started] (AIRFLOW-6348) security - cli.py is currently printing logs with password if you use cli to add connection with conn_password

2019-12-26 Thread t oo (Jira)
[ https://issues.apache.org/jira/browse/AIRFLOW-6348?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Work on AIRFLOW-6348 started by t oo. - > security - cli.py is currently printing logs with password if you use cli to > add connec

[jira] [Work started] (AIRFLOW-6351) security - ui - Add Cross Site Scripting defence

2019-12-26 Thread t oo (Jira)
[ https://issues.apache.org/jira/browse/AIRFLOW-6351?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Work on AIRFLOW-6351 started by t oo. - > security - ui - Add Cross Site Scripting defence > ---

[jira] [Work started] (AIRFLOW-6359) spark_submit_hook.py polling too frequent

2019-12-26 Thread t oo (Jira)
[ https://issues.apache.org/jira/browse/AIRFLOW-6359?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Work on AIRFLOW-6359 started by t oo. - > spark_submit_hook.py polling too frequent > - > >

[jira] [Work started] (AIRFLOW-6353) security - ui - add click jacking defence

2019-12-26 Thread t oo (Jira)
[ https://issues.apache.org/jira/browse/AIRFLOW-6353?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Work on AIRFLOW-6353 started by t oo. - > security - ui - add click jacking defence > - > >

[jira] [Work started] (AIRFLOW-6352) security - ui - add timeout

2019-12-26 Thread t oo (Jira)
[ https://issues.apache.org/jira/browse/AIRFLOW-6352?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Work on AIRFLOW-6352 started by t oo. - > security - ui - add timeout > --- > > Key: AIRFLOW

[jira] [Work started] (AIRFLOW-6358) print details of failed task

2019-12-26 Thread t oo (Jira)
[ https://issues.apache.org/jira/browse/AIRFLOW-6358?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Work on AIRFLOW-6358 started by t oo. - > print details of failed task > > > Key: AIRFL

[jira] [Work started] (AIRFLOW-6349) security - api should deny access by default

2019-12-26 Thread t oo (Jira)
[ https://issues.apache.org/jira/browse/AIRFLOW-6349?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Work on AIRFLOW-6349 started by t oo. - > security - api should deny access by default > ---

[jira] [Updated] (AIRFLOW-5406) Allow spark_submit_hook to run without pip install kubernetes/airflow[kubernetes]

2019-12-26 Thread t oo (Jira)
[ https://issues.apache.org/jira/browse/AIRFLOW-5406?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] t oo updated AIRFLOW-5406: -- Description: Allow spark_submit_hook to run without pip install kubernetes/airflow[kubernetes] k8s dependen

[jira] [Created] (AIRFLOW-6360) config option to skip task_stats from getting completed dagruns/tis

2019-12-26 Thread t oo (Jira)
t oo created AIRFLOW-6360: - Summary: config option to skip task_stats from getting completed dagruns/tis Key: AIRFLOW-6360 URL: https://issues.apache.org/jira/browse/AIRFLOW-6360 Project: Apache Airflow

[jira] [Updated] (AIRFLOW-6349) security - api should deny access by default

2019-12-26 Thread t oo (Jira)
[ https://issues.apache.org/jira/browse/AIRFLOW-6349?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] t oo updated AIRFLOW-6349: -- Description: below should be 'airflow.api.auth.backend.deny_all' by default: |[api]| | # How to authenticate

[jira] [Updated] (AIRFLOW-6352) security - ui - add timeout

2019-12-26 Thread t oo (Jira)
[ https://issues.apache.org/jira/browse/AIRFLOW-6352?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] t oo updated AIRFLOW-6352: -- Description: www/app.py Add WebUI login timeout of 60 minutes   Fix: import datetime import flask import f

[jira] [Comment Edited] (AIRFLOW-5385) SparkSubmit status spend lot of time

2019-12-26 Thread t oo (Jira)
[ https://issues.apache.org/jira/browse/AIRFLOW-5385?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16986097#comment-16986097 ] t oo edited comment on AIRFLOW-5385 at 12/26/19 4:56 PM: - [~ser

[jira] [Comment Edited] (AIRFLOW-5385) SparkSubmit status spend lot of time

2019-12-26 Thread t oo (Jira)
[ https://issues.apache.org/jira/browse/AIRFLOW-5385?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17003406#comment-17003406 ] t oo edited comment on AIRFLOW-5385 at 12/26/19 4:55 PM: - [~ser

[jira] [Created] (AIRFLOW-6359) spark_submit_hook.py polling too frequent

2019-12-26 Thread t oo (Jira)
t oo created AIRFLOW-6359: - Summary: spark_submit_hook.py polling too frequent Key: AIRFLOW-6359 URL: https://issues.apache.org/jira/browse/AIRFLOW-6359 Project: Apache Airflow Issue Type: Improvemen

[jira] [Comment Edited] (AIRFLOW-4355) Externally triggered DAG is marked as 'success' even if a task has been 'removed'!

2019-12-26 Thread t oo (Jira)
[ https://issues.apache.org/jira/browse/AIRFLOW-4355?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17003394#comment-17003394 ] t oo edited comment on AIRFLOW-4355 at 12/26/19 4:25 PM: - [~pra

[jira] [Comment Edited] (AIRFLOW-4355) Externally triggered DAG is marked as 'success' even if a task has been 'removed'!

2019-12-26 Thread t oo (Jira)
[ https://issues.apache.org/jira/browse/AIRFLOW-4355?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17003394#comment-17003394 ] t oo edited comment on AIRFLOW-4355 at 12/26/19 4:19 PM: - [~pra

[jira] [Created] (AIRFLOW-6358) print details of failed task

2019-12-26 Thread t oo (Jira)
t oo created AIRFLOW-6358: - Summary: print details of failed task Key: AIRFLOW-6358 URL: https://issues.apache.org/jira/browse/AIRFLOW-6358 Project: Apache Airflow Issue Type: Improvement C

[jira] [Updated] (AIRFLOW-6356) make cli only print output relevant to dag argument passed in rather than from any random dag in bag

2019-12-26 Thread t oo (Jira)
[ https://issues.apache.org/jira/browse/AIRFLOW-6356?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] t oo updated AIRFLOW-6356: -- Description: I have 5 different .py that each generate dynamic dags. One of the .py had a bug where it was p

[jira] [Created] (AIRFLOW-6356) make cli only print output relevant to dag argument passed in rather than from any random dag in bag

2019-12-26 Thread t oo (Jira)
t oo created AIRFLOW-6356: - Summary: make cli only print output relevant to dag argument passed in rather than from any random dag in bag Key: AIRFLOW-6356 URL: https://issues.apache.org/jira/browse/AIRFLOW-6356

[jira] [Work started] (AIRFLOW-5149) Config flag to skip SLA checks

2019-12-25 Thread t oo (Jira)
[ https://issues.apache.org/jira/browse/AIRFLOW-5149?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Work on AIRFLOW-5149 started by t oo. - > Config flag to skip SLA checks > -- > > Key: A

[jira] [Updated] (AIRFLOW-5149) Config flag to skip SLA checks

2019-12-25 Thread t oo (Jira)
[ https://issues.apache.org/jira/browse/AIRFLOW-5149?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] t oo updated AIRFLOW-5149: -- Fix Version/s: 1.10.8 > Config flag to skip SLA checks > -- > > K

[jira] [Created] (AIRFLOW-6353) security - ui - add click jacking defence

2019-12-25 Thread t oo (Jira)
t oo created AIRFLOW-6353: - Summary: security - ui - add click jacking defence Key: AIRFLOW-6353 URL: https://issues.apache.org/jira/browse/AIRFLOW-6353 Project: Apache Airflow Issue Type: Bug

[jira] [Created] (AIRFLOW-6352) security - ui - add timeout

2019-12-25 Thread t oo (Jira)
t oo created AIRFLOW-6352: - Summary: security - ui - add timeout Key: AIRFLOW-6352 URL: https://issues.apache.org/jira/browse/AIRFLOW-6352 Project: Apache Airflow Issue Type: Bug Components

[jira] [Created] (AIRFLOW-6351) security - ui - Add Cross Site Scripting defence

2019-12-25 Thread t oo (Jira)
t oo created AIRFLOW-6351: - Summary: security - ui - Add Cross Site Scripting defence Key: AIRFLOW-6351 URL: https://issues.apache.org/jira/browse/AIRFLOW-6351 Project: Apache Airflow Issue Type: Bug

[jira] [Commented] (AIRFLOW-5385) SparkSubmit status spend lot of time

2019-12-25 Thread t oo (Jira)
[ https://issues.apache.org/jira/browse/AIRFLOW-5385?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17003406#comment-17003406 ] t oo commented on AIRFLOW-5385: --- [~sergio.soto] [~Diego García] do u have a PR? another

[jira] [Created] (AIRFLOW-6350) security - spark submit operator logging+exceptions should mask passwords

2019-12-25 Thread t oo (Jira)
t oo created AIRFLOW-6350: - Summary: security - spark submit operator logging+exceptions should mask passwords Key: AIRFLOW-6350 URL: https://issues.apache.org/jira/browse/AIRFLOW-6350 Project: Apache Airflo

[jira] [Created] (AIRFLOW-6349) security - api should deny access by default

2019-12-25 Thread t oo (Jira)
t oo created AIRFLOW-6349: - Summary: security - api should deny access by default Key: AIRFLOW-6349 URL: https://issues.apache.org/jira/browse/AIRFLOW-6349 Project: Apache Airflow Issue Type: Bug

[jira] [Created] (AIRFLOW-6348) security - cli.py is currently printing logs with password if you use cli to add connection with conn_password

2019-12-25 Thread t oo (Jira)
t oo created AIRFLOW-6348: - Summary: security - cli.py is currently printing logs with password if you use cli to add connection with conn_password Key: AIRFLOW-6348 URL: https://issues.apache.org/jira/browse/AIRFLOW-6348

[jira] [Commented] (AIRFLOW-2516) Deadlock found when trying to update task_instance table

2019-12-25 Thread t oo (Jira)
[ https://issues.apache.org/jira/browse/AIRFLOW-2516?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17003397#comment-17003397 ] t oo commented on AIRFLOW-2516: --- should https://github.com/apache/airflow/blob/1.10.7/ai

[jira] [Comment Edited] (AIRFLOW-4355) Externally triggered DAG is marked as 'success' even if a task has been 'removed'!

2019-12-25 Thread t oo (Jira)
[ https://issues.apache.org/jira/browse/AIRFLOW-4355?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17003394#comment-17003394 ] t oo edited comment on AIRFLOW-4355 at 12/25/19 10:52 PM: -- [~p

[jira] [Commented] (AIRFLOW-4355) Externally triggered DAG is marked as 'success' even if a task has been 'removed'!

2019-12-25 Thread t oo (Jira)
[ https://issues.apache.org/jira/browse/AIRFLOW-4355?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17003394#comment-17003394 ] t oo commented on AIRFLOW-4355: --- [~pramodpsb] maybe add REMOVED to the unfinished list:

[jira] [Reopened] (AIRFLOW-4389) trigger_dag cli 'force' option

2019-12-25 Thread t oo (Jira)
[ https://issues.apache.org/jira/browse/AIRFLOW-4389?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] t oo reopened AIRFLOW-4389: --- > trigger_dag cli 'force' option > -- > > Key: AIRFLOW-4389 >

[jira] [Reopened] (AIRFLOW-4428) Externally triggered DagRun marked as 'success' state but all tasks for it are 'None' state

2019-12-25 Thread t oo (Jira)
[ https://issues.apache.org/jira/browse/AIRFLOW-4428?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] t oo reopened AIRFLOW-4428: --- Changing to 'improvement' to make it an error (rather than success) to create a DAG run for exec_date before t

[jira] [Updated] (AIRFLOW-4428) Externally triggered DagRun marked as 'success' state but all tasks for it are 'None' state

2019-12-25 Thread t oo (Jira)
[ https://issues.apache.org/jira/browse/AIRFLOW-4428?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] t oo updated AIRFLOW-4428: -- Issue Type: Improvement (was: Bug) > Externally triggered DagRun marked as 'success' state but all tasks for

[jira] [Commented] (AIRFLOW-4796) DOCO - DaskExecutor logs

2019-12-25 Thread t oo (Jira)
[ https://issues.apache.org/jira/browse/AIRFLOW-4796?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17003386#comment-17003386 ] t oo commented on AIRFLOW-4796: --- [~jlowin] [~bolke] [~mariusvniekerk]  cc > DOCO - DaskE

<    1   2   3   4   >