[jira] [Updated] (AIRFLOW-921) 1.8.0rc Issues
[ https://issues.apache.org/jira/browse/AIRFLOW-921?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dan Davydov updated AIRFLOW-921: Description: These are the pending issues for the Airflow 1.8.0 release: Blockers: [~bolke] please merge into the next RC and then remove from the list the issues below once they are merged into master - https://issues.apache.org/jira/browse/AIRFLOW-900 - https://github.com/apache/incubator-airflow/pull/2102 - https://issues.apache.org/jira/browse/AIRFLOW-910 - https://github.com/apache/incubator-airflow/pull/2107 - https://issues.apache.org/jira/browse/AIRFLOW-919 - https://github.com/apache/incubator-airflow/pull/2110 - https://issues.apache.org/jira/browse/AIRFLOW-920 - Skipped tasks potentially cause a dagrun to be marked as failure/success prematurely (one theory is that this is the same issue as https://issues.apache.org/jira/browse/AIRFLOW-872) Other Issues: - High DB Load (~8x more than 1.7) - We are still investigating but it's probably not a blocker for the release - (Theories: Might need execution_date index on dag_run (based on slow process list) OR it might be this query which is long running SELECT union_ti.dag_id AS union_ti_dag_id, union_ti.state AS union_ti_state, count( *) AS count_1 FR)) [~bolke] was: These are the pending issues for the Airflow 1.8.0 release: Blockers: [~bolke] please merge into the next RC and then remove from the list the issues below once they are merged into master - https://issues.apache.org/jira/browse/AIRFLOW-900 - https://github.com/apache/incubator-airflow/pull/2102 - https://issues.apache.org/jira/browse/AIRFLOW-910 - https://github.com/apache/incubator-airflow/pull/2107 - https://issues.apache.org/jira/browse/AIRFLOW-919 - https://github.com/apache/incubator-airflow/pull/2110 - https://issues.apache.org/jira/browse/AIRFLOW-920 - Skipped tasks potentially cause a dagrun to be marked as failure/success prematurely (one theory is that this is the same issue as https://issues.apache.org/jira/browse/AIRFLOW-872) Other Issues (could also be blockers): - High DB Load (~8x more than 1.7) - We are still investigating but it's probably not a blocker for the release - (Theories: Might need execution_date index on dag_run (based on slow process list) OR it might be this query which is long running SELECT union_ti.dag_id AS union_ti_dag_id, union_ti.state AS union_ti_state, count( *) AS count_1 FR)) [~bolke] > 1.8.0rc Issues > -- > > Key: AIRFLOW-921 > URL: https://issues.apache.org/jira/browse/AIRFLOW-921 > Project: Apache Airflow > Issue Type: Task > Reporter: Dan Davydov >Priority: Blocker > > These are the pending issues for the Airflow 1.8.0 release: > Blockers: > [~bolke] please merge into the next RC and then remove from the list the > issues below once they are merged into master > - https://issues.apache.org/jira/browse/AIRFLOW-900 - > https://github.com/apache/incubator-airflow/pull/2102 > - https://issues.apache.org/jira/browse/AIRFLOW-910 - > https://github.com/apache/incubator-airflow/pull/2107 > - https://issues.apache.org/jira/browse/AIRFLOW-919 - > https://github.com/apache/incubator-airflow/pull/2110 > - https://issues.apache.org/jira/browse/AIRFLOW-920 > - Skipped tasks potentially cause a dagrun to be marked as failure/success > prematurely (one theory is that this is the same issue as > https://issues.apache.org/jira/browse/AIRFLOW-872) > Other Issues: > - High DB Load (~8x more than 1.7) - We are still investigating but it's > probably not a blocker for the release - (Theories: Might need execution_date > index on dag_run (based on slow process list) OR it might be this query which > is long running SELECT union_ti.dag_id AS union_ti_dag_id, union_ti.state AS > union_ti_state, count( *) AS count_1 > FR)) > [~bolke] -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Assigned] (AIRFLOW-919) Setting a task to running manually breaks a DAGs UI
[
https://issues.apache.org/jira/browse/AIRFLOW-919?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Dan Davydov reassigned AIRFLOW-919:
---
Assignee: Dan Davydov
> Setting a task to running manually breaks a DAGs UI
> ---
>
> Key: AIRFLOW-919
> URL: https://issues.apache.org/jira/browse/AIRFLOW-919
> Project: Apache Airflow
> Issue Type: Bug
> Components: webserver
> Reporter: Dan Davydov
> Assignee: Dan Davydov
>
> If you set a task instance with no start date to the RUNNING state manually
> (e.g. mark a task as success and then change the state to running in the CRUD
> UI), you get the following traceback due to the lack of a start date for the
> task instance:
> {code}
> Traceback (most recent call last):
> File "/usr/local/lib/python2.7/dist-packages/flask/app.py", line 1817, in
> wsgi_app
> response = self.full_dispatch_request()
> File "/usr/local/lib/python2.7/dist-packages/flask/app.py", line 1477, in
> full_dispatch_request
> rv = self.handle_user_exception(e)
> File "/usr/local/lib/python2.7/dist-packages/flask/app.py", line 1381, in
> handle_user_exception
> reraise(exc_type, exc_value, tb)
> File "/usr/local/lib/python2.7/dist-packages/flask/app.py", line 1475, in
> full_dispatch_request
> rv = self.dispatch_request()
> File "/usr/local/lib/python2.7/dist-packages/flask/app.py", line 1461, in
> dispatch_request
> return self.view_functions[rule.endpoint](**req.view_args)
> File "/usr/local/lib/python2.7/dist-packages/flask_admin/base.py", line 68,
> in inner
> return self._run_view(f, *args, **kwargs)
> File "/usr/local/lib/python2.7/dist-packages/flask_admin/base.py", line
> 367, in _run_view
> return fn(self, *args, **kwargs)
> File "/usr/local/lib/python2.7/dist-packages/flask_login.py", line 758, in
> decorated_view
> return func(*args, **kwargs)
> File "/usr/local/lib/python2.7/dist-packages/airflow/www/utils.py", line
> 221, in view_func
> return f(*args, **kwargs)
> File "/usr/local/lib/python2.7/dist-packages/airflow/www/utils.py", line
> 125, in wrapper
> return f(*args, **kwargs)
> File "/usr/local/lib/python2.7/dist-packages/airflow/www/views.py", line
> 1248, in tree
> 'children': [recurse_nodes(t, set()) for t in dag.roots],
> File "/usr/local/lib/python2.7/dist-packages/airflow/www/views.py", line
> 1235, in recurse_nodes
> for d in dates],
> File "/usr/local/lib/python2.7/dist-packages/airflow/www/views.py", line
> 1224, in set_duration
> d = datetime.now() - dateutil.parser.parse(tid["start_date"])
> File "/usr/local/lib/python2.7/dist-packages/dateutil/parser.py", line
> 1164, in parse
> return DEFAULTPARSER.parse(timestr, **kwargs)
> File "/usr/local/lib/python2.7/dist-packages/dateutil/parser.py", line 552,
> in parse
> res, skipped_tokens = self._parse(timestr, **kwargs)
> File "/usr/local/lib/python2.7/dist-packages/dateutil/parser.py", line 671,
> in _parse
> l = _timelex.split(timestr) # Splits the timestr into tokens
> File "/usr/local/lib/python2.7/dist-packages/dateutil/parser.py", line 188,
> in split
> return list(cls(s))
> File "/usr/local/lib/python2.7/dist-packages/dateutil/parser.py", line 184,
> in next
> return self.__next__() # Python 2.x support
> File "/usr/local/lib/python2.7/dist-packages/dateutil/parser.py", line 177,
> in __next__
> token = self.get_token()
> File "/usr/local/lib/python2.7/dist-packages/dateutil/parser.py", line 93,
> in get_token
> nextchar = self.instream.read(1)
> AttributeError: 'NoneType' object has no attribute 'read'
> {code}
> Instead we should display an empty start date.
> [~bolke]
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Updated] (AIRFLOW-921) 1.8.0rc Issues
[ https://issues.apache.org/jira/browse/AIRFLOW-921?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dan Davydov updated AIRFLOW-921: Description: These are the pending issues for the Airflow 1.8.0 release: Blockers: - https://issues.apache.org/jira/browse/AIRFLOW-900 - https://github.com/apache/incubator-airflow/pull/2102 - https://issues.apache.org/jira/browse/AIRFLOW-910 - https://github.com/apache/incubator-airflow/pull/2107 - https://issues.apache.org/jira/browse/AIRFLOW-919 - https://github.com/apache/incubator-airflow/pull/2110 - https://issues.apache.org/jira/browse/AIRFLOW-920 - Skipped tasks potentially cause a dagrun to be marked as failure/success prematurely (one theory is that this is the same issue as https://issues.apache.org/jira/browse/AIRFLOW-872) Other Issues (could also be blockers): - High DB Load (~8x more than 1.7) - We are still investigating but it's probably not a blocker for the release - (Theories: Might need execution_date index on dag_run (based on slow process list) OR it might be this query which is long running SELECT union_ti.dag_id AS union_ti_dag_id, union_ti.state AS union_ti_state, count( *) AS count_1 FR)) [~bolke] was: These are the pending issues for the Airflow 1.8.0 release: Blockers: - https://issues.apache.org/jira/browse/AIRFLOW-900 - https://github.com/apache/incubator-airflow/pull/2102 - https://issues.apache.org/jira/browse/AIRFLOW-910 - https://github.com/apache/incubator-airflow/pull/2107 - https://issues.apache.org/jira/browse/AIRFLOW-920 - Skipped tasks potentially cause a dagrun to be marked as failure/success prematurely (one theory is that this is the same issue as https://issues.apache.org/jira/browse/AIRFLOW-872) Other Issues (could also be blockers): - https://issues.apache.org/jira/browse/AIRFLOW-919 - High DB Load (~8x more than 1.7) - We are still investigating but it's probably not a blocker for the release - (Theories: Might need execution_date index on dag_run (based on slow process list) OR it might be this query which is long running SELECT union_ti.dag_id AS union_ti_dag_id, union_ti.state AS union_ti_state, count( *) AS count_1 FR)) [~bolke] > 1.8.0rc Issues > -- > > Key: AIRFLOW-921 > URL: https://issues.apache.org/jira/browse/AIRFLOW-921 > Project: Apache Airflow > Issue Type: Task > Reporter: Dan Davydov >Priority: Blocker > > These are the pending issues for the Airflow 1.8.0 release: > Blockers: > - https://issues.apache.org/jira/browse/AIRFLOW-900 - > https://github.com/apache/incubator-airflow/pull/2102 > - https://issues.apache.org/jira/browse/AIRFLOW-910 - > https://github.com/apache/incubator-airflow/pull/2107 > - https://issues.apache.org/jira/browse/AIRFLOW-919 - > https://github.com/apache/incubator-airflow/pull/2110 > - https://issues.apache.org/jira/browse/AIRFLOW-920 > - Skipped tasks potentially cause a dagrun to be marked as failure/success > prematurely (one theory is that this is the same issue as > https://issues.apache.org/jira/browse/AIRFLOW-872) > Other Issues (could also be blockers): > - High DB Load (~8x more than 1.7) - We are still investigating but it's > probably not a blocker for the release - (Theories: Might need execution_date > index on dag_run (based on slow process list) OR it might be this query which > is long running SELECT union_ti.dag_id AS union_ti_dag_id, union_ti.state AS > union_ti_state, count( *) AS count_1 > FR)) > [~bolke] -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Updated] (AIRFLOW-921) 1.8.0rc Issues
[ https://issues.apache.org/jira/browse/AIRFLOW-921?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dan Davydov updated AIRFLOW-921: Description: These are the pending issues for the Airflow 1.8.0 release: Blockers: [~bolke] please merge into the next RC and then remove from the list the issues below once they are merged into master - https://issues.apache.org/jira/browse/AIRFLOW-900 - https://github.com/apache/incubator-airflow/pull/2102 - https://issues.apache.org/jira/browse/AIRFLOW-910 - https://github.com/apache/incubator-airflow/pull/2107 - https://issues.apache.org/jira/browse/AIRFLOW-919 - https://github.com/apache/incubator-airflow/pull/2110 - https://issues.apache.org/jira/browse/AIRFLOW-920 - Skipped tasks potentially cause a dagrun to be marked as failure/success prematurely (one theory is that this is the same issue as https://issues.apache.org/jira/browse/AIRFLOW-872) Other Issues (could also be blockers): - High DB Load (~8x more than 1.7) - We are still investigating but it's probably not a blocker for the release - (Theories: Might need execution_date index on dag_run (based on slow process list) OR it might be this query which is long running SELECT union_ti.dag_id AS union_ti_dag_id, union_ti.state AS union_ti_state, count( *) AS count_1 FR)) [~bolke] was: These are the pending issues for the Airflow 1.8.0 release: Blockers: - https://issues.apache.org/jira/browse/AIRFLOW-900 - https://github.com/apache/incubator-airflow/pull/2102 - https://issues.apache.org/jira/browse/AIRFLOW-910 - https://github.com/apache/incubator-airflow/pull/2107 - https://issues.apache.org/jira/browse/AIRFLOW-919 - https://github.com/apache/incubator-airflow/pull/2110 - https://issues.apache.org/jira/browse/AIRFLOW-920 - Skipped tasks potentially cause a dagrun to be marked as failure/success prematurely (one theory is that this is the same issue as https://issues.apache.org/jira/browse/AIRFLOW-872) Other Issues (could also be blockers): - High DB Load (~8x more than 1.7) - We are still investigating but it's probably not a blocker for the release - (Theories: Might need execution_date index on dag_run (based on slow process list) OR it might be this query which is long running SELECT union_ti.dag_id AS union_ti_dag_id, union_ti.state AS union_ti_state, count( *) AS count_1 FR)) [~bolke] > 1.8.0rc Issues > -- > > Key: AIRFLOW-921 > URL: https://issues.apache.org/jira/browse/AIRFLOW-921 > Project: Apache Airflow > Issue Type: Task > Reporter: Dan Davydov >Priority: Blocker > > These are the pending issues for the Airflow 1.8.0 release: > Blockers: > [~bolke] please merge into the next RC and then remove from the list the > issues below once they are merged into master > - https://issues.apache.org/jira/browse/AIRFLOW-900 - > https://github.com/apache/incubator-airflow/pull/2102 > - https://issues.apache.org/jira/browse/AIRFLOW-910 - > https://github.com/apache/incubator-airflow/pull/2107 > - https://issues.apache.org/jira/browse/AIRFLOW-919 - > https://github.com/apache/incubator-airflow/pull/2110 > - https://issues.apache.org/jira/browse/AIRFLOW-920 > - Skipped tasks potentially cause a dagrun to be marked as failure/success > prematurely (one theory is that this is the same issue as > https://issues.apache.org/jira/browse/AIRFLOW-872) > Other Issues (could also be blockers): > - High DB Load (~8x more than 1.7) - We are still investigating but it's > probably not a blocker for the release - (Theories: Might need execution_date > index on dag_run (based on slow process list) OR it might be this query which > is long running SELECT union_ti.dag_id AS union_ti_dag_id, union_ti.state AS > union_ti_state, count( *) AS count_1 > FR)) > [~bolke] -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Created] (AIRFLOW-494) Add per-operator success/failure metrics.
Dan Davydov created AIRFLOW-494: --- Summary: Add per-operator success/failure metrics. Key: AIRFLOW-494 URL: https://issues.apache.org/jira/browse/AIRFLOW-494 Project: Apache Airflow Issue Type: Improvement Components: rics Reporter: Dan Davydov Assignee: Dan Davydov Priority: Minor It would be good to have metrics for success/failure rates of each operator, that way when we e.g. do a new release we will have some signal if there is a regression in an operator. It will also be useful if e.g. a user wants to upgrade their infrastructure and make sure that all of the operators still work as expected. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (AIRFLOW-149) Blocked TI Explainer + Dependency Engine
[ https://issues.apache.org/jira/browse/AIRFLOW-149?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dan Davydov updated AIRFLOW-149: External issue URL: https://github.com/apache/incubator-airflow/pull/1729 (was: https://github.com/apache/incubator-airflow/pull/1525) > Blocked TI Explainer + Dependency Engine > > > Key: AIRFLOW-149 > URL: https://issues.apache.org/jira/browse/AIRFLOW-149 > Project: Apache Airflow > Issue Type: New Feature > Components: core, scheduler > Reporter: Dan Davydov > Assignee: Dan Davydov > > Created from: https://github.com/apache/incubator-airflow/issues/1383 > - Simplify, consolidate, and make consistent the logic of whether or not a > task should be run > - Provide a view that gives insight into why a task instance is not currently > running (no more viewing the scheduler logs to find out why a task instance > isn't running for the majority of cases) e.g. (this will not be the final > product): -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Resolved] (AIRFLOW-149) Blocked TI Explainer + Dependency Engine
[ https://issues.apache.org/jira/browse/AIRFLOW-149?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dan Davydov resolved AIRFLOW-149. - Resolution: Fixed > Blocked TI Explainer + Dependency Engine > > > Key: AIRFLOW-149 > URL: https://issues.apache.org/jira/browse/AIRFLOW-149 > Project: Apache Airflow > Issue Type: New Feature > Components: core, scheduler > Reporter: Dan Davydov > Assignee: Dan Davydov > > Created from: https://github.com/apache/incubator-airflow/issues/1383 > - Simplify, consolidate, and make consistent the logic of whether or not a > task should be run > - Provide a view that gives insight into why a task instance is not currently > running (no more viewing the scheduler logs to find out why a task instance > isn't running for the majority of cases) e.g. (this will not be the final > product): -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (AIRFLOW-536) Dags only get one dagrun is scheduled per scheduler loop
Dan Davydov created AIRFLOW-536: --- Summary: Dags only get one dagrun is scheduled per scheduler loop Key: AIRFLOW-536 URL: https://issues.apache.org/jira/browse/AIRFLOW-536 Project: Apache Airflow Issue Type: Bug Reporter: Dan Davydov At the moment Dags only get one dag scheduled per scheduler loop (which can be quite slow, on the order of minutes), but what should happen is that all pending dagruns for a dagrun should be scheduled. In addition to dagruns being scheduled more quickly for dags that havem ultiple dag runs queued up (e.g. paused dags), fixing this issue will enable a "only_run_latest" option for dags (so e.g. if the dag is paused and then later unpaused, only the most recent ds will be run), as well as facilitating compatibility between backfills and the scheduler (if you run a backfill over 100 dag runs the backfill will be waiting on 100 scheduler cycles). -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (AIRFLOW-633) Show task instance attributes in task instance view
Dan Davydov created AIRFLOW-633: --- Summary: Show task instance attributes in task instance view Key: AIRFLOW-633 URL: https://issues.apache.org/jira/browse/AIRFLOW-633 Project: Apache Airflow Issue Type: Improvement Components: webserver Reporter: Dan Davydov Assignee: Dan Davydov Priority: Minor Currently the task instance view exposes "task" attributes but does not expose "task instance" attributes like "hostname" which are important (e.g. if the produced log is extremely large or inaccessible). -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (AIRFLOW-536) Dags only get one dagrun is scheduled per scheduler loop
[ https://issues.apache.org/jira/browse/AIRFLOW-536?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dan Davydov updated AIRFLOW-536: Description: At the moment Dags only get one dag scheduled per scheduler loop (which can be quite slow, on the order of minutes), but what should happen is that all pending dagruns for a dag should be scheduled. In addition to dagruns being scheduled more quickly for dags that havem ultiple dag runs queued up (e.g. paused dags), fixing this issue will enable a "only_run_latest" option for dags (so e.g. if the dag is paused and then later unpaused, only the most recent ds will be run), as well as facilitating compatibility between backfills and the scheduler (if you run a backfill over 100 dag runs the backfill will be waiting on 100 scheduler cycles). was: At the moment Dags only get one dag scheduled per scheduler loop (which can be quite slow, on the order of minutes), but what should happen is that all pending dagruns for a dagrun should be scheduled. In addition to dagruns being scheduled more quickly for dags that havem ultiple dag runs queued up (e.g. paused dags), fixing this issue will enable a "only_run_latest" option for dags (so e.g. if the dag is paused and then later unpaused, only the most recent ds will be run), as well as facilitating compatibility between backfills and the scheduler (if you run a backfill over 100 dag runs the backfill will be waiting on 100 scheduler cycles). > Dags only get one dagrun is scheduled per scheduler loop > > > Key: AIRFLOW-536 > URL: https://issues.apache.org/jira/browse/AIRFLOW-536 > Project: Apache Airflow > Issue Type: Bug >Reporter: Dan Davydov > > At the moment Dags only get one dag scheduled per scheduler loop (which can > be quite slow, on the order of minutes), but what should happen is that all > pending dagruns for a dag should be scheduled. > In addition to dagruns being scheduled more quickly for dags that havem > ultiple dag runs queued up (e.g. paused dags), fixing this issue will enable > a "only_run_latest" option for dags (so e.g. if the dag is paused and then > later unpaused, only the most recent ds will be run), as well as facilitating > compatibility between backfills and the scheduler (if you run a backfill over > 100 dag runs the backfill will be waiting on 100 scheduler cycles). -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (AIRFLOW-682) Bump MAX_PERIODS
Dan Davydov created AIRFLOW-682: --- Summary: Bump MAX_PERIODS Key: AIRFLOW-682 URL: https://issues.apache.org/jira/browse/AIRFLOW-682 Project: Apache Airflow Issue Type: Bug Reporter: Dan Davydov Assignee: Dan Davydov It is not possible to mark success on some large DAGs due to the MAX_PERIODS being set to 1000. We should temporarily bump it up until work can be done to scale the mark success endpoint much higher. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (AIRFLOW-684) Add task log pagination to webservers
Dan Davydov created AIRFLOW-684: --- Summary: Add task log pagination to webservers Key: AIRFLOW-684 URL: https://issues.apache.org/jira/browse/AIRFLOW-684 Project: Apache Airflow Issue Type: Improvement Components: webserver Reporter: Dan Davydov At the moment very large task instances logs do not load in the browser, they should be paginated in an efficient manner. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (AIRFLOW-698) dag_run "scheduled" property should be it's own DB column
Dan Davydov created AIRFLOW-698: --- Summary: dag_run "scheduled" property should be it's own DB column Key: AIRFLOW-698 URL: https://issues.apache.org/jira/browse/AIRFLOW-698 Project: Apache Airflow Issue Type: Improvement Reporter: Dan Davydov The airflow schedule only executes dag_runs that have a run_id that start with "scheduled__". This can be very confusing, especially when manually creating a dagrun and forgetting the "scheduled__" prefix. The "scheduled" part should be pulled into a separate column so that it is very clear in the UI that a user is creating a DAG that isn't scheduled. cc [~maxime.beauche...@apache.org] -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (AIRFLOW-747) retry_delay not honored
[
https://issues.apache.org/jira/browse/AIRFLOW-747?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15819176#comment-15819176
]
Dan Davydov commented on AIRFLOW-747:
-
The cgroups PR is my priority right now but happy to look at this after.
> retry_delay not honored
> ---
>
> Key: AIRFLOW-747
> URL: https://issues.apache.org/jira/browse/AIRFLOW-747
> Project: Apache Airflow
> Issue Type: Bug
>Reporter: Harvey Xia
>Assignee: Bolke de Bruin
>Priority: Blocker
>
> In Airflow 1.8 alpha 2, using LocalExecutor, DAGs do not seem to honor the
> retry_delay parameter, i.e. the retries happen immediately one after the
> other without waiting the specific retry_delay time. However, the number of
> retries is honored. I am testing with the following code:
> from airflow import DAG
> from airflow.operators.bash_operator import BashOperator
> from datetime import datetime, timedelta
> default_args = {
> 'owner': 'airflow',
> 'depends_on_past': False,
> 'start_date': datetime(2016, 10, 5, 19),
> 'end_date': datetime(2016, 10, 6, 19),
> 'email': ['airf...@airflow.com'],
> 'email_on_failure': False,
> 'email_on_retry': False,
> 'retries': 10,
> 'retry_delay': timedelta(0, 500)
> }
> dag = DAG('test_retry_handling_job', default_args=default_args,
> schedule_interval='@once')
> task1 = BashOperator(
> task_id='test_retry_handling_op1',
> bash_command='exit 1',
> dag=dag)
> task2 = BashOperator(
> task_id='test_retry_handling_op2',
> bash_command='exit 1',
> dag=dag)
> task2.set_upstream(task1)
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (AIRFLOW-536) Dags only get one dagrun is scheduled per scheduler loop
[ https://issues.apache.org/jira/browse/AIRFLOW-536?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dan Davydov updated AIRFLOW-536: Description: At the moment Dags only get one dag scheduled per scheduler loop (which can be quite slow, on the order of minutes), but what should happen is that all pending dagruns for a dag should be scheduled. In addition to dagruns being scheduled more quickly for dags that have multiple dag runs queued up (e.g. previously paused dags), fixing this issue will enable a "only_run_latest" option for dags (so e.g. if the dag is paused and then later unpaused, only the most recent ds will be run), as well as facilitating compatibility between backfills and the scheduler (if you run a backfill over 100 dag runs the backfill will be waiting on 100 scheduler cycles). was: At the moment Dags only get one dag scheduled per scheduler loop (which can be quite slow, on the order of minutes), but what should happen is that all pending dagruns for a dag should be scheduled. In addition to dagruns being scheduled more quickly for dags that have multiple dag runs queued up (e.g. paused dags), fixing this issue will enable a "only_run_latest" option for dags (so e.g. if the dag is paused and then later unpaused, only the most recent ds will be run), as well as facilitating compatibility between backfills and the scheduler (if you run a backfill over 100 dag runs the backfill will be waiting on 100 scheduler cycles). > Dags only get one dagrun is scheduled per scheduler loop > > > Key: AIRFLOW-536 > URL: https://issues.apache.org/jira/browse/AIRFLOW-536 > Project: Apache Airflow > Issue Type: Bug >Reporter: Dan Davydov > > At the moment Dags only get one dag scheduled per scheduler loop (which can > be quite slow, on the order of minutes), but what should happen is that all > pending dagruns for a dag should be scheduled. > In addition to dagruns being scheduled more quickly for dags that have > multiple dag runs queued up (e.g. previously paused dags), fixing this issue > will enable a "only_run_latest" option for dags (so e.g. if the dag is paused > and then later unpaused, only the most recent ds will be run), as well as > facilitating compatibility between backfills and the scheduler (if you run a > backfill over 100 dag runs the backfill will be waiting on 100 scheduler > cycles). -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (AIRFLOW-536) Dags only get one dagrun is scheduled per scheduler loop
[ https://issues.apache.org/jira/browse/AIRFLOW-536?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dan Davydov updated AIRFLOW-536: Description: At the moment Dags only get one dag scheduled per scheduler loop (which can be quite slow, on the order of minutes), but what should happen is that all pending dagruns for a dag should be scheduled. In addition to dagruns being scheduled more quickly for dags that have multiple dag runs queued up (e.g. paused dags), fixing this issue will enable a "only_run_latest" option for dags (so e.g. if the dag is paused and then later unpaused, only the most recent ds will be run), as well as facilitating compatibility between backfills and the scheduler (if you run a backfill over 100 dag runs the backfill will be waiting on 100 scheduler cycles). was: At the moment Dags only get one dag scheduled per scheduler loop (which can be quite slow, on the order of minutes), but what should happen is that all pending dagruns for a dag should be scheduled. In addition to dagruns being scheduled more quickly for dags that havem ultiple dag runs queued up (e.g. paused dags), fixing this issue will enable a "only_run_latest" option for dags (so e.g. if the dag is paused and then later unpaused, only the most recent ds will be run), as well as facilitating compatibility between backfills and the scheduler (if you run a backfill over 100 dag runs the backfill will be waiting on 100 scheduler cycles). > Dags only get one dagrun is scheduled per scheduler loop > > > Key: AIRFLOW-536 > URL: https://issues.apache.org/jira/browse/AIRFLOW-536 > Project: Apache Airflow > Issue Type: Bug >Reporter: Dan Davydov > > At the moment Dags only get one dag scheduled per scheduler loop (which can > be quite slow, on the order of minutes), but what should happen is that all > pending dagruns for a dag should be scheduled. > In addition to dagruns being scheduled more quickly for dags that have > multiple dag runs queued up (e.g. paused dags), fixing this issue will enable > a "only_run_latest" option for dags (so e.g. if the dag is paused and then > later unpaused, only the most recent ds will be run), as well as facilitating > compatibility between backfills and the scheduler (if you run a backfill over > 100 dag runs the backfill will be waiting on 100 scheduler cycles). -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (AIRFLOW-672) Allow logs to be piped into another process
Dan Davydov created AIRFLOW-672: --- Summary: Allow logs to be piped into another process Key: AIRFLOW-672 URL: https://issues.apache.org/jira/browse/AIRFLOW-672 Project: Apache Airflow Issue Type: Improvement Components: logging Reporter: Dan Davydov Instead of writing logs to a file Airflow should be able to pipe them to a process similar to how Apache Webserver does it: https://httpd.apache.org/docs/1.3/logs.html#piped . The most important reason for this is to allow log rotation of tasks which can get quite large on the local worker disks. Part of this task could be moving the existing s3 log exporting logic (and other custom logging logic) out of the airflow core. FWIW I have the code ready to add log rotation to Airflow itself, but this isn't as ideal as the piping solution (separation of concerns). If there is interest in getting this merged as a temporary measure I can do so. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (AIRFLOW-674) Ability to add descriptions for DAGs
Dan Davydov created AIRFLOW-674: --- Summary: Ability to add descriptions for DAGs Key: AIRFLOW-674 URL: https://issues.apache.org/jira/browse/AIRFLOW-674 Project: Apache Airflow Issue Type: Improvement Components: webserver Reporter: Dan Davydov Assignee: Dan Davydov Priority: Minor Add a "description" property to DAGs that will be shown in various places in the webserver. This is useful to e.g. see what a DAG does at a high level without having to resort to looking at code comments/git blame. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (AIRFLOW-674) Ability to add descriptions for DAGs
[ https://issues.apache.org/jira/browse/AIRFLOW-674?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dan Davydov updated AIRFLOW-674: Description: Add a "description" property to DAGs that will be shown in various places in the webserver. This is useful to e.g. see what a DAG does at a high level without having to resort to looking at code comments/git blame. Kind of like doc_md but more lightweight and visible without having to drill into task details. (was: Add a "description" property to DAGs that will be shown in various places in the webserver. This is useful to e.g. see what a DAG does at a high level without having to resort to looking at code comments/git blame.) > Ability to add descriptions for DAGs > > > Key: AIRFLOW-674 > URL: https://issues.apache.org/jira/browse/AIRFLOW-674 > Project: Apache Airflow > Issue Type: Improvement > Components: webserver >Reporter: Dan Davydov >Assignee: Dan Davydov >Priority: Minor > > Add a "description" property to DAGs that will be shown in various places in > the webserver. This is useful to e.g. see what a DAG does at a high level > without having to resort to looking at code comments/git blame. Kind of like > doc_md but more lightweight and visible without having to drill into task > details. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Resolved] (AIRFLOW-682) Bump MAX_PERIODS
[ https://issues.apache.org/jira/browse/AIRFLOW-682?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dan Davydov resolved AIRFLOW-682. - Resolution: Fixed > Bump MAX_PERIODS > > > Key: AIRFLOW-682 > URL: https://issues.apache.org/jira/browse/AIRFLOW-682 > Project: Apache Airflow > Issue Type: Bug > Reporter: Dan Davydov > Assignee: Dan Davydov > > It is not possible to mark success on some large DAGs due to the MAX_PERIODS > being set to 1000. We should temporarily bump it up until work can be done to > scale the mark success endpoint much higher. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (AIRFLOW-719) Skipped operations make DAG finish prematurely
[ https://issues.apache.org/jira/browse/AIRFLOW-719?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15930930#comment-15930930 ] Dan Davydov commented on AIRFLOW-719: - [~dgies] We found the same, this is considered a bug and will be reverted in 1.8.1 (this commit should be reverted). > Skipped operations make DAG finish prematurely > --- > > Key: AIRFLOW-719 > URL: https://issues.apache.org/jira/browse/AIRFLOW-719 > Project: Apache Airflow > Issue Type: Sub-task >Reporter: Alex Van Boxel >Assignee: Alex Van Boxel > Fix For: 1.8.0rc5 > > > Since revision 2630361ca24737c28f458825b20ab11c9c996b17 the SKIPPED and > SUCCESS are treated the same, this causes the DAG to finish prematurely when > using a Branching operator where on branch is finished. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Comment Edited] (AIRFLOW-992) Skipped tasks do not propagate correctly
[ https://issues.apache.org/jira/browse/AIRFLOW-992?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15930940#comment-15930940 ] Dan Davydov edited comment on AIRFLOW-992 at 3/18/17 12:39 AM: --- Good point I'll do both edit + comment in the future, haven't tested it yet, we are going to release the latest rc on Monday and then I'll be out for a bit but when I'm back I will try this fix was (Author: aoen): Good point I'll do both edit + comment in the future > Skipped tasks do not propagate correctly > > > Key: AIRFLOW-992 > URL: https://issues.apache.org/jira/browse/AIRFLOW-992 > Project: Apache Airflow > Issue Type: Bug > Reporter: Dan Davydov >Priority: Critical > > We are seeing skipped tasks not being propagated correctly: > E.g. > A->B > `-->C > Task A depends on task B and C > If B gets skipped and C gets run then: > Expected: > A will get skipped > EDIT: Upon further investigation this was caused by a change in the semantics > of ALL_SUCCESS, which I have these feelings about: > Intuitively you would expect to skip any task that has dependencies that > weren't run by default, i.e. the trigger rule is called ALL_SUCCESS and > skipped tasks are not successful ones, and that was also the old behavior in > 1.7.3. > This is going to break some use cases which could be alright, but I feel > these new semantics make less sense than before so it's a bad reason to break > existing use cases. > I will get started on a PR for a new ALL_SUCCESS_NOT_SKIPPED trigger rule but > again I feel this is hacky and really we should have the old ALL_SUCCESS > (default) and a new ALL_SUCCESS_OR_SKIPPED trigger rule if desired. > Actual: > A gets run > It's possible the reason for the change in semantics (which was to fix a bug) > is fixed already by https://github.com/apache/incubator-airflow/pull/2154 and > the only thing we need to do is revert > https://github.com/apache/incubator-airflow/pull/2125 > [~bolke] -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Updated] (AIRFLOW-992) Skipped tasks do not propagate correctly
[ https://issues.apache.org/jira/browse/AIRFLOW-992?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dan Davydov updated AIRFLOW-992: Description: We are seeing skipped tasks not being propagated correctly: E.g. A->B `-->C Task A depends on task B and C If B gets skipped and C gets run then: Expected: A will get skipped EDIT: Upon further investigation this was caused by a change in the semantics of ALL_SUCCESS, which I have these feelings about: Intuitively you would expect to skip any task that has dependencies that weren't run by default, i.e. the trigger rule is called ALL_SUCCESS and skipped tasks are not successful ones, and that was also the old behavior in 1.7.3. This is going to break some use cases which could be alright, but I feel these new semantics make less sense than before so it's a bad reason to break existing use cases. I will get started on a PR for a new ALL_SUCCESS_NOT_SKIPPED trigger rule but again I feel this is hacky and really we should have the old ALL_SUCCESS (default) and a new ALL_SUCCESS_OR_SKIPPED trigger rule if desired. Actual: A gets run It's possible the reason for the change in semantics (which was to fix a bug) is fixed already by https://github.com/apache/incubator-airflow/pull/2154 and the only thing we need to do is revert https://github.com/apache/incubator-airflow/pull/2125 [~bolke] was: We are seeing skipped tasks not being propagated correctly: E.g. A->B `-->C Task A depends on task B and C If B gets skipped and C gets run then: Expected: A will get skipped EDIT: Upon further investigation this was caused by a change in the semantics of ALL_SUCCESS, which I have these feelings about: Intuitively you would expect to skip any task that has dependencies that weren't run by default, i.e. the trigger rule is called ALL_SUCCESS and skipped tasks are not successful ones, and that was also the old behavior in 1.7.3. This is going to break some use cases which could be alright, but I feel these new semantics make less sense than before so it's a bad reason to break existing use cases. I will get started on a PR for a new ALL_SUCCESS_NOT_SKIPPED trigger rule but again I feel this is hacky and really we should have the old ALL_SUCCESS (default) and a new ALL_SUCCESS_OR_SKIPPED trigger rule if desired. Actual: A gets run [~bolke] > Skipped tasks do not propagate correctly > > > Key: AIRFLOW-992 > URL: https://issues.apache.org/jira/browse/AIRFLOW-992 > Project: Apache Airflow > Issue Type: Bug >Reporter: Dan Davydov >Priority: Critical > > We are seeing skipped tasks not being propagated correctly: > E.g. > A->B > `-->C > Task A depends on task B and C > If B gets skipped and C gets run then: > Expected: > A will get skipped > EDIT: Upon further investigation this was caused by a change in the semantics > of ALL_SUCCESS, which I have these feelings about: > Intuitively you would expect to skip any task that has dependencies that > weren't run by default, i.e. the trigger rule is called ALL_SUCCESS and > skipped tasks are not successful ones, and that was also the old behavior in > 1.7.3. > This is going to break some use cases which could be alright, but I feel > these new semantics make less sense than before so it's a bad reason to break > existing use cases. > I will get started on a PR for a new ALL_SUCCESS_NOT_SKIPPED trigger rule but > again I feel this is hacky and really we should have the old ALL_SUCCESS > (default) and a new ALL_SUCCESS_OR_SKIPPED trigger rule if desired. > Actual: > A gets run > It's possible the reason for the change in semantics (which was to fix a bug) > is fixed already by https://github.com/apache/incubator-airflow/pull/2154 and > the only thing we need to do is revert > https://github.com/apache/incubator-airflow/pull/2125 > [~bolke] -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (AIRFLOW-992) Skipped tasks do not propagate correctly
[ https://issues.apache.org/jira/browse/AIRFLOW-992?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15930940#comment-15930940 ] Dan Davydov commented on AIRFLOW-992: - Good point I'll do both edit + comment in the future > Skipped tasks do not propagate correctly > > > Key: AIRFLOW-992 > URL: https://issues.apache.org/jira/browse/AIRFLOW-992 > Project: Apache Airflow > Issue Type: Bug > Reporter: Dan Davydov >Priority: Critical > > We are seeing skipped tasks not being propagated correctly: > E.g. > A->B > `-->C > Task A depends on task B and C > If B gets skipped and C gets run then: > Expected: > A will get skipped > EDIT: Upon further investigation this was caused by a change in the semantics > of ALL_SUCCESS, which I have these feelings about: > Intuitively you would expect to skip any task that has dependencies that > weren't run by default, i.e. the trigger rule is called ALL_SUCCESS and > skipped tasks are not successful ones, and that was also the old behavior in > 1.7.3. > This is going to break some use cases which could be alright, but I feel > these new semantics make less sense than before so it's a bad reason to break > existing use cases. > I will get started on a PR for a new ALL_SUCCESS_NOT_SKIPPED trigger rule but > again I feel this is hacky and really we should have the old ALL_SUCCESS > (default) and a new ALL_SUCCESS_OR_SKIPPED trigger rule if desired. > Actual: > A gets run > It's possible the reason for the change in semantics (which was to fix a bug) > is fixed already by https://github.com/apache/incubator-airflow/pull/2154 and > the only thing we need to do is revert > https://github.com/apache/incubator-airflow/pull/2125 > [~bolke] -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Updated] (AIRFLOW-983) Make trigger rules more explicit regarding success vs skipped
[ https://issues.apache.org/jira/browse/AIRFLOW-983?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dan Davydov updated AIRFLOW-983: Priority: Blocker (was: Major) > Make trigger rules more explicit regarding success vs skipped > - > > Key: AIRFLOW-983 > URL: https://issues.apache.org/jira/browse/AIRFLOW-983 > Project: Apache Airflow > Issue Type: Improvement > Components: dependencies >Reporter: Daniel Huang >Priority: Blocker > > Since AIRFLOW-719, the trigger rules all_success/one_success include both > success and skipped states. > We should probably make ALL_SUCCESS strictly success (again) and add a > separate ALL_SUCCESS_OR_SKIPPED/ALL_FAILED_OR_SKIPPED. ALL_SUCCESS_OR_SKIPPED > may be a more appropriate default trigger rule as well. Otherwise, we need to > note in LatestOnly/ShortCircuit/Branch operators of the appropriate trigger > rule to use there. > Some previous discussion in > https://github.com/apache/incubator-airflow/pull/1961 -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Created] (AIRFLOW-1017) get_task_instance should return None instead of throw an exception for non-existent TIs
Dan Davydov created AIRFLOW-1017:
Summary: get_task_instance should return None instead of throw an
exception for non-existent TIs
Key: AIRFLOW-1017
URL: https://issues.apache.org/jira/browse/AIRFLOW-1017
Project: Apache Airflow
Issue Type: Bug
Reporter: Dan Davydov
Assignee: Dan Davydov
Priority: Critical
We were seeing errors in our scheduler like the following due to this issue:
File "/usr/local/lib/python2.7/dist-packages/sqlalchemy/orm/query.py", line
2796, in one
raise orm_exc.NoResultFound("No row was found for one()")
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Updated] (AIRFLOW-983) Make trigger rules more explicit regarding success vs skipped
[ https://issues.apache.org/jira/browse/AIRFLOW-983?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dan Davydov updated AIRFLOW-983: Priority: Critical (was: Blocker) > Make trigger rules more explicit regarding success vs skipped > - > > Key: AIRFLOW-983 > URL: https://issues.apache.org/jira/browse/AIRFLOW-983 > Project: Apache Airflow > Issue Type: Improvement > Components: dependencies >Reporter: Daniel Huang >Priority: Critical > > Since AIRFLOW-719, the trigger rules all_success/one_success include both > success and skipped states. > We should probably make ALL_SUCCESS strictly success (again) and add a > separate ALL_SUCCESS_OR_SKIPPED/ALL_FAILED_OR_SKIPPED. ALL_SUCCESS_OR_SKIPPED > may be a more appropriate default trigger rule as well. Otherwise, we need to > note in LatestOnly/ShortCircuit/Branch operators of the appropriate trigger > rule to use there. > Some previous discussion in > https://github.com/apache/incubator-airflow/pull/1961 -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Created] (AIRFLOW-1019) active_dagruns shouldn't include paused DAGs
Dan Davydov created AIRFLOW-1019: Summary: active_dagruns shouldn't include paused DAGs Key: AIRFLOW-1019 URL: https://issues.apache.org/jira/browse/AIRFLOW-1019 Project: Apache Airflow Issue Type: Bug Reporter: Dan Davydov Priority: Critical Since 1.8.0 Airflow resets orphaned tasks (tasks that are in the DB but not in the executor's memory). The problem is that Airflow counts dagruns in paused DAGs as running as long as the dagruns state is running. Instead we should join against non-paused DAGs everywhere we calculate active dagruns (e.g. in _process_task_instances in the Scheduler class in jobs.py). If there are enough paused DAGs it brings the scheduler to a halt especially on scheduler restarts. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Created] (AIRFLOW-992) Skipped tasks do not propagate correctly
Dan Davydov created AIRFLOW-992: --- Summary: Skipped tasks do not propagate correctly Key: AIRFLOW-992 URL: https://issues.apache.org/jira/browse/AIRFLOW-992 Project: Apache Airflow Issue Type: Bug Reporter: Dan Davydov Priority: Critical We are seeing skipped tasks not being propagated correctly: E.g. A->B `-->C Task A depends on task B and C If B gets skipped and C gets run then: Expected: A will get skipped Actual: A gets run [~bolke] -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (AIRFLOW-992) Skipped tasks do not propagate correctly
[ https://issues.apache.org/jira/browse/AIRFLOW-992?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15927332#comment-15927332 ] Dan Davydov commented on AIRFLOW-992: - This is the expected behavior after the semantics changes made here: https://github.com/apache/incubator-airflow/pull/2125 and later documented/clarified here: https://github.com/apache/incubator-airflow/pull/2151/files > Skipped tasks do not propagate correctly > > > Key: AIRFLOW-992 > URL: https://issues.apache.org/jira/browse/AIRFLOW-992 > Project: Apache Airflow > Issue Type: Bug > Reporter: Dan Davydov >Priority: Critical > > We are seeing skipped tasks not being propagated correctly: > E.g. > A->B > `-->C > Task A depends on task B and C > If B gets skipped and C gets run then: > Expected: > A will get skipped > EDIT: Upon further investigation this was caused by a change in the semantics > of ALL_SUCCESS, which I have these feelings about: > Intuitively you would expect to skip any task that has dependencies that > weren't run by default, i.e. the trigger rule is called ALL_SUCCESS and > skipped tasks are not successful ones, and that was also the old behavior in > 1.7.3. > This is going to break some use cases which could be alright, but I feel > these new semantics make less sense than before so it's a bad reason to break > existing use cases. > I will get started on a PR for a new ALL_SUCCESS_NOT_SKIPPED trigger rule but > again I feel this is hacky and really we should have the old ALL_SUCCESS > (default) and a new ALL_SUCCESS_OR_SKIPPED trigger rule if desired. > Actual: > A gets run > [~bolke] -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Created] (AIRFLOW-1024) Handle CeleryExecutor errors gracefully
Dan Davydov created AIRFLOW-1024: Summary: Handle CeleryExecutor errors gracefully Key: AIRFLOW-1024 URL: https://issues.apache.org/jira/browse/AIRFLOW-1024 Project: Apache Airflow Issue Type: Bug Reporter: Dan Davydov If the Airflow celery executor receives a bad response from a worker (e.g. unpickled response), then it will crash the scheduler and cause it to restart. We should code defensively around the interactions with celery so that we just log errors instead of crashing the scheduler. It might makes sense to make the try catches one level higher (to catch errors from all executors), but this needs some investigation. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Updated] (AIRFLOW-983) Make trigger rules more explicit regarding success vs skipped
[ https://issues.apache.org/jira/browse/AIRFLOW-983?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dan Davydov updated AIRFLOW-983: Description: Since AIRFLOW-719, the trigger rules all_success/one_success include both success and skipped states. We should probably make ALL_SUCCESS strictly success (again) and add a separate ALL_SUCCESS_OR_SKIPPED/ALL_FAILED_OR_SKIPPED. ALL_SUCCESS_OR_SKIPPED may be a more appropriate default trigger rule as well. Otherwise, we need to note in LatestOnly/ShortCircuit/Branch operators of the appropriate trigger rule to use there. Some previous discussion in https://github.com/apache/incubator-airflow/pull/1961 It's possible the reason for the change in semantics (which was to fix a bug) is fixed already by https://github.com/apache/incubator-airflow/pull/2154 and the only thing we need to do is revert https://github.com/apache/incubator-airflow/pull/2125 was: Since AIRFLOW-719, the trigger rules all_success/one_success include both success and skipped states. We should probably make ALL_SUCCESS strictly success (again) and add a separate ALL_SUCCESS_OR_SKIPPED/ALL_FAILED_OR_SKIPPED. ALL_SUCCESS_OR_SKIPPED may be a more appropriate default trigger rule as well. Otherwise, we need to note in LatestOnly/ShortCircuit/Branch operators of the appropriate trigger rule to use there. Some previous discussion in https://github.com/apache/incubator-airflow/pull/1961 > Make trigger rules more explicit regarding success vs skipped > - > > Key: AIRFLOW-983 > URL: https://issues.apache.org/jira/browse/AIRFLOW-983 > Project: Apache Airflow > Issue Type: Improvement > Components: dependencies >Reporter: Daniel Huang >Priority: Critical > > Since AIRFLOW-719, the trigger rules all_success/one_success include both > success and skipped states. > We should probably make ALL_SUCCESS strictly success (again) and add a > separate ALL_SUCCESS_OR_SKIPPED/ALL_FAILED_OR_SKIPPED. ALL_SUCCESS_OR_SKIPPED > may be a more appropriate default trigger rule as well. Otherwise, we need to > note in LatestOnly/ShortCircuit/Branch operators of the appropriate trigger > rule to use there. > Some previous discussion in > https://github.com/apache/incubator-airflow/pull/1961 > It's possible the reason for the change in semantics (which was to fix a bug) > is fixed already by https://github.com/apache/incubator-airflow/pull/2154 and > the only thing we need to do is revert > https://github.com/apache/incubator-airflow/pull/2125 -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (AIRFLOW-983) Make trigger rules more explicit regarding success vs skipped
[ https://issues.apache.org/jira/browse/AIRFLOW-983?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15935420#comment-15935420 ] Dan Davydov commented on AIRFLOW-983: - It's possible the reason for the change in semantics (which was to fix a bug) is fixed already by https://github.com/apache/incubator-airflow/pull/2154 and the only thing we need to do is revert https://github.com/apache/incubator-airflow/pull/2125] > Make trigger rules more explicit regarding success vs skipped > - > > Key: AIRFLOW-983 > URL: https://issues.apache.org/jira/browse/AIRFLOW-983 > Project: Apache Airflow > Issue Type: Improvement > Components: dependencies >Reporter: Daniel Huang >Priority: Critical > > Since AIRFLOW-719, the trigger rules all_success/one_success include both > success and skipped states. > We should probably make ALL_SUCCESS strictly success (again) and add a > separate ALL_SUCCESS_OR_SKIPPED/ALL_FAILED_OR_SKIPPED. ALL_SUCCESS_OR_SKIPPED > may be a more appropriate default trigger rule as well. Otherwise, we need to > note in LatestOnly/ShortCircuit/Branch operators of the appropriate trigger > rule to use there. > Some previous discussion in > https://github.com/apache/incubator-airflow/pull/1961 > It's possible the reason for the change in semantics (which was to fix a bug) > is fixed already by https://github.com/apache/incubator-airflow/pull/2154 and > the only thing we need to do is revert > https://github.com/apache/incubator-airflow/pull/2125 -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (AIRFLOW-983) Make trigger rules more explicit regarding success vs skipped
[ https://issues.apache.org/jira/browse/AIRFLOW-983?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15935444#comment-15935444 ] Dan Davydov commented on AIRFLOW-983: - No, and I'll be out for a couple of weeks unfortunately. > Make trigger rules more explicit regarding success vs skipped > - > > Key: AIRFLOW-983 > URL: https://issues.apache.org/jira/browse/AIRFLOW-983 > Project: Apache Airflow > Issue Type: Improvement > Components: dependencies >Reporter: Daniel Huang >Priority: Critical > > Since AIRFLOW-719, the trigger rules all_success/one_success include both > success and skipped states. > We should probably make ALL_SUCCESS strictly success (again) and add a > separate ALL_SUCCESS_OR_SKIPPED/ALL_FAILED_OR_SKIPPED. ALL_SUCCESS_OR_SKIPPED > may be a more appropriate default trigger rule as well. Otherwise, we need to > note in LatestOnly/ShortCircuit/Branch operators of the appropriate trigger > rule to use there. > Some previous discussion in > https://github.com/apache/incubator-airflow/pull/1961 > It's possible the reason for the change in semantics (which was to fix a bug) > is fixed already by https://github.com/apache/incubator-airflow/pull/2154 and > the only thing we need to do is revert > https://github.com/apache/incubator-airflow/pull/2125 -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (AIRFLOW-1019) active_dagruns shouldn't include paused DAGs
[ https://issues.apache.org/jira/browse/AIRFLOW-1019?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15965082#comment-15965082 ] Dan Davydov commented on AIRFLOW-1019: -- Probably not a blocker, just a performance degradation. > active_dagruns shouldn't include paused DAGs > > > Key: AIRFLOW-1019 > URL: https://issues.apache.org/jira/browse/AIRFLOW-1019 > Project: Apache Airflow > Issue Type: Improvement > Components: scheduler >Affects Versions: 1.8.0 > Reporter: Dan Davydov >Priority: Critical > Fix For: 1.8.1 > > > Since 1.8.0 Airflow resets orphaned tasks (tasks that are in the DB but not > in the executor's memory). The problem is that Airflow counts dagruns in > paused DAGs as running as long as the dagruns state is running. Instead we > should join against non-paused DAGs everywhere we calculate active dagruns > (e.g. in _process_task_instances in the Scheduler class in jobs.py). If there > are enough paused DAGs it brings the scheduler to a halt especially on > scheduler restarts. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (AIRFLOW-983) Make trigger rules more explicit regarding success vs skipped
[ https://issues.apache.org/jira/browse/AIRFLOW-983?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15965060#comment-15965060 ] Dan Davydov commented on AIRFLOW-983: - It seems like the commit was reverted but I'm not sure if testing was done: https://github.com/apache/incubator-airflow/pull/2195? If not I can try to take a look soon but just wanted to confirm first [~bolke] > Make trigger rules more explicit regarding success vs skipped > - > > Key: AIRFLOW-983 > URL: https://issues.apache.org/jira/browse/AIRFLOW-983 > Project: Apache Airflow > Issue Type: Improvement > Components: dependencies >Affects Versions: 1.8.1 >Reporter: Daniel Huang >Assignee: Bolke de Bruin > > We should add a separate ALL_SUCCESS_OR_SKIPPED/ALL_FAILED_OR_SKIPPED. > ALL_SUCCESS_OR_SKIPPED may be a more appropriate default trigger rule as > well. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Updated] (AIRFLOW-940) Failing to unencrypt one variable shouldn't break the admin/variable page
[
https://issues.apache.org/jira/browse/AIRFLOW-940?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Dan Davydov updated AIRFLOW-940:
Labels: beginner starter (was: )
> Failing to unencrypt one variable shouldn't break the admin/variable page
> -
>
> Key: AIRFLOW-940
> URL: https://issues.apache.org/jira/browse/AIRFLOW-940
> Project: Apache Airflow
> Issue Type: Bug
> Components: webserver
> Reporter: Dan Davydov
> Labels: beginner, starter
>
> At the moment failing to decrypt one of the airflow variables in the database
> using the fernet key will cause the admin/variable page to crash. Instead
> there should be an error message for only the broken variables at the top of
> the page, and the other ones should still be loaded and displayed.
> This is what the trace on admin/variable currently looks like:
> {code}
> Traceback (most recent call last):
> File "/usr/local/lib/python2.7/dist-packages/flask/app.py", line 1817, in
> wsgi_app
> response = self.full_dispatch_request()
> File "/usr/local/lib/python2.7/dist-packages/flask/app.py", line 1477, in
> full_dispatch_request
> rv = self.handle_user_exception(e)
> File "/usr/local/lib/python2.7/dist-packages/flask/app.py", line 1381, in
> handle_user_exception
> reraise(exc_type, exc_value, tb)
> File "/usr/local/lib/python2.7/dist-packages/flask/app.py", line 1475, in
> full_dispatch_request
> rv = self.dispatch_request()
> File "/usr/local/lib/python2.7/dist-packages/flask/app.py", line 1461, in
> dispatch_request
> return self.view_functions[rule.endpoint](**req.view_args)
> File "/usr/local/lib/python2.7/dist-packages/flask_admin/base.py", line 68,
> in inner
> return self._run_view(f, *args, **kwargs)
> File "/usr/local/lib/python2.7/dist-packages/flask_admin/base.py", line
> 367, in _run_view
> return fn(self, *args, **kwargs)
> File "/usr/local/lib/python2.7/dist-packages/flask_admin/model/base.py",
> line 1814, in index_view
> return_url=self._get_list_url(view_args),
> File "/usr/local/lib/python2.7/dist-packages/flask_admin/base.py", line
> 307, in render
> return render_template(template, **kwargs)
> File "/usr/local/lib/python2.7/dist-packages/flask/templating.py", line
> 128, in render_template
> context, ctx.app)
> File "/usr/local/lib/python2.7/dist-packages/flask/templating.py", line
> 110, in _render
> rv = template.render(context)
> File "/usr/local/lib/python2.7/dist-packages/jinja2/environment.py", line
> 989, in render
> return self.environment.handle_exception(exc_info, True)
> File "/usr/local/lib/python2.7/dist-packages/jinja2/environment.py", line
> 754, in handle_exception
> reraise(exc_type, exc_value, tb)
> File
> "/usr/local/lib/python2.7/dist-packages/airflow/www/templates/airflow/variable_list.html",
> line 18, in top-level template code
> {% extends 'admin/model/list.html' %}
> File
> "/usr/local/lib/python2.7/dist-packages/flask_admin/templates/bootstrap3/admin/model/list.html",
> line 5, in top-level template code
> {% import 'admin/actions.html' as actionlib with context %}
> File
> "/usr/local/lib/python2.7/dist-packages/airflow/www/templates/admin/master.html",
> line 18, in top-level template code
> {% extends 'admin/base.html' %}
> File
> "/usr/local/lib/python2.7/dist-packages/flask_admin/templates/bootstrap3/admin/base.html",
> line 30, in top-level template code
> {% block page_body %}
> File
> "/usr/local/lib/python2.7/dist-packages/airflow/www/templates/admin/master.html",
> line 96, in block "page_body"
> {% block body %}
> File
> "/usr/local/lib/python2.7/dist-packages/flask_admin/templates/bootstrap3/admin/model/list.html",
> line 62, in block "body"
> {% block model_list_table %}
> File
> "/usr/local/lib/python2.7/dist-packages/flask_admin/templates/bootstrap3/admin/model/list.html",
> line 110, in block "model_list_table"
> {% block list_row scoped %}
> File
> "/usr/local/lib/python2.7/dist-packages/flask_admin/templates/bootstrap3/admin/model/list.html",
> line 165, in block "list_row"
> {{ get_value(row, c) }}
> File "/usr/local/lib/python2.7/dist-packages/flask_admin/model/base.py",
> line 1666, in get_
[jira] [Created] (AIRFLOW-940) Failing to unencrypt one variable shouldn't break the admin/variable page
Dan Davydov created AIRFLOW-940:
---
Summary: Failing to unencrypt one variable shouldn't break the
admin/variable page
Key: AIRFLOW-940
URL: https://issues.apache.org/jira/browse/AIRFLOW-940
Project: Apache Airflow
Issue Type: Bug
Components: webserver
Reporter: Dan Davydov
At the moment failing to decrypt one of the airflow variables in the database
using the fernet key will cause the admin/variable page to crash. Instead there
should be an error message for only the broken variables at the top of the
page, and the other ones should still be loaded and displayed.
This is what the trace on admin/variable currently looks like:
{code}
Traceback (most recent call last):
File "/usr/local/lib/python2.7/dist-packages/flask/app.py", line 1817, in
wsgi_app
response = self.full_dispatch_request()
File "/usr/local/lib/python2.7/dist-packages/flask/app.py", line 1477, in
full_dispatch_request
rv = self.handle_user_exception(e)
File "/usr/local/lib/python2.7/dist-packages/flask/app.py", line 1381, in
handle_user_exception
reraise(exc_type, exc_value, tb)
File "/usr/local/lib/python2.7/dist-packages/flask/app.py", line 1475, in
full_dispatch_request
rv = self.dispatch_request()
File "/usr/local/lib/python2.7/dist-packages/flask/app.py", line 1461, in
dispatch_request
return self.view_functions[rule.endpoint](**req.view_args)
File "/usr/local/lib/python2.7/dist-packages/flask_admin/base.py", line 68,
in inner
return self._run_view(f, *args, **kwargs)
File "/usr/local/lib/python2.7/dist-packages/flask_admin/base.py", line 367,
in _run_view
return fn(self, *args, **kwargs)
File "/usr/local/lib/python2.7/dist-packages/flask_admin/model/base.py", line
1814, in index_view
return_url=self._get_list_url(view_args),
File "/usr/local/lib/python2.7/dist-packages/flask_admin/base.py", line 307,
in render
return render_template(template, **kwargs)
File "/usr/local/lib/python2.7/dist-packages/flask/templating.py", line 128,
in render_template
context, ctx.app)
File "/usr/local/lib/python2.7/dist-packages/flask/templating.py", line 110,
in _render
rv = template.render(context)
File "/usr/local/lib/python2.7/dist-packages/jinja2/environment.py", line
989, in render
return self.environment.handle_exception(exc_info, True)
File "/usr/local/lib/python2.7/dist-packages/jinja2/environment.py", line
754, in handle_exception
reraise(exc_type, exc_value, tb)
File
"/usr/local/lib/python2.7/dist-packages/airflow/www/templates/airflow/variable_list.html",
line 18, in top-level template code
{% extends 'admin/model/list.html' %}
File
"/usr/local/lib/python2.7/dist-packages/flask_admin/templates/bootstrap3/admin/model/list.html",
line 5, in top-level template code
{% import 'admin/actions.html' as actionlib with context %}
File
"/usr/local/lib/python2.7/dist-packages/airflow/www/templates/admin/master.html",
line 18, in top-level template code
{% extends 'admin/base.html' %}
File
"/usr/local/lib/python2.7/dist-packages/flask_admin/templates/bootstrap3/admin/base.html",
line 30, in top-level template code
{% block page_body %}
File
"/usr/local/lib/python2.7/dist-packages/airflow/www/templates/admin/master.html",
line 96, in block "page_body"
{% block body %}
File
"/usr/local/lib/python2.7/dist-packages/flask_admin/templates/bootstrap3/admin/model/list.html",
line 62, in block "body"
{% block model_list_table %}
File
"/usr/local/lib/python2.7/dist-packages/flask_admin/templates/bootstrap3/admin/model/list.html",
line 110, in block "model_list_table"
{% block list_row scoped %}
File
"/usr/local/lib/python2.7/dist-packages/flask_admin/templates/bootstrap3/admin/model/list.html",
line 165, in block "list_row"
{{ get_value(row, c) }}
File "/usr/local/lib/python2.7/dist-packages/flask_admin/model/base.py", line
1666, in get_list_value
self.column_type_formatters,
File "/usr/local/lib/python2.7/dist-packages/flask_admin/model/base.py", line
1631, in _get_list_value
value = column_fmt(self, context, model, name)
File "/usr/local/lib/python2.7/dist-packages/airflow/www/views.py", line
2146, in hidden_field_formatter
return getattr(model, name)
File "/usr/local/lib/python2.7/dist-packages/sqlalchemy/orm/attributes.py",
line 293, in __get__
return self.descriptor.__get__(instance, owner)
File "/usr/local/lib/python2.7/dist-packages/airflow/models.py", line 3487,
in get_val
return FERNET.decrypt(bytes(self._val, 'utf-8')).decode()
File "/usr/local/lib/python2.7/dist-packages/cryptography/fernet.py", line
82, in decrypt
raise InvalidToken
InvalidToken
{quote}
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Updated] (AIRFLOW-940) Failing to unencrypt one variable shouldn't break the admin/variable page
[
https://issues.apache.org/jira/browse/AIRFLOW-940?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Dan Davydov updated AIRFLOW-940:
Description:
At the moment failing to decrypt one of the airflow variables in the database
using the fernet key will cause the admin/variable page to crash. Instead there
should be an error message for only the broken variables at the top of the
page, and the other ones should still be loaded and displayed.
This is what the trace on admin/variable currently looks like:
{code}
Traceback (most recent call last):
File "/usr/local/lib/python2.7/dist-packages/flask/app.py", line 1817, in
wsgi_app
response = self.full_dispatch_request()
File "/usr/local/lib/python2.7/dist-packages/flask/app.py", line 1477, in
full_dispatch_request
rv = self.handle_user_exception(e)
File "/usr/local/lib/python2.7/dist-packages/flask/app.py", line 1381, in
handle_user_exception
reraise(exc_type, exc_value, tb)
File "/usr/local/lib/python2.7/dist-packages/flask/app.py", line 1475, in
full_dispatch_request
rv = self.dispatch_request()
File "/usr/local/lib/python2.7/dist-packages/flask/app.py", line 1461, in
dispatch_request
return self.view_functions[rule.endpoint](**req.view_args)
File "/usr/local/lib/python2.7/dist-packages/flask_admin/base.py", line 68,
in inner
return self._run_view(f, *args, **kwargs)
File "/usr/local/lib/python2.7/dist-packages/flask_admin/base.py", line 367,
in _run_view
return fn(self, *args, **kwargs)
File "/usr/local/lib/python2.7/dist-packages/flask_admin/model/base.py", line
1814, in index_view
return_url=self._get_list_url(view_args),
File "/usr/local/lib/python2.7/dist-packages/flask_admin/base.py", line 307,
in render
return render_template(template, **kwargs)
File "/usr/local/lib/python2.7/dist-packages/flask/templating.py", line 128,
in render_template
context, ctx.app)
File "/usr/local/lib/python2.7/dist-packages/flask/templating.py", line 110,
in _render
rv = template.render(context)
File "/usr/local/lib/python2.7/dist-packages/jinja2/environment.py", line
989, in render
return self.environment.handle_exception(exc_info, True)
File "/usr/local/lib/python2.7/dist-packages/jinja2/environment.py", line
754, in handle_exception
reraise(exc_type, exc_value, tb)
File
"/usr/local/lib/python2.7/dist-packages/airflow/www/templates/airflow/variable_list.html",
line 18, in top-level template code
{% extends 'admin/model/list.html' %}
File
"/usr/local/lib/python2.7/dist-packages/flask_admin/templates/bootstrap3/admin/model/list.html",
line 5, in top-level template code
{% import 'admin/actions.html' as actionlib with context %}
File
"/usr/local/lib/python2.7/dist-packages/airflow/www/templates/admin/master.html",
line 18, in top-level template code
{% extends 'admin/base.html' %}
File
"/usr/local/lib/python2.7/dist-packages/flask_admin/templates/bootstrap3/admin/base.html",
line 30, in top-level template code
{% block page_body %}
File
"/usr/local/lib/python2.7/dist-packages/airflow/www/templates/admin/master.html",
line 96, in block "page_body"
{% block body %}
File
"/usr/local/lib/python2.7/dist-packages/flask_admin/templates/bootstrap3/admin/model/list.html",
line 62, in block "body"
{% block model_list_table %}
File
"/usr/local/lib/python2.7/dist-packages/flask_admin/templates/bootstrap3/admin/model/list.html",
line 110, in block "model_list_table"
{% block list_row scoped %}
File
"/usr/local/lib/python2.7/dist-packages/flask_admin/templates/bootstrap3/admin/model/list.html",
line 165, in block "list_row"
{{ get_value(row, c) }}
File "/usr/local/lib/python2.7/dist-packages/flask_admin/model/base.py", line
1666, in get_list_value
self.column_type_formatters,
File "/usr/local/lib/python2.7/dist-packages/flask_admin/model/base.py", line
1631, in _get_list_value
value = column_fmt(self, context, model, name)
File "/usr/local/lib/python2.7/dist-packages/airflow/www/views.py", line
2146, in hidden_field_formatter
return getattr(model, name)
File "/usr/local/lib/python2.7/dist-packages/sqlalchemy/orm/attributes.py",
line 293, in __get__
return self.descriptor.__get__(instance, owner)
File "/usr/local/lib/python2.7/dist-packages/airflow/models.py", line 3487,
in get_val
return FERNET.decrypt(bytes(self._val, 'utf-8')).decode()
File "/usr/local/lib/python2.7/dist-packages/cryptography/fernet.py", line
82, in decrypt
raise InvalidToken
InvalidToken
{code}
was:
At the moment failing to decrypt one of the airflow
[jira] [Commented] (AIRFLOW-976) Mark success running task causes it to fail
[ https://issues.apache.org/jira/browse/AIRFLOW-976?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15922930#comment-15922930 ] Dan Davydov commented on AIRFLOW-976: - It is but I wouldn't say it's a blocker for release. > Mark success running task causes it to fail > --- > > Key: AIRFLOW-976 > URL: https://issues.apache.org/jira/browse/AIRFLOW-976 > Project: Apache Airflow > Issue Type: Bug > Reporter: Dan Davydov > > Marking success on a running task in the UI causes it to fail. > Expected Behavior: > Task instance is killed and marked as successful > Actual Behavior: > Task instance is killed and marked as failed > [~saguziel] [~bolke] -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Created] (AIRFLOW-976) Mark success running task causes it to fail
Dan Davydov created AIRFLOW-976: --- Summary: Mark success running task causes it to fail Key: AIRFLOW-976 URL: https://issues.apache.org/jira/browse/AIRFLOW-976 Project: Apache Airflow Issue Type: Bug Reporter: Dan Davydov Marking success on a running task in the UI causes it to fail. Expected Behavior: Task instance is killed and marked as successful Actual Behavior: Task instance is killed and marked as failed [~saguziel] [~bolke] -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (AIRFLOW-932) Backfills delete existing task instances and mark them as removed
[ https://issues.apache.org/jira/browse/AIRFLOW-932?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15893135#comment-15893135 ] Dan Davydov commented on AIRFLOW-932: - I think tasks should only be verified/removed in the scheduler dagrun loop. That would avoid this issue if I understand correctly. > Backfills delete existing task instances and mark them as removed > - > > Key: AIRFLOW-932 > URL: https://issues.apache.org/jira/browse/AIRFLOW-932 > Project: Apache Airflow > Issue Type: Sub-task > Components: backfill > Reporter: Dan Davydov >Priority: Blocker > > I'm still investigating. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (AIRFLOW-932) Backfills delete existing task instances and mark them as removed
[ https://issues.apache.org/jira/browse/AIRFLOW-932?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15893157#comment-15893157 ] Dan Davydov commented on AIRFLOW-932: - If the goal is to eventually move all the logic to the scheduler I think it's OK (but maybe there are some edge cases I'm not considering). It will also simplify the logic/reasoning around the REMOVED state and what could have created it. > Backfills delete existing task instances and mark them as removed > - > > Key: AIRFLOW-932 > URL: https://issues.apache.org/jira/browse/AIRFLOW-932 > Project: Apache Airflow > Issue Type: Sub-task > Components: backfill > Reporter: Dan Davydov >Assignee: Bolke de Bruin >Priority: Blocker > > I'm still investigating. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (AIRFLOW-932) Backfills delete existing task instances and mark them as removed
[ https://issues.apache.org/jira/browse/AIRFLOW-932?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15893158#comment-15893158 ] Dan Davydov commented on AIRFLOW-932: - Thanks for fixing! > Backfills delete existing task instances and mark them as removed > - > > Key: AIRFLOW-932 > URL: https://issues.apache.org/jira/browse/AIRFLOW-932 > Project: Apache Airflow > Issue Type: Sub-task > Components: backfill > Reporter: Dan Davydov >Assignee: Bolke de Bruin >Priority: Blocker > > I'm still investigating. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Created] (AIRFLOW-936) Ability to clear/mark success all tasks in a dagrun in the UI
Dan Davydov created AIRFLOW-936: --- Summary: Ability to clear/mark success all tasks in a dagrun in the UI Key: AIRFLOW-936 URL: https://issues.apache.org/jira/browse/AIRFLOW-936 Project: Apache Airflow Issue Type: New Feature Components: webserver Reporter: Dan Davydov The same way you can clear/mark success a task instance recursively, you should be able to do for a dagrun circle in the UI in the tree view. Users have been resorting to creating dummy root tasks for their DAGs just to clear the other tasks in a dagrun which is a hack. This would require changing the current behavior of clicking a dagrun (from editing the dagrun), to having a popup similar to what you get when you click on a task instance. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Comment Edited] (AIRFLOW-932) Backfills delete existing task instances and mark them as removed
[ https://issues.apache.org/jira/browse/AIRFLOW-932?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15892818#comment-15892818 ] Dan Davydov edited comment on AIRFLOW-932 at 3/2/17 7:14 PM: - This is off of master (88d9b0dc96e7528c87326c8070ee276e8565545f ). I just reproduced again (you are right about the start date though, should have been 2016). was (Author: aoen): This is off of master. I just reproduced again (you are right about the start date though, should have been 2016). > Backfills delete existing task instances and mark them as removed > - > > Key: AIRFLOW-932 > URL: https://issues.apache.org/jira/browse/AIRFLOW-932 > Project: Apache Airflow > Issue Type: Sub-task > Components: backfill > Reporter: Dan Davydov >Priority: Blocker > > I'm still investigating. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Comment Edited] (AIRFLOW-932) Backfills delete existing task instances and mark them as removed
[ https://issues.apache.org/jira/browse/AIRFLOW-932?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15892823#comment-15892823 ] Dan Davydov edited comment on AIRFLOW-932 at 3/2/17 7:17 PM: - Ah I think I know why it's not reproducing for you. Can you try letting the scheduler run a dagrun, and then backfill the dagrun once it completes (same DAG)? was (Author: aoen): Ah I think I know why it's not reproducing for you. Can you try letting the scheduler run a dagrun, and then backfill the dagrun once it completes? > Backfills delete existing task instances and mark them as removed > - > > Key: AIRFLOW-932 > URL: https://issues.apache.org/jira/browse/AIRFLOW-932 > Project: Apache Airflow > Issue Type: Sub-task > Components: backfill > Reporter: Dan Davydov >Priority: Blocker > > I'm still investigating. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Comment Edited] (AIRFLOW-932) Backfills delete existing task instances and mark them as removed
[ https://issues.apache.org/jira/browse/AIRFLOW-932?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15892823#comment-15892823 ] Dan Davydov edited comment on AIRFLOW-932 at 3/2/17 7:21 PM: - Ah I think I know why it's not reproducing for you. Can you try letting the scheduler run a dagrun, and then backfill the dagrun once it completes (same DAG)? so e.g. let it run for one dagrun via scheduler (might need to unpause it if it's paused) then airflow backfill teztg -t create_and_transfer_roster -s 2016-03-15 -e 2016-03-15 was (Author: aoen): Ah I think I know why it's not reproducing for you. Can you try letting the scheduler run a dagrun, and then backfill the dagrun once it completes (same DAG)? > Backfills delete existing task instances and mark them as removed > - > > Key: AIRFLOW-932 > URL: https://issues.apache.org/jira/browse/AIRFLOW-932 > Project: Apache Airflow > Issue Type: Sub-task > Components: backfill > Reporter: Dan Davydov >Priority: Blocker > > I'm still investigating. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Created] (AIRFLOW-932) Backfills delete existing task instances and mark them as removed
Dan Davydov created AIRFLOW-932: --- Summary: Backfills delete existing task instances and mark them as removed Key: AIRFLOW-932 URL: https://issues.apache.org/jira/browse/AIRFLOW-932 Project: Apache Airflow Issue Type: Sub-task Components: backfill Reporter: Dan Davydov Priority: Blocker I'm still investigating. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (AIRFLOW-932) Backfills delete existing task instances and mark them as removed
[ https://issues.apache.org/jira/browse/AIRFLOW-932?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15890822#comment-15890822 ] Dan Davydov commented on AIRFLOW-932: - I'm still investigating and will update the ticket with more details as soon as I am able to. My first guess is that the regex field of backfills makes Airflow think that a task doesn't exist anymore if it's not included in the regex or something like that. This is a blocker because it overwrites existing task instances that were e.g. successful with the removed state. > Backfills delete existing task instances and mark them as removed > - > > Key: AIRFLOW-932 > URL: https://issues.apache.org/jira/browse/AIRFLOW-932 > Project: Apache Airflow > Issue Type: Sub-task > Components: backfill > Reporter: Dan Davydov >Priority: Blocker > > I'm still investigating. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (AIRFLOW-920) Can't mark non-existent tasks as successful from graph view
[ https://issues.apache.org/jira/browse/AIRFLOW-920?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15890870#comment-15890870 ] Dan Davydov commented on AIRFLOW-920: - [~bolke] Where does ```from subdags.subdag import subdag``` come from? I don't see this in either our branch or master. > Can't mark non-existent tasks as successful from graph view > --- > > Key: AIRFLOW-920 > URL: https://issues.apache.org/jira/browse/AIRFLOW-920 > Project: Apache Airflow > Issue Type: Sub-task >Affects Versions: 1.8.0rc4 > Reporter: Dan Davydov >Assignee: Bolke de Bruin > Attachments: airflow_code_test_subdag.png, > airflow_dialog_test_subdag.png, airflow_treeview_test_subdag.png, > airfow_confirm_test_subdag.png > > > If you want to add a subdag operator with depends on past set to an existing > dag you need to mark the subdag as successful in the UI in order for it to > start running. Previously this was possible by mark-successing the > non-existent task instance/dagrun in the UI via the graph view. Now when you > try to mark the task successful Airflow gives an empty list of tasks that it > is going to try to mark successful, probably because the task instance/dagrun > do not exist. > Repro steps: > - Create a DAG with a dummy task > - Let this DAG run for one dagrun > - Add a new subdag operator that contains a dummy operator to this DAG that > has depends_on_past set to true > - click on the white square for the new subdag operator in the DAGs first > dagrun > - Click "Zoom into subdag" (takes you to the graph view for the subdag) > - Click the dummy task in the graph view and click "Mark Success" > - Observe that the list of tasks to mark as success is empty (it should > contain the dummy task) > [~bolke] -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Comment Edited] (AIRFLOW-920) Can't mark non-existent tasks as successful from graph view
[
https://issues.apache.org/jira/browse/AIRFLOW-920?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15890870#comment-15890870
]
Dan Davydov edited comment on AIRFLOW-920 at 3/1/17 7:29 PM:
-
[~bolke]
Where does
{quote}from subdags.subdag import subdag{quote}
come from? I don't see this in either our branch or master.
was (Author: aoen):
[~bolke]
Where does
```from subdags.subdag import subdag```
come from? I don't see this in either our branch or master.
> Can't mark non-existent tasks as successful from graph view
> ---
>
> Key: AIRFLOW-920
> URL: https://issues.apache.org/jira/browse/AIRFLOW-920
> Project: Apache Airflow
> Issue Type: Sub-task
>Affects Versions: 1.8.0rc4
> Reporter: Dan Davydov
>Assignee: Bolke de Bruin
> Attachments: airflow_code_test_subdag.png,
> airflow_dialog_test_subdag.png, airflow_treeview_test_subdag.png,
> airfow_confirm_test_subdag.png
>
>
> If you want to add a subdag operator with depends on past set to an existing
> dag you need to mark the subdag as successful in the UI in order for it to
> start running. Previously this was possible by mark-successing the
> non-existent task instance/dagrun in the UI via the graph view. Now when you
> try to mark the task successful Airflow gives an empty list of tasks that it
> is going to try to mark successful, probably because the task instance/dagrun
> do not exist.
> Repro steps:
> - Create a DAG with a dummy task
> - Let this DAG run for one dagrun
> - Add a new subdag operator that contains a dummy operator to this DAG that
> has depends_on_past set to true
> - click on the white square for the new subdag operator in the DAGs first
> dagrun
> - Click "Zoom into subdag" (takes you to the graph view for the subdag)
> - Click the dummy task in the graph view and click "Mark Success"
> - Observe that the list of tasks to mark as success is empty (it should
> contain the dummy task)
> [~bolke]
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Issue Comment Deleted] (AIRFLOW-920) Can't mark non-existent tasks as successful from graph view
[
https://issues.apache.org/jira/browse/AIRFLOW-920?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Dan Davydov updated AIRFLOW-920:
Comment: was deleted
(was: [~bolke]
Where does
{quote}from subdags.subdag import subdag{quote}
come from? I don't see this in either our branch or master.)
> Can't mark non-existent tasks as successful from graph view
> ---
>
> Key: AIRFLOW-920
> URL: https://issues.apache.org/jira/browse/AIRFLOW-920
> Project: Apache Airflow
> Issue Type: Sub-task
>Affects Versions: 1.8.0rc4
> Reporter: Dan Davydov
>Assignee: Bolke de Bruin
> Attachments: airflow_code_test_subdag.png,
> airflow_dialog_test_subdag.png, airflow_treeview_test_subdag.png,
> airfow_confirm_test_subdag.png
>
>
> If you want to add a subdag operator with depends on past set to an existing
> dag you need to mark the subdag as successful in the UI in order for it to
> start running. Previously this was possible by mark-successing the
> non-existent task instance/dagrun in the UI via the graph view. Now when you
> try to mark the task successful Airflow gives an empty list of tasks that it
> is going to try to mark successful, probably because the task instance/dagrun
> do not exist.
> Repro steps:
> - Create a DAG with a dummy task
> - Let this DAG run for one dagrun
> - Add a new subdag operator that contains a dummy operator to this DAG that
> has depends_on_past set to true
> - click on the white square for the new subdag operator in the DAGs first
> dagrun
> - Click "Zoom into subdag" (takes you to the graph view for the subdag)
> - Click the dummy task in the graph view and click "Mark Success"
> - Observe that the list of tasks to mark as success is empty (it should
> contain the dummy task)
> [~bolke]
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Commented] (AIRFLOW-920) Can't mark non-existent tasks as successful from graph view
[ https://issues.apache.org/jira/browse/AIRFLOW-920?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15888701#comment-15888701 ] Dan Davydov commented on AIRFLOW-920: - [~bolke] I'm not seeing this behavior. Marking the SubDagOperator task instance successful doesn't mark the tasks in the subdag for that execution dat as successful. Let me know if I'm missing something. > Can't mark non-existent tasks as successful from graph view > --- > > Key: AIRFLOW-920 > URL: https://issues.apache.org/jira/browse/AIRFLOW-920 > Project: Apache Airflow > Issue Type: Sub-task > Reporter: Dan Davydov > > If you want to add a subdag operator with depends on past set to an existing > dag you need to mark the subdag as successful in the UI in order for it to > start running. Previously this was possible by mark-successing the > non-existent task instance/dagrun in the UI via the graph view. Now when you > try to mark the task successful Airflow gives an empty list of tasks that it > is going to try to mark successful, probably because the task instance/dagrun > do not exist. > Repro steps: > - Create a DAG with a dummy task > - Let this DAG run for one dagrun > - Add a new subdag operator that contains a dummy operator to this DAG that > has depends_on_past set to true > - click on the white square for the new subdag operator in the DAGs first > dagrun > - Click "Zoom into subdag" (takes you to the graph view for the subdag) > - Click the dummy task in the graph view and click "Mark Success" > - Observe that the list of tasks to mark as success is empty (it should > contain the dummy task) > [~bolke] -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Updated] (AIRFLOW-921) 1.8.0rc Issues
[ https://issues.apache.org/jira/browse/AIRFLOW-921?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dan Davydov updated AIRFLOW-921: Description: These are the pending issues for the Airflow 1.8.0 release: Blockers: [~bolke] please merge into the next RC and then remove from the list the issues below once they are merged into master Sub-tasks linked in this JIRA Other Issues: - High DB Load (~8x more than 1.7) - We are still investigating but it's probably not a blocker for the release - (Theories: Might need execution_date index on dag_run (based on slow process list) OR it might be this query which is long running SELECT union_ti.dag_id AS union_ti_dag_id, union_ti.state AS union_ti_state, count( *) AS count_1 FR)) - Front page loading time is a lot slower [~bolke] was: These are the pending issues for the Airflow 1.8.0 release: Blockers: [~bolke] please merge into the next RC and then remove from the list the issues below once they are merged into master - Sub-tasks linked in this JIRA - Skipped tasks potentially cause a dagrun to be marked as failure/success prematurely (one theory is that this is the same issue as https://issues.apache.org/jira/browse/AIRFLOW-872) Other Issues: - High DB Load (~8x more than 1.7) - We are still investigating but it's probably not a blocker for the release - (Theories: Might need execution_date index on dag_run (based on slow process list) OR it might be this query which is long running SELECT union_ti.dag_id AS union_ti_dag_id, union_ti.state AS union_ti_state, count( *) AS count_1 FR)) - Front page loading time is a lot slower [~bolke] > 1.8.0rc Issues > -- > > Key: AIRFLOW-921 > URL: https://issues.apache.org/jira/browse/AIRFLOW-921 > Project: Apache Airflow > Issue Type: Task > Reporter: Dan Davydov >Assignee: Bolke de Bruin >Priority: Blocker > > These are the pending issues for the Airflow 1.8.0 release: > Blockers: > [~bolke] please merge into the next RC and then remove from the list the > issues below once they are merged into master > Sub-tasks linked in this JIRA > Other Issues: > - High DB Load (~8x more than 1.7) - We are still investigating but it's > probably not a blocker for the release - (Theories: Might need execution_date > index on dag_run (based on slow process list) OR it might be this query which > is long running SELECT union_ti.dag_id AS union_ti_dag_id, union_ti.state AS > union_ti_state, count( *) AS count_1 > FR)) > - Front page loading time is a lot slower > [~bolke] -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Resolved] (AIRFLOW-920) Can't mark non-existent tasks as successful from graph view
[ https://issues.apache.org/jira/browse/AIRFLOW-920?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dan Davydov resolved AIRFLOW-920. - Resolution: Cannot Reproduce > Can't mark non-existent tasks as successful from graph view > --- > > Key: AIRFLOW-920 > URL: https://issues.apache.org/jira/browse/AIRFLOW-920 > Project: Apache Airflow > Issue Type: Sub-task >Affects Versions: 1.8.0rc4 > Reporter: Dan Davydov >Assignee: Bolke de Bruin > Attachments: airflow_code_test_subdag.png, > airflow_dialog_test_subdag.png, airflow_treeview_test_subdag.png, > airfow_confirm_test_subdag.png > > > If you want to add a subdag operator with depends on past set to an existing > dag you need to mark the subdag as successful in the UI in order for it to > start running. Previously this was possible by mark-successing the > non-existent task instance/dagrun in the UI via the graph view. Now when you > try to mark the task successful Airflow gives an empty list of tasks that it > is going to try to mark successful, probably because the task instance/dagrun > do not exist. > Repro steps: > - Create a DAG with a dummy task > - Let this DAG run for one dagrun > - Add a new subdag operator that contains a dummy operator to this DAG that > has depends_on_past set to true > - click on the white square for the new subdag operator in the DAGs first > dagrun > - Click "Zoom into subdag" (takes you to the graph view for the subdag) > - Click the dummy task in the graph view and click "Mark Success" > - Observe that the list of tasks to mark as success is empty (it should > contain the dummy task) > [~bolke] -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (AIRFLOW-920) Can't mark non-existent tasks as successful from graph view
[ https://issues.apache.org/jira/browse/AIRFLOW-920?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15891122#comment-15891122 ] Dan Davydov commented on AIRFLOW-920: - I can confirm that this subdag works (even on Airbnb's internal fork of the release), I think this might be related to previous DB state or something, but I'm going to resolve it for now unless I can show otherwise. Thanks for looking at this with me Bolke > Can't mark non-existent tasks as successful from graph view > --- > > Key: AIRFLOW-920 > URL: https://issues.apache.org/jira/browse/AIRFLOW-920 > Project: Apache Airflow > Issue Type: Sub-task >Affects Versions: 1.8.0rc4 > Reporter: Dan Davydov >Assignee: Bolke de Bruin > Attachments: airflow_code_test_subdag.png, > airflow_dialog_test_subdag.png, airflow_treeview_test_subdag.png, > airfow_confirm_test_subdag.png > > > If you want to add a subdag operator with depends on past set to an existing > dag you need to mark the subdag as successful in the UI in order for it to > start running. Previously this was possible by mark-successing the > non-existent task instance/dagrun in the UI via the graph view. Now when you > try to mark the task successful Airflow gives an empty list of tasks that it > is going to try to mark successful, probably because the task instance/dagrun > do not exist. > Repro steps: > - Create a DAG with a dummy task > - Let this DAG run for one dagrun > - Add a new subdag operator that contains a dummy operator to this DAG that > has depends_on_past set to true > - click on the white square for the new subdag operator in the DAGs first > dagrun > - Click "Zoom into subdag" (takes you to the graph view for the subdag) > - Click the dummy task in the graph view and click "Mark Success" > - Observe that the list of tasks to mark as success is empty (it should > contain the dummy task) > [~bolke] -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Updated] (AIRFLOW-953) Tasks marked as successful should have complete state set
[ https://issues.apache.org/jira/browse/AIRFLOW-953?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dan Davydov updated AIRFLOW-953: Summary: Tasks marked as successful should have complete state set (was: Tasks marked as successful should have start/end_date/duration set) > Tasks marked as successful should have complete state set > - > > Key: AIRFLOW-953 > URL: https://issues.apache.org/jira/browse/AIRFLOW-953 > Project: Apache Airflow > Issue Type: Bug > Components: webserver > Reporter: Dan Davydov > Labels: beginner, starter > > Tasks marked as successful should have start/end_date/duration/operator set . > start/end dates should be the same and denote the time the task was marked as > successful, duration should be 0, and the operator should be filled in > correctly with the task's operator. > This should be fixed because otherwise the task instance state is not > complete which could break some operations in Airflow, and prevents things > like scripts that delete old tasks from Airflow (since e.g. start_date is not > specified for these tasks). -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Created] (AIRFLOW-953) Tasks marked as successful should have start/end_date/duration set
Dan Davydov created AIRFLOW-953: --- Summary: Tasks marked as successful should have start/end_date/duration set Key: AIRFLOW-953 URL: https://issues.apache.org/jira/browse/AIRFLOW-953 Project: Apache Airflow Issue Type: Bug Components: webserver Reporter: Dan Davydov Tasks marked as successful should have start/end_date/duration/operator set . start/end dates should be the same and denote the time the task was marked as successful, duration should be 0, and the operator should be filled in correctly with the task's operator. This should be fixed because otherwise the task instance state is not complete which could break some operations in Airflow, and prevents things like scripts that delete old tasks from Airflow (since e.g. start_date is not specified for these tasks). -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Updated] (AIRFLOW-1111) Make logs easier to read and separate infra/task logs
[
https://issues.apache.org/jira/browse/AIRFLOW-?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Dan Davydov updated AIRFLOW-:
-
Description:
Ideally there would be tabs for logs:
Dag-parsing (needs a better name)
1 (Short for "Run 1")
2
...
Dag-parsing would contain the output like:
{code}
[2017-04-14 15:24:54,341] {models.py:168} INFO - Filling up the DagBag from
/srv/data/airflow/airflow_canaries/canaries.py
[2017-04-14 15:24:54,342] {models.py:253} DEBUG - Importing
/srv/data/airflow/airflow_canaries/canaries.py
[2017-04-14 15:24:54,356] {models.py:373} DEBUG - Loaded DAG
[2017-04-14 15:24:54,356] {models.py:373} DEBUG - Loaded DAG
[2017-04-14 15:24:54,356] {models.py:373} DEBUG - Loaded DAG
[2017-04-14 15:24:54,356] {models.py:373} DEBUG - Loaded DAG
[2017-04-14 15:24:54,356] {models.py:373} DEBUG - Loaded DAG
[2017-04-14 15:24:54,357] {models.py:373} DEBUG - Loaded DAG
[2017-04-14 15:24:54,553] {cgroup_task_runner.py:82} DEBUG - Not creating
cgroup memory in / since it already exists
[2017-04-14 15:24:54,554] {cgroup_task_runner.py:82} DEBUG - Not creating
cgroup airflow in /memory since it already exists
[2017-04-14 15:24:54,554] {cgroup_task_runner.py:82} DEBUG - Not creating
cgroup 2017-04-14 in /memory/airflow since it already exists
[2017-04-14 15:24:54,554] {cgroup_task_runner.py:77} DEBUG - Creating cgroup
82d42ce2-2126-11e7-ad73-22000a896f7d in /memory/airflow/2017-04-14
{code}
and "1" would contain
{code}
2017-04-14 15:24:56,146] {base_task_runner.py:95} INFO - Subtask: [2017-04-14
15:24:56,146] {models.py:1344} INFO - Executing <Task(HiveOperator):
insert_canary_row> on 2017-04-14 15:10:00
[2017-04-14 15:24:56,173] {base_task_runner.py:95} INFO - Subtask: [2017-04-14
15:24:56,172] {hive_operator.py:98} INFO - Executing: USE datainfra;
[2017-04-14 15:24:56,173] {base_task_runner.py:95} INFO - Subtask:
CREATE TABLE IF NOT EXISTS datainfra.airflow_precious_canary_gold_large (key
STRING);
[2017-04-14 15:24:56,173] {base_task_runner.py:95} INFO - Subtask: INSERT
OVERWRITE TABLE datainfra.airflow_precious_canary_gold_large
[2017-04-14 15:24:56,173] {base_task_runner.py:95} INFO - Subtask: SELECT key
FROM datainfra.one_row LIMIT 1;
[2017-04-14 15:24:56,187] {base_task_runner.py:95} INFO - Subtask: [2017-04-14
15:24:56,186] {hive_hooks.py:207} INFO - hive -hiveconf
airflow.ctx.dag.dag_id=airflow_precious_canary_gold_large -hiveconf
airflow.ctx.dag_run.execution_date=2017-04-14T15:10:00 -hiveconf
airflow.ctx.task_instance.execution_date=2017-04-14T15:10:00 -hiveconf
airflow.ctx.task.task_id=insert_canary_row -hiveconf
mapreduce.job.queuename=airflow_canary_gold -f
/tmp/airflow_hiveop_ZixLMN/tmphgm9MR
{code}
(note that "Starting attempt X of Y has been removed from this log).
By default "1" would be opened if it exists when a user clicks into the log,
otherwise dag-parsing would be opened.
was:
Ideally there would be tabs for logs:
Dag-parsing (needs a better name)
1 (Short for "Run 1")
2
...
Dag-parsing would contain the output like:
{code}
[2017-04-14 15:24:54,341] {models.py:168} INFO - Filling up the DagBag from
/srv/data/airflow/airflow_canaries/canaries.py
[2017-04-14 15:24:54,342] {models.py:253} DEBUG - Importing
/srv/data/airflow/airflow_canaries/canaries.py
[2017-04-14 15:24:54,356] {models.py:373} DEBUG - Loaded DAG
[2017-04-14 15:24:54,356] {models.py:373} DEBUG - Loaded DAG
[2017-04-14 15:24:54,356] {models.py:373} DEBUG - Loaded DAG
[2017-04-14 15:24:54,356] {models.py:373} DEBUG - Loaded DAG
[2017-04-14 15:24:54,356] {models.py:373} DEBUG - Loaded DAG
[2017-04-14 15:24:54,357] {models.py:373} DEBUG - Loaded DAG
{code}
By default "1" would be opened if it exists when a user clicks into the log,
otherwise dag-parsing would be opened.
> Make logs easier to read and separate infra/task logs
> -
>
> Key: AIRFLOW-
> URL: https://issues.apache.org/jira/browse/AIRFLOW-
> Project: Apache Airflow
> Issue Type: Improvement
>Reporter: Dan Davydov
>
> Ideally there would be tabs for logs:
> Dag-parsing (needs a better name)
> 1 (Short for "Run 1")
> 2
> ...
> Dag-parsing would contain the output like:
> {code}
> [2017-04-14 15:24:54,341] {models.py:168} INFO - Filling up the DagBag from
> /srv/data/airflow/airflow_canaries/canaries.py
> [2017-04-14 15:24:54,342] {models.py:253} DEBUG - Importing
> /srv/data/airflow/airflow_canaries/canaries.py
> [2017-04-14 15:24:54,356] {models.py:373} DEBUG - Loaded DAG airflow_precious_canary_silver_medium>
> [2017-04-14 15:24:54,356] {models.py:373} DEBUG - Loaded DAG airflow_precious_canary_silver_r>
> [2017-04-14 15:24:54,356] {models.py:373} DEBUG - L
[jira] [Created] (AIRFLOW-1111) Make logs easier to read and separate infra/task logs
Dan Davydov created AIRFLOW-:
Summary: Make logs easier to read and separate infra/task logs
Key: AIRFLOW-
URL: https://issues.apache.org/jira/browse/AIRFLOW-
Project: Apache Airflow
Issue Type: Improvement
Reporter: Dan Davydov
Ideally there would be tabs for logs:
Dag-parsing (needs a better name)
1 (Short for "Run 1")
2
...
Dag-parsing would contain the output like:
{code}
[2017-04-14 15:24:54,341] {models.py:168} INFO - Filling up the DagBag from
/srv/data/airflow/airflow_canaries/canaries.py
[2017-04-14 15:24:54,342] {models.py:253} DEBUG - Importing
/srv/data/airflow/airflow_canaries/canaries.py
[2017-04-14 15:24:54,356] {models.py:373} DEBUG - Loaded DAG
[2017-04-14 15:24:54,356] {models.py:373} DEBUG - Loaded DAG
[2017-04-14 15:24:54,356] {models.py:373} DEBUG - Loaded DAG
[2017-04-14 15:24:54,356] {models.py:373} DEBUG - Loaded DAG
[2017-04-14 15:24:54,356] {models.py:373} DEBUG - Loaded DAG
[2017-04-14 15:24:54,357] {models.py:373} DEBUG - Loaded DAG
{code}
By default "1" would be opened if it exists when a user clicks into the log,
otherwise dag-parsing would be opened.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Created] (AIRFLOW-1123) Airflow workers should handle shutdowns gracefully
Dan Davydov created AIRFLOW-1123: Summary: Airflow workers should handle shutdowns gracefully Key: AIRFLOW-1123 URL: https://issues.apache.org/jira/browse/AIRFLOW-1123 Project: Apache Airflow Issue Type: New Feature Components: worker Reporter: Dan Davydov Airflow should handle airflow worker shutdowns gracefully, i.e. it shouldn't send failure emails, shouldn't trigger failure callbacks, and shouldn't increment the retry number if a worker receives a signal to terminate. This shutdown should still be communicated in the task logs/the UI though. This change will probably require adding a new state for task instances, like KILLED (which we can potentially reuse later for preemption if we choose to implement it). [~bolke][~saguziel] -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (AIRFLOW-774) dagbag_size/collect_dags/dagbag_import_errors stats incorrect
[ https://issues.apache.org/jira/browse/AIRFLOW-774?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16111825#comment-16111825 ] Dan Davydov commented on AIRFLOW-774: - Yes to master, IIRC they should appear starting from release 1.8.1. This is the JIRA: https://issues.apache.org/jira/browse/AIRFLOW-780 Yep only fixed the import errors, not the stats. The changes should be similar though (pull the stats logic out to the top-level which aggregates each individual parsing subprocesses errors). > dagbag_size/collect_dags/dagbag_import_errors stats incorrect > - > > Key: AIRFLOW-774 > URL: https://issues.apache.org/jira/browse/AIRFLOW-774 > Project: Apache Airflow > Issue Type: Bug > Components: logging > Reporter: Dan Davydov > > After the multiprocessor change was made (dag folders are processed in > parallel), the number of dags reported by airflow is for each of these > subprocesses which is inaccurate, and potentially orders of magnitude less > than the actual number of dags. These individual processes stats should be > aggregated. The collect_dags/dagbag_import_errors stats should also be fixed > (time it takes to parse the dags). -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Created] (AIRFLOW-1513) Come up with an abstraction to set defaults per task
Dan Davydov created AIRFLOW-1513: Summary: Come up with an abstraction to set defaults per task Key: AIRFLOW-1513 URL: https://issues.apache.org/jira/browse/AIRFLOW-1513 Project: Apache Airflow Issue Type: Bug Reporter: Dan Davydov Come up with an abstraction to set defaults per task, e.g. all HiveOperator tasks will have some specific retry policy that can be set globally by Airflow users. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Assigned] (AIRFLOW-1311) Improve Webserver Load Time For Large DAGs
[ https://issues.apache.org/jira/browse/AIRFLOW-1311?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dan Davydov reassigned AIRFLOW-1311: Assignee: Dan Davydov > Improve Webserver Load Time For Large DAGs > -- > > Key: AIRFLOW-1311 > URL: https://issues.apache.org/jira/browse/AIRFLOW-1311 > Project: Apache Airflow > Issue Type: Bug > Components: webserver > Reporter: Dan Davydov > Assignee: Dan Davydov > > Large DAGs can take an extremely long time to load in the Airflow UI > (minutes/timeout). > The fixes are as follows: > 1. Lazy load DAGs (load up to a certain # of tasks by default, prioritizing > tasks by their depth, and allow users to expand sections for these DAGs, > ideally prefetch deeper tasks once the initial set of tasks has rendered ) > 2. Identify bottlenecks/performance issues in both the frontend/backend for > rendering DAGs on the webserver and fix them. Airflow should be more > performant for displaying DAGs that are somewhat large, e.g. DAGs that have > up to 500 nodes and 2000 edges (dependencies from one task to another) should > render within a couple of seconds. > 3. Make DAG loading asynchronous in the UI (once the top-level tasks have > loaded display them immediately). We might not want to do this as users might > try to click something only to have the UI change from underneath them > [~saguziel] -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Assigned] (AIRFLOW-1311) Improve Webserver Load Time For Large DAGs
[ https://issues.apache.org/jira/browse/AIRFLOW-1311?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dan Davydov reassigned AIRFLOW-1311: Assignee: Edgar Rodriguez (was: Dan Davydov) > Improve Webserver Load Time For Large DAGs > -- > > Key: AIRFLOW-1311 > URL: https://issues.apache.org/jira/browse/AIRFLOW-1311 > Project: Apache Airflow > Issue Type: Bug > Components: webserver > Reporter: Dan Davydov >Assignee: Edgar Rodriguez > > Large DAGs can take an extremely long time to load in the Airflow UI > (minutes/timeout). > The fixes are as follows: > 1. Lazy load DAGs (load up to a certain # of tasks by default, prioritizing > tasks by their depth, and allow users to expand sections for these DAGs, > ideally prefetch deeper tasks once the initial set of tasks has rendered ) > 2. Identify bottlenecks/performance issues in both the frontend/backend for > rendering DAGs on the webserver and fix them. Airflow should be more > performant for displaying DAGs that are somewhat large, e.g. DAGs that have > up to 500 nodes and 2000 edges (dependencies from one task to another) should > render within a couple of seconds. > 3. Make DAG loading asynchronous in the UI (once the top-level tasks have > loaded display them immediately). We might not want to do this as users might > try to click something only to have the UI change from underneath them > [~saguziel] -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Resolved] (AIRFLOW-1442) Airflow Ignore All Deps Command Generation Has an Extra Space
[ https://issues.apache.org/jira/browse/AIRFLOW-1442?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dan Davydov resolved AIRFLOW-1442. -- Resolution: Fixed > Airflow Ignore All Deps Command Generation Has an Extra Space > - > > Key: AIRFLOW-1442 > URL: https://issues.apache.org/jira/browse/AIRFLOW-1442 > Project: Apache Airflow > Issue Type: Bug > Reporter: Dan Davydov > Assignee: Dan Davydov > > Airflow Ignore All Deps Command Generation Has an Extra Space. This causes > airflow commands e.g. generated by the webserver to fail. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (AIRFLOW-1311) Improve Webserver Load Time For Large DAGs
[ https://issues.apache.org/jira/browse/AIRFLOW-1311?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16128134#comment-16128134 ] Dan Davydov commented on AIRFLOW-1311: -- Edgar assigning to you, feel free to mark as dupe or merge if you have an existing JIRA ticket. > Improve Webserver Load Time For Large DAGs > -- > > Key: AIRFLOW-1311 > URL: https://issues.apache.org/jira/browse/AIRFLOW-1311 > Project: Apache Airflow > Issue Type: Bug > Components: webserver > Reporter: Dan Davydov >Assignee: Edgar Rodriguez > > Large DAGs can take an extremely long time to load in the Airflow UI > (minutes/timeout). > The fixes are as follows: > 1. Lazy load DAGs (load up to a certain # of tasks by default, prioritizing > tasks by their depth, and allow users to expand sections for these DAGs, > ideally prefetch deeper tasks once the initial set of tasks has rendered ) > 2. Identify bottlenecks/performance issues in both the frontend/backend for > rendering DAGs on the webserver and fix them. Airflow should be more > performant for displaying DAGs that are somewhat large, e.g. DAGs that have > up to 500 nodes and 2000 edges (dependencies from one task to another) should > render within a couple of seconds. > 3. Make DAG loading asynchronous in the UI (once the top-level tasks have > loaded display them immediately). We might not want to do this as users might > try to click something only to have the UI change from underneath them > [~saguziel] -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Resolved] (AIRFLOW-1266) Long task names are truncated in gannt view
[ https://issues.apache.org/jira/browse/AIRFLOW-1266?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dan Davydov resolved AIRFLOW-1266. -- Resolution: Fixed > Long task names are truncated in gannt view > --- > > Key: AIRFLOW-1266 > URL: https://issues.apache.org/jira/browse/AIRFLOW-1266 > Project: Apache Airflow > Issue Type: Bug > Components: webserver > Reporter: Dan Davydov > Fix For: 1.8.2 > > > Long task names are truncated in gannt view, we should make the text smaller > instead. To reproduce create a task called > abczzzdef and note that the start/end characters > are not fully visible in the gant view. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Assigned] (AIRFLOW-1247) CLI: ignore all dependencies argument ignored
[ https://issues.apache.org/jira/browse/AIRFLOW-1247?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dan Davydov reassigned AIRFLOW-1247: Assignee: Dan Davydov (was: Matti Remes) > CLI: ignore all dependencies argument ignored > - > > Key: AIRFLOW-1247 > URL: https://issues.apache.org/jira/browse/AIRFLOW-1247 > Project: Apache Airflow > Issue Type: Bug > Components: cli >Affects Versions: 1.8.1 >Reporter: Matti Remes >Assignee: Dan Davydov >Priority: Trivial > Fix For: 1.9.0 > > > Missing comma in Arg object creation -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Created] (AIRFLOW-1442) Airflow Ignore All Deps Command Generation Has an Extra Space
Dan Davydov created AIRFLOW-1442: Summary: Airflow Ignore All Deps Command Generation Has an Extra Space Key: AIRFLOW-1442 URL: https://issues.apache.org/jira/browse/AIRFLOW-1442 Project: Apache Airflow Issue Type: Bug Reporter: Dan Davydov Assignee: Dan Davydov Airflow Ignore All Deps Command Generation Has an Extra Space. This causes airflow commands e.g. generated by the webserver to fail. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Created] (AIRFLOW-1448) Revert PR 2433 which added merge conflicts to master
Dan Davydov created AIRFLOW-1448: Summary: Revert PR 2433 which added merge conflicts to master Key: AIRFLOW-1448 URL: https://issues.apache.org/jira/browse/AIRFLOW-1448 Project: Apache Airflow Issue Type: Bug Reporter: Dan Davydov Assignee: Dan Davydov https://github.com/apache/incubator-airflow/pull/2433 has logical merge conflicts in master and causes tests to fail, it needs to be reverted. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Created] (AIRFLOW-1432) NVD3 Charts do not have labeled axes and units change dynamically
Dan Davydov created AIRFLOW-1432: Summary: NVD3 Charts do not have labeled axes and units change dynamically Key: AIRFLOW-1432 URL: https://issues.apache.org/jira/browse/AIRFLOW-1432 Project: Apache Airflow Issue Type: Bug Components: webserver Reporter: Dan Davydov E.g. for the landing times chart, the y axis isn't labeled (e.g. minutes/hours/days), and changes dynamically based on the data points in the chart/the pixel height of the chart. The y axis should be labeled in these charts. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Created] (AIRFLOW-1351) airflow cli commands should log error/debug to stderr
Dan Davydov created AIRFLOW-1351:
Summary: airflow cli commands should log error/debug to stderr
Key: AIRFLOW-1351
URL: https://issues.apache.org/jira/browse/AIRFLOW-1351
Project: Apache Airflow
Issue Type: Bug
Components: cli
Reporter: Dan Davydov
Commands like airflow list_task are printing messages like:
{code}[2017-06-27 20:36:57,116] {models.py:373} DEBUG - Loaded DAG {code}
to stdout when the only thing being printed to stdout should be the listed dag
ids.
This makes it hard for users to parse the output of these commands.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Created] (AIRFLOW-1152) Backfill dagruns should not hang DAGs
Dan Davydov created AIRFLOW-1152: Summary: Backfill dagruns should not hang DAGs Key: AIRFLOW-1152 URL: https://issues.apache.org/jira/browse/AIRFLOW-1152 Project: Apache Airflow Issue Type: Bug Components: backfill Reporter: Dan Davydov Backfills should have some kind of heartbeat mechanism that causes the scheduler to time out and fail their dagruns if e.g. the backfill is interrupted. Otherwise you run into the case where you have something like max_active_dagruns = 1 and a cancelled backrun that leaves its dagrun in a running state causes the whole dag to hang. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (AIRFLOW-1143) Tasks rejected by workers get stuck in QUEUED
[ https://issues.apache.org/jira/browse/AIRFLOW-1143?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15987188#comment-15987188 ] Dan Davydov commented on AIRFLOW-1143: -- I agree the NONE solution is OK for now (need to be careful though e.g. if the "task is already in running state" dep fails we don't want to set the state to NONE) but long term I think we want something more explicit (fixing heartbeat so it supports this and having the scheduler set the state instead of the worker). The reason is all of the state changes should eventually be done by the scheduler (workers should be "dumb"). I don't see that message "FIXME: Rescheduling due to concurrency limits" but that looks like some new TODO and we are running off of a subset of master at the moment (between 1.8.0 and 1.8.1 with some stuff after 1.8.1) so I'm guessing that's why. > Tasks rejected by workers get stuck in QUEUED > - > > Key: AIRFLOW-1143 > URL: https://issues.apache.org/jira/browse/AIRFLOW-1143 > Project: Apache Airflow > Issue Type: Bug > Components: scheduler >Reporter: Dan Davydov >Assignee: Gerard Toonstra > > If the scheduler schedules a task that is sent to a worker that then rejects > the task (e.g. because one of the dependencies of the tasks became bad, like > the pool became full), the task will be stuck in the QUEUED state. We hit > this trying to switch from invoking the scheduler "airflow scheduler -n 5" to > just "airflow scheduler". > Restarting the scheduler fixes this because it cleans up orphans, but we > shouldn't have to restart the scheduler to fix these problems (the missing > job heartbeats should make the scheduler requeue the task). -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Updated] (AIRFLOW-1143) Tasks rejected by workers get stuck in QUEUED
[ https://issues.apache.org/jira/browse/AIRFLOW-1143?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dan Davydov updated AIRFLOW-1143: - Description: If the scheduler schedules a task that is sent to a worker that then rejects the task (e.g. because one of the dependencies of the tasks became bad, like the pool became full), the task will be stuck in the QUEUED state. We hit this trying to switch from invoking the scheduler "airflow scheduler -n 5" to just "airflow scheduler". Restarting the scheduler fixes this because it cleans up orphans, but we shouldn't have to restart the scheduler to fix these problems (the missing job heartbeats should make the scheduler requeue the task). was: If the scheduler schedules a task that is sent to a worker that then rejects the task (e.g. because one of the dependencies of the tasks became bad, like the pool became full), the task will be stuck in the QUEUED state. Restarting the scheduler fixes this because it cleans up orphans, but we shouldn't have to restart the scheduler to fix these problems (the missing job heartbeats should make the scheduler requeue the task). > Tasks rejected by workers get stuck in QUEUED > - > > Key: AIRFLOW-1143 > URL: https://issues.apache.org/jira/browse/AIRFLOW-1143 > Project: Apache Airflow > Issue Type: Bug > Components: scheduler >Reporter: Dan Davydov > > If the scheduler schedules a task that is sent to a worker that then rejects > the task (e.g. because one of the dependencies of the tasks became bad, like > the pool became full), the task will be stuck in the QUEUED state. We hit > this trying to switch from invoking the scheduler "airflow scheduler -n 5" to > just "airflow scheduler". > Restarting the scheduler fixes this because it cleans up orphans, but we > shouldn't have to restart the scheduler to fix these problems (the missing > job heartbeats should make the scheduler requeue the task). -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Closed] (AIRFLOW-672) Allow logs to be piped into another process
[ https://issues.apache.org/jira/browse/AIRFLOW-672?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dan Davydov closed AIRFLOW-672. --- Resolution: Fixed Not really relevant with the new logging abstraction anymore. We can create a piping to a file logging handler if we want now. > Allow logs to be piped into another process > --- > > Key: AIRFLOW-672 > URL: https://issues.apache.org/jira/browse/AIRFLOW-672 > Project: Apache Airflow > Issue Type: Improvement > Components: logging > Reporter: Dan Davydov > > Instead of writing logs to a file Airflow should be able to pipe them to a > process similar to how Apache Webserver does it: > https://httpd.apache.org/docs/1.3/logs.html#piped . The most important reason > for this is to allow log rotation of tasks which can get quite large on the > local worker disks. Part of this task could be moving the existing s3 log > exporting logic (and other custom logging logic) out of the airflow core. > FWIW I have the code ready to add log rotation to Airflow itself, but this > isn't as ideal as the piping solution (separation of concerns). If there is > interest in getting this merged as a temporary measure I can do so. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Resolved] (AIRFLOW-1349) Max active dagrun check for backfills shouldn't include the backfilled dagrun
[ https://issues.apache.org/jira/browse/AIRFLOW-1349?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dan Davydov resolved AIRFLOW-1349. -- Resolution: Fixed > Max active dagrun check for backfills shouldn't include the backfilled dagrun > - > > Key: AIRFLOW-1349 > URL: https://issues.apache.org/jira/browse/AIRFLOW-1349 > Project: Apache Airflow > Issue Type: Bug > Components: backfill > Reporter: Dan Davydov >Assignee: Edgar Rodriguez > > When you backfill a dag with e.g. 1 max active dagrun, if that dagrun is > already running then it shouldn't count against the max active dagruns of the > backfill and make the backfill fail. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (AIRFLOW-908) Airflow run should print worker name at top of log
[ https://issues.apache.org/jira/browse/AIRFLOW-908?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dan Davydov updated AIRFLOW-908: Description: Airflow run should log the worker hostname at top of log. (was: Airflow run should print the worker hostname at top of log.) > Airflow run should print worker name at top of log > -- > > Key: AIRFLOW-908 > URL: https://issues.apache.org/jira/browse/AIRFLOW-908 > Project: Apache Airflow > Issue Type: New Feature > Reporter: Dan Davydov > Labels: beginner, starter > > Airflow run should log the worker hostname at top of log. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Closed] (AIRFLOW-907) Ability to clear all task_instances for a dagrun in the UI
[ https://issues.apache.org/jira/browse/AIRFLOW-907?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dan Davydov closed AIRFLOW-907. --- Resolution: Duplicate > Ability to clear all task_instances for a dagrun in the UI > -- > > Key: AIRFLOW-907 > URL: https://issues.apache.org/jira/browse/AIRFLOW-907 > Project: Apache Airflow > Issue Type: New Feature > Components: webserver > Reporter: Dan Davydov > > Clicking the dagrun circle in the Airflow UI should provide an option to > clear all task_instances for that dagrun (or go to the dagrun's state as is > the current implementation). At Airbnb a lot of users resort to creating a > top-level task so that they can clear using that which isn't great. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Resolved] (AIRFLOW-395) Remove commas from resoures in config
[ https://issues.apache.org/jira/browse/AIRFLOW-395?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dan Davydov resolved AIRFLOW-395. - Resolution: Fixed > Remove commas from resoures in config > - > > Key: AIRFLOW-395 > URL: https://issues.apache.org/jira/browse/AIRFLOW-395 > Project: Apache Airflow > Issue Type: Bug > Reporter: Dan Davydov > -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Resolved] (AIRFLOW-385) Scheduler logs should have a "latest" directory
[ https://issues.apache.org/jira/browse/AIRFLOW-385?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dan Davydov resolved AIRFLOW-385. - Resolution: Fixed > Scheduler logs should have a "latest" directory > --- > > Key: AIRFLOW-385 > URL: https://issues.apache.org/jira/browse/AIRFLOW-385 > Project: Apache Airflow > Issue Type: Improvement > Components: logging >Reporter: Dan Davydov >Assignee: Vijay Bhat >Priority: Minor > Labels: beginner, logging > > Right now the scheduler logs for each parsed file look like > /tmp/airflow/scheduler/logs/2016-07-17/some-dag/... > The problem is this makes it hard to find the latest logs which is the most > frequent use case. > Ideally we would create and have the scheduler keep up to date a symlink to > the latest date /tmp/airflow/scheduler/logs/latest which would point to e.g. > /tmp/airflow/scheduler/logs/2016-07-17 > We might also want to consider changing the structure to dag/date instead of > date/dag too, but that can be done as a separate task if desired. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (AIRFLOW-908) Airflow run should print worker name at top of log
[ https://issues.apache.org/jira/browse/AIRFLOW-908?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16011029#comment-16011029 ] Dan Davydov commented on AIRFLOW-908: - I'm planning on giving this task to my intern as part of a larger logging refactor (I should have updated the ticket sorry), but if you haven't already started on this one here is another good starter ticket: https://issues.apache.org/jira/browse/AIRFLOW-936 > Airflow run should print worker name at top of log > -- > > Key: AIRFLOW-908 > URL: https://issues.apache.org/jira/browse/AIRFLOW-908 > Project: Apache Airflow > Issue Type: New Feature > Reporter: Dan Davydov > Assignee: Dan Davydov > Labels: beginner, starter > > Airflow run should log the worker hostname at top of log. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Assigned] (AIRFLOW-908) Airflow run should print worker name at top of log
[ https://issues.apache.org/jira/browse/AIRFLOW-908?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dan Davydov reassigned AIRFLOW-908: --- Assignee: Dan Davydov (was: Phawin Khongkhasawan) > Airflow run should print worker name at top of log > -- > > Key: AIRFLOW-908 > URL: https://issues.apache.org/jira/browse/AIRFLOW-908 > Project: Apache Airflow > Issue Type: New Feature > Reporter: Dan Davydov > Assignee: Dan Davydov > Labels: beginner, starter > > Airflow run should log the worker hostname at top of log. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Assigned] (AIRFLOW-936) Ability to clear/mark success all tasks in a dagrun in the UI
[ https://issues.apache.org/jira/browse/AIRFLOW-936?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dan Davydov reassigned AIRFLOW-936: --- Assignee: Dan Davydov > Ability to clear/mark success all tasks in a dagrun in the UI > - > > Key: AIRFLOW-936 > URL: https://issues.apache.org/jira/browse/AIRFLOW-936 > Project: Apache Airflow > Issue Type: New Feature > Components: webserver > Reporter: Dan Davydov > Assignee: Dan Davydov > > The same way you can clear/mark success a task instance recursively, you > should be able to do for a dagrun circle in the UI in the tree view. Users > have been resorting to creating dummy root tasks for their DAGs just to clear > the other tasks in a dagrun which is a hack. > This would require changing the current behavior of clicking a dagrun (from > editing the dagrun), to having a popup similar to what you get when you click > on a task instance. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Resolved] (AIRFLOW-1024) Handle CeleryExecutor errors gracefully
[ https://issues.apache.org/jira/browse/AIRFLOW-1024?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dan Davydov resolved AIRFLOW-1024. -- Resolution: Fixed > Handle CeleryExecutor errors gracefully > --- > > Key: AIRFLOW-1024 > URL: https://issues.apache.org/jira/browse/AIRFLOW-1024 > Project: Apache Airflow > Issue Type: Bug > Reporter: Dan Davydov >Priority: Critical > Fix For: 1.8.1 > > > If the Airflow celery executor receives a bad response from a worker (e.g. > unpickled response), then it will crash the scheduler and cause it to restart. > We should code defensively around the interactions with celery so that we > just log errors instead of crashing the scheduler. > It might makes sense to make the try catches one level higher (to catch > errors from all executors), but this needs some investigation. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Created] (AIRFLOW-1241) Allow airflow test and airflow run to take arguments to pass to DAGs
Dan Davydov created AIRFLOW-1241: Summary: Allow airflow test and airflow run to take arguments to pass to DAGs Key: AIRFLOW-1241 URL: https://issues.apache.org/jira/browse/AIRFLOW-1241 Project: Apache Airflow Issue Type: Improvement Components: cli Reporter: Dan Davydov A common use case it to have conditionals in a DAG to run dev/prod code, e.g. DEV_MODE=True if DEV_MODE: hit_test_db else: hit_prod_db it would be good if you could instead pass arguments to tasks with airflow run/test e.g. airflow run some_dag some_task some_date --args foo=1 and then in the dag: something like execution_args.foo could be used (which would have value 1). -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Updated] (AIRFLOW-1249) Running tasks from backfills with unmet conditions are stuck running
[
https://issues.apache.org/jira/browse/AIRFLOW-1249?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Dan Davydov updated AIRFLOW-1249:
-
Description:
Running tasks from backfills with unmet conditions are stuck running. E.g.
create a DAG and pause it, mark the dagrun for a specific date as failed in the
UI, and then start a backfill for that date. Note that the task is stuck in the
running or queued state with no start/end date set. I believe this occurs for
the Celery executor but it may affect other/all executors.
This can e.g. cause pools to fill up.
This query cleans up the tasks as a workaround:
{code}
DELETE ti FROM task_instance ti
JOIN dag_run dr
ON ti.execution_date = dr.execution_date AND
ti.dag_id = dr.dag_id
JOIN dag dg
ON dr.dag_id=dg.dag_id
WHERE ISNULL(ti.start_date) and (ti.state="queued" or
ti.state="running")
and dr.state="failed" and dg.is_paused="1";
{code}
was:
Running tasks from backfills with unmet conditions are stuck running. E.g.
create a DAG and pause it, mark the dagrun for a specific date as failed in the
UI, and then start a backfill for that date. Note that the task is stuck in the
running or queued state with no start/end date set. I believe this occurs for
the Celery executor but it may affect other/all executors.
This can e.g. cause pools to fill up.
> Running tasks from backfills with unmet conditions are stuck running
>
>
> Key: AIRFLOW-1249
> URL: https://issues.apache.org/jira/browse/AIRFLOW-1249
> Project: Apache Airflow
> Issue Type: Bug
>Reporter: Dan Davydov
>
> Running tasks from backfills with unmet conditions are stuck running. E.g.
> create a DAG and pause it, mark the dagrun for a specific date as failed in
> the UI, and then start a backfill for that date. Note that the task is stuck
> in the running or queued state with no start/end date set. I believe this
> occurs for the Celery executor but it may affect other/all executors.
> This can e.g. cause pools to fill up.
> This query cleans up the tasks as a workaround:
> {code}
> DELETE ti FROM task_instance ti
> JOIN dag_run dr
> ON ti.execution_date = dr.execution_date AND
>ti.dag_id = dr.dag_id
>JOIN dag dg
>ON dr.dag_id=dg.dag_id
> WHERE ISNULL(ti.start_date) and (ti.state="queued" or
> ti.state="running")
> and dr.state="failed" and dg.is_paused="1";
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Updated] (AIRFLOW-1249) Running tasks from backfills with unmet conditions are stuck running
[ https://issues.apache.org/jira/browse/AIRFLOW-1249?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dan Davydov updated AIRFLOW-1249: - Description: Running tasks from backfills with unmet conditions are stuck running. E.g. create a DAG and pause it, mark the dagrun for a specific date as failed in the UI, and then start a backfill for that date. Note that the task is stuck in the running or queued state with no start/end date set. I believe this occurs for the Celery executor but it may affect other/all executors. This can e.g. cause pools to fill up. was: Running tasks from backfills with unmet conditions are stuck running. E.g. create a DAG and pause it, mark the dagrun for a specific date as failed in the UI, and then start a backfill for that date. Note that the task is stuck in the running state with no start/end date set. I believe this occurs for the Celery executor but it may affect other/all executors. This can e.g. cause pools to fill up. > Running tasks from backfills with unmet conditions are stuck running > > > Key: AIRFLOW-1249 > URL: https://issues.apache.org/jira/browse/AIRFLOW-1249 > Project: Apache Airflow > Issue Type: Bug > Reporter: Dan Davydov > > Running tasks from backfills with unmet conditions are stuck running. E.g. > create a DAG and pause it, mark the dagrun for a specific date as failed in > the UI, and then start a backfill for that date. Note that the task is stuck > in the running or queued state with no start/end date set. I believe this > occurs for the Celery executor but it may affect other/all executors. > This can e.g. cause pools to fill up. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Updated] (AIRFLOW-1249) Running tasks from backfills with unmet conditions are stuck running
[
https://issues.apache.org/jira/browse/AIRFLOW-1249?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Dan Davydov updated AIRFLOW-1249:
-
Description:
Running tasks from backfills with unmet conditions are stuck running. E.g.
create a DAG and pause it, mark the dagrun for a specific date as failed in the
UI, and then start a backfill for that date. Note that the task is stuck in the
running or queued state with no start/end date set. I believe this occurs for
the Celery executor but it may affect other/all executors.
This can e.g. cause pools to fill up.
This query cleans up the tasks as a workaround:
{code}
DELETE ti FROM task_instance ti
JOIN dag_run dr
ON ti.execution_date = dr.execution_date AND
ti.dag_id = dr.dag_id
JOIN dag dg
ON dr.dag_id=dg.dag_id
WHERE ISNULL(ti.start_date) and (ti.state="queued" or
ti.state="running")
and dr.state="failed" and (dg.is_paused="1" or dg.is_active="0");
{code}
was:
Running tasks from backfills with unmet conditions are stuck running. E.g.
create a DAG and pause it, mark the dagrun for a specific date as failed in the
UI, and then start a backfill for that date. Note that the task is stuck in the
running or queued state with no start/end date set. I believe this occurs for
the Celery executor but it may affect other/all executors.
This can e.g. cause pools to fill up.
This query cleans up the tasks as a workaround:
{code}
DELETE ti FROM task_instance ti
JOIN dag_run dr
ON ti.execution_date = dr.execution_date AND
ti.dag_id = dr.dag_id
JOIN dag dg
ON dr.dag_id=dg.dag_id
WHERE ISNULL(ti.start_date) and (ti.state="queued" or
ti.state="running")
and dr.state="failed" and dg.is_paused="1";
{code}
> Running tasks from backfills with unmet conditions are stuck running
>
>
> Key: AIRFLOW-1249
> URL: https://issues.apache.org/jira/browse/AIRFLOW-1249
> Project: Apache Airflow
> Issue Type: Bug
>Reporter: Dan Davydov
>
> Running tasks from backfills with unmet conditions are stuck running. E.g.
> create a DAG and pause it, mark the dagrun for a specific date as failed in
> the UI, and then start a backfill for that date. Note that the task is stuck
> in the running or queued state with no start/end date set. I believe this
> occurs for the Celery executor but it may affect other/all executors.
> This can e.g. cause pools to fill up.
> This query cleans up the tasks as a workaround:
> {code}
> DELETE ti FROM task_instance ti
> JOIN dag_run dr
> ON ti.execution_date = dr.execution_date AND
>ti.dag_id = dr.dag_id
>JOIN dag dg
>ON dr.dag_id=dg.dag_id
> WHERE ISNULL(ti.start_date) and (ti.state="queued" or
> ti.state="running")
> and dr.state="failed" and (dg.is_paused="1" or dg.is_active="0");
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Created] (AIRFLOW-1263) Airflow Task Tries Page
Dan Davydov created AIRFLOW-1263: Summary: Airflow Task Tries Page Key: AIRFLOW-1263 URL: https://issues.apache.org/jira/browse/AIRFLOW-1263 Project: Apache Airflow Issue Type: Bug Components: webserver Reporter: Dan Davydov Currently Airflow has a fixed height of 600px for the airflow task tries page (admin/airflow/tries) for DAGs. This becomes unreadable for large dags where the legend squishes the chart itself to the bottom since there are so many tasks. To fix this we might need to fix the charting library dependency we have to allow for dynamic heights. It's possible there are other airflow charts that are affected by this bug too. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Created] (AIRFLOW-1266) Long task names are truncated in gannt view
Dan Davydov created AIRFLOW-1266: Summary: Long task names are truncated in gannt view Key: AIRFLOW-1266 URL: https://issues.apache.org/jira/browse/AIRFLOW-1266 Project: Apache Airflow Issue Type: Bug Components: webserver Reporter: Dan Davydov Long task names are truncated in gannt view, we should make the text smaller instead. To reproduce create a task called abczzzdef and note that the start/end characters are not fully visible in the gant view. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Updated] (AIRFLOW-1263) Airflow Charts Pages Should Have Dynamic Heights
[ https://issues.apache.org/jira/browse/AIRFLOW-1263?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dan Davydov updated AIRFLOW-1263: - Summary: Airflow Charts Pages Should Have Dynamic Heights (was: Airflow Task Tries Page) > Airflow Charts Pages Should Have Dynamic Heights > > > Key: AIRFLOW-1263 > URL: https://issues.apache.org/jira/browse/AIRFLOW-1263 > Project: Apache Airflow > Issue Type: Bug > Components: webserver > Reporter: Dan Davydov > > Currently Airflow has a fixed height of 600px for the airflow task tries page > (admin/airflow/tries) for DAGs. This becomes unreadable for large dags where > the legend squishes the chart itself to the bottom since there are so many > tasks. To fix this we might need to fix the charting library dependency we > have to allow for dynamic heights. > It's possible there are other airflow charts that are affected by this bug > too. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Updated] (AIRFLOW-1263) Airflow Charts Pages Should Have Dynamic Heights
[ https://issues.apache.org/jira/browse/AIRFLOW-1263?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dan Davydov updated AIRFLOW-1263: - Description: Currently Airflow has a fixed height of 600px for the airflow task tries page and the other chart pages (e.g. admin/airflow/tries) for DAGs. This becomes unreadable for large dags where the legend squishes the chart itself to the bottom since there are so many tasks. To fix this we might need to fix the charting library dependency we have to allow for dynamic heights. (was: Currently Airflow has a fixed height of 600px for the airflow task tries page (admin/airflow/tries) for DAGs. This becomes unreadable for large dags where the legend squishes the chart itself to the bottom since there are so many tasks. To fix this we might need to fix the charting library dependency we have to allow for dynamic heights. It's possible there are other airflow charts that are affected by this bug too.) > Airflow Charts Pages Should Have Dynamic Heights > > > Key: AIRFLOW-1263 > URL: https://issues.apache.org/jira/browse/AIRFLOW-1263 > Project: Apache Airflow > Issue Type: Bug > Components: webserver > Reporter: Dan Davydov > > Currently Airflow has a fixed height of 600px for the airflow task tries page > and the other chart pages (e.g. admin/airflow/tries) for DAGs. This becomes > unreadable for large dags where the legend squishes the chart itself to the > bottom since there are so many tasks. To fix this we might need to fix the > charting library dependency we have to allow for dynamic heights. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Updated] (AIRFLOW-1288) Bad owners field in DAGs breaks Airflow front page
[
https://issues.apache.org/jira/browse/AIRFLOW-1288?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Dan Davydov updated AIRFLOW-1288:
-
Description:
DAGs that have owners set to a bad value break the front page of the webserver
with an error like below. Instead these should just cause import errors for the
specific dags in question.
{code}
Ooops.
/ ( () ) \___
/( ( ( ) _)) ) )\
(( ( )() ) ( ) )
((/ ( _( ) ( _) ) ( () ) )
( ( ( (_) ((( ) .((_ ) . )_
( ( )( ( )) ) . ) ( )
( ( ( ( ) ( _ ( _) ). ) . ) ) ( )
( ( ( ) ( ) ( )) ) _)( ) ) )
( ( ( \ ) ((_ ( ) ( ) ) ) ) )) ( )
( ( ( ( (_ ( ) ( _) ) ( ) ) )
( ( ( ( ( ) (_ ) ) ) _) ) _( ( )
(( ( )(( _) _) _(_ ( (_ )
(_((__(_(__(( ( ( | ) ) ) )_))__))_)___)
((__)\\||lll|l||/// \_))
( /(/ ( ) ) )\ )
(( ( ( | | ) ) )\ )
( /(| / ( )) ) ) )) )
( ( _(|)_) )
( ||\(|(|)|/|| )
(|(||(||))
( //|/l|||)|\\ \ )
(/ / // /|//\\ \ \ \ _)
---
Node: i-0dbbddfb63fb2cfbc.inst.aws.airbnb.com
---
Traceback (most recent call last):
File "/usr/local/lib/python2.7/dist-packages/flask/app.py", line 1817, in
wsgi_app
response = self.full_dispatch_request()
File "/usr/local/lib/python2.7/dist-packages/flask/app.py", line 1477, in
full_dispatch_request
rv = self.handle_user_exception(e)
File
"/usr/local/lib/python2.7/dist-packages/newrelic/hooks/framework_flask.py",
line 103, in _nr_wrapper_Flask_handle_exception_
return wrapped(*args, **kwargs)
File "/usr/local/lib/python2.7/dist-packages/flask/app.py", line 1381, in
handle_user_exception
reraise(exc_type, exc_value, tb)
File "/usr/local/lib/python2.7/dist-packages/flask/app.py", line 1475, in
full_dispatch_request
rv = self.dispatch_request()
File "/usr/local/lib/python2.7/dist-packages/flask/app.py", line 1461, in
dispatch_request
return self.view_functions[rule.endpoint](**req.view_args)
File
"/usr/local/lib/python2.7/dist-packages/newrelic/hooks/framework_flask.py",
line 40, in _nr_wrapper_handler_
return wrapped(*args, **kwargs)
File "/usr/local/lib/python2.7/dist-packages/flask_admin/base.py", line 68,
in inner
return self._run_view(f, *args, **kwargs)
File "/usr/local/lib/python2.7/dist-packages/flask_admin/base.py", line 367,
in _run_view
return fn(self, *args, **kwargs)
File "/usr/local/lib/python2.7/dist-packages/flask_login.py", line 758, in
decorated_view
return func(*args, **kwargs)
File "/usr/local/lib/python2.7/dist-packages/airflow/www/views.py", line
1909, in index
all_dag_ids=all_dag_ids)
File "/usr/local/lib/python2.7/dist-packages/flask_admin/base.py", line 307,
in render
return render_template(template, **kwargs)
File "/usr/local/lib/python2.7/dist-packages/newrelic/api/function_trace.py",
line 110, in literal_wrapper
return wrapped(*args, **kwargs)
File "/usr/local/lib/python2.7/dist-packages/flask/templating.py", line 128,
in render_template
context, ctx.app)
File "/usr/local/lib/python2.7/dist-packages/flask/templating.py", line 110,
in _render
rv = template.render(context)
File "/usr/local/lib/python2.7/dist-packages/newrelic/api/function_trace.py",
line 98, in dynamic_wrapper
return wrapped(*args, **kwargs)
File "/usr/local/lib/python2.7/dist-packages/jinja2/environment.py", line
989, in render
return self.environment.handle_exception(exc_info, True)
File "/usr/local/lib/python2.7/dist-packages/jinja2/environment.py", line
754, in handle_exception
reraise(exc_type, exc_value, tb)
File
"/usr/local/lib/python2.7/dist-packages/airflow/www/templates/airflow/dags.html",
line 18, in top-level template code
{% extends "airflow/master.html" %}
File
"/usr/local/lib/python2.7/dist-packages/airflow/www/templates/airflow/master.html",
line 18, in top-level template code
{% extends "admin/master.html" %}
File
"/usr/local/lib/
[jira] [Created] (AIRFLOW-1288) Bad owners field in DAGs breaks Airflow front page
Dan Davydov created AIRFLOW-1288:
Summary: Bad owners field in DAGs breaks Airflow front page
Key: AIRFLOW-1288
URL: https://issues.apache.org/jira/browse/AIRFLOW-1288
Project: Apache Airflow
Issue Type: Bug
Components: webserver
Reporter: Dan Davydov
DAGs that have owners set to a bad value break the front page of the webserver
with an error like below. Instead these should just cause import errors for the
specific dags in question.
{{code}}
Ooops.
/ ( () ) \___
/( ( ( ) _)) ) )\
(( ( )() ) ( ) )
((/ ( _( ) ( _) ) ( () ) )
( ( ( (_) ((( ) .((_ ) . )_
( ( )( ( )) ) . ) ( )
( ( ( ( ) ( _ ( _) ). ) . ) ) ( )
( ( ( ) ( ) ( )) ) _)( ) ) )
( ( ( \ ) ((_ ( ) ( ) ) ) ) )) ( )
( ( ( ( (_ ( ) ( _) ) ( ) ) )
( ( ( ( ( ) (_ ) ) ) _) ) _( ( )
(( ( )(( _) _) _(_ ( (_ )
(_((__(_(__(( ( ( | ) ) ) )_))__))_)___)
((__)\\||lll|l||/// \_))
( /(/ ( ) ) )\ )
(( ( ( | | ) ) )\ )
( /(| / ( )) ) ) )) )
( ( _(|)_) )
( ||\(|(|)|/|| )
(|(||(||))
( //|/l|||)|\\ \ )
(/ / // /|//\\ \ \ \ _)
---
Node: i-0dbbddfb63fb2cfbc.inst.aws.airbnb.com
---
Traceback (most recent call last):
File "/usr/local/lib/python2.7/dist-packages/flask/app.py", line 1817, in
wsgi_app
response = self.full_dispatch_request()
File "/usr/local/lib/python2.7/dist-packages/flask/app.py", line 1477, in
full_dispatch_request
rv = self.handle_user_exception(e)
File
"/usr/local/lib/python2.7/dist-packages/newrelic/hooks/framework_flask.py",
line 103, in _nr_wrapper_Flask_handle_exception_
return wrapped(*args, **kwargs)
File "/usr/local/lib/python2.7/dist-packages/flask/app.py", line 1381, in
handle_user_exception
reraise(exc_type, exc_value, tb)
File "/usr/local/lib/python2.7/dist-packages/flask/app.py", line 1475, in
full_dispatch_request
rv = self.dispatch_request()
File "/usr/local/lib/python2.7/dist-packages/flask/app.py", line 1461, in
dispatch_request
return self.view_functions[rule.endpoint](**req.view_args)
File
"/usr/local/lib/python2.7/dist-packages/newrelic/hooks/framework_flask.py",
line 40, in _nr_wrapper_handler_
return wrapped(*args, **kwargs)
File "/usr/local/lib/python2.7/dist-packages/flask_admin/base.py", line 68,
in inner
return self._run_view(f, *args, **kwargs)
File "/usr/local/lib/python2.7/dist-packages/flask_admin/base.py", line 367,
in _run_view
return fn(self, *args, **kwargs)
File "/usr/local/lib/python2.7/dist-packages/flask_login.py", line 758, in
decorated_view
return func(*args, **kwargs)
File "/usr/local/lib/python2.7/dist-packages/airflow/www/views.py", line
1909, in index
all_dag_ids=all_dag_ids)
File "/usr/local/lib/python2.7/dist-packages/flask_admin/base.py", line 307,
in render
return render_template(template, **kwargs)
File "/usr/local/lib/python2.7/dist-packages/newrelic/api/function_trace.py",
line 110, in literal_wrapper
return wrapped(*args, **kwargs)
File "/usr/local/lib/python2.7/dist-packages/flask/templating.py", line 128,
in render_template
context, ctx.app)
File "/usr/local/lib/python2.7/dist-packages/flask/templating.py", line 110,
in _render
rv = template.render(context)
File "/usr/local/lib/python2.7/dist-packages/newrelic/api/function_trace.py",
line 98, in dynamic_wrapper
return wrapped(*args, **kwargs)
File "/usr/local/lib/python2.7/dist-packages/jinja2/environment.py", line
989, in render
return self.environment.handle_exception(exc_info, True)
File "/usr/local/lib/python2.7/dist-packages/jinja2/environment.py", line
754, in handle_exception
reraise(exc_type, exc_value, tb)
File
"/usr/local/lib/python2.7/dist-packages/airflow/www/templates/airflow/dags.html",
line 18, in top-level template code
{% extends "airflow/master.html" %}
File
"/usr/local/lib/python2.7/dis
[jira] [Created] (AIRFLOW-1257) Manually Created Dagruns Are Broken In Multiple Ways
Dan Davydov created AIRFLOW-1257: Summary: Manually Created Dagruns Are Broken In Multiple Ways Key: AIRFLOW-1257 URL: https://issues.apache.org/jira/browse/AIRFLOW-1257 Project: Apache Airflow Issue Type: Bug Reporter: Dan Davydov Manually triggered dagruns via the UI (the first button in "Links" on the main airflow page) have issues, e.g. you can't mark-success tasks in that dagrun (when you try an empty list of tasks show up), and in the webserver they appear without a border. This could be because the dagrun execution date is not aligned with the DAG. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Created] (AIRFLOW-1311) Improve Webserver Load Time For Large DAGs
Dan Davydov created AIRFLOW-1311: Summary: Improve Webserver Load Time For Large DAGs Key: AIRFLOW-1311 URL: https://issues.apache.org/jira/browse/AIRFLOW-1311 Project: Apache Airflow Issue Type: Bug Components: webserver Reporter: Dan Davydov Large DAGs can take an extremely long time to load in the Airflow UI (minutes/timeout). The fixes are as follows: 1. Lazy load DAGs (load up to a certain # of tasks by default, prioritizing tasks by their depth, and allow users to expand sections for these DAGs, ideally prefetch deeper tasks once the initial set of tasks has rendered ) 2. Identify bottlenecks/performance issues in both the frontend/backend for rendering DAGs on the webserver and fix them. Airflow should be more performant for displaying DAGs that are somewhat large, e.g. DAGs that have up to 500 nodes and 2000 edges (dependencies from one task to another) should render within a couple of seconds. 3. Make DAG loading asynchronous in the UI (once the top-level tasks have loaded display them immediately). We might not want to do this as users might try to click something only to have the UI change from underneath them [~saguziel] -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Created] (AIRFLOW-1280) Gantt Chart Height Isn't Set Properly
Dan Davydov created AIRFLOW-1280: Summary: Gantt Chart Height Isn't Set Properly Key: AIRFLOW-1280 URL: https://issues.apache.org/jira/browse/AIRFLOW-1280 Project: Apache Airflow Issue Type: Bug Components: webserver Reporter: Dan Davydov The Gantt view has a dynamic height for the chart body based on the number of tasks but it's only a heuristic. Instead we should calculate the height of the chart legend and make the height of the chart equal to the height of the legend + desired height for the chart body. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
