Hi All,
We are running master in our environment and have noticed something new (that
wasn't present in release 1.7.1.3).
I have the following DAG, which I ran a backfill on:
# ------ BuildTask6 ------
BuildTask7 ------ BuildTask8 ------ BuildTask9
# /
/
# InitBuildTask ------ BuildTask1 ------ BuildTask2 ------ BuildTask3
/
# \ \
/
# ------ BuildTask4 ------------------------------------------
BuildTask5
#
When the backfill begins, the outputs gives warnings for all tasks that have
not yet had their dependencies met (full output attached) i.e. all tasks except
the one task that has no dependencies. This continues until the backfill has
completed. Is this expected behaviour?
Cheers,
Luke Maycock
OLIVER WYMAN
[email protected]<mailto:[email protected]>
www.oliverwyman.com<http://www.oliverwyman.com/>
________________________________
From: Chris Riccomini <[email protected]>
Sent: 29 September 2016 18:14:02
To: [email protected]
Subject: Re: Airflow Releases
Hey Luke,
> Is there anything we can do to help get the next release in place?
One thing that would definitely help is running master somewhere in
your environment, and reporting any issues that see. Over the next few
weeks, AirBNB and a few other folks will be doing the same in an
effort to harden the 1.8 release.
Cheers,
Chris
On Thu, Sep 29, 2016 at 3:08 AM, Maycock, Luke
<[email protected]> wrote:
> Airflow Developers,
>
>
> We were looking at writing a workflow framework in Python when we found
> Airflow. We have carried out some proof of concept work for using Airflow and
> wish to continue using it as it comes with lots of great features
> out-of-the-box.
>
>
> We have created our own fork here:
>
> https://github.com/owlabs/incubator-airflow
>
>
> So far, the only thing we have committed back to the main repository is the
> following fix to the mssql_hook:
>
> https://github.com/apache/incubator-airflow/pull/1626
>
>
> Among other types of tasks, we wish to be able to run mssql tasks using
> Airflow. In order to do so, the above and below fixes are required:
>
> https://github.com/apache/incubator-airflow/pull/1458<https://github.com/apache/incubator-airflow/pull/1458/commits/e7e655fde3c29742149d047028cbb21aecba86ed>
>
>
> We have created a Chef cookbook for configuring VMs with Airflow and its
> prerequisites. As part of this cookbook, we are installing the latest release
> of Airflow. However, it appears that the latest release does not have the
> aforementioned fixes.
>
> Do you know when the next release of Airflow is expected? Is there anything
> we can do to help get the next release in place?
>
>
> Luke Maycock
> OLIVER WYMAN
> [email protected]<mailto:[email protected]>
> www.oliverwyman.com<http://www.oliverwyman.com/>
>
>
> ________________________________
> This e-mail and any attachments may be confidential or legally privileged. If
> you received this message in error or are not the intended recipient, you
> should destroy the e-mail message and any attachments or copies, and you are
> prohibited from retaining, distributing, disclosing or using any information
> contained herein. Please inform us of the erroneous delivery by return
> e-mail. Thank you for your cooperation.
________________________________
This e-mail and any attachments may be confidential or legally privileged. If
you received this message in error or are not the intended recipient, you
should destroy the e-mail message and any attachments or copies, and you are
prohibited from retaining, distributing, disclosing or using any information
contained herein. Please inform us of the erroneous delivery by return e-mail.
Thank you for your cooperation.
DAG:
#Define dependencies
#
# ------ BuildTask6 ------
BuildTask7 ------ BuildTask8 ------ BuildTask9
# /
/
# InitBuildTask ------ BuildTask1 ------ BuildTask2 ------ BuildTask3
/
# \ \
/
# ------ BuildTask4 ------------------------------------------
BuildTask5
#
[2016-10-03 09:52:26,996] {models.py:168} INFO - Filling up the DagBag from
/usr/local/lib/airflow/dags
[2016-10-03 09:52:27,378] {jobs.py:1680} INFO - Checking run <DagRun sqlDag @
2016-10-02 00:00:00: backfill_2016-10-02T00:00:00, externally triggered: False>
[2016-10-03 09:52:27,387] {models.py:1032} WARNING - Dependencies not met for
<TaskInstance: sqlDag.Build_Task_5 2016-10-02 00:00:00 [None]>, dependency
'Trigger Rule' FAILED: Task's trigger rule 'all_success' requires all upstream
tasks to have succeeded, but found 2 non-success(es).
[2016-10-03 09:52:27,396] {models.py:1032} WARNING - Dependencies not met for
<TaskInstance: sqlDag.Build_Task_2 2016-10-02 00:00:00 [None]>, dependency
'Trigger Rule' FAILED: Task's trigger rule 'all_success' requires all upstream
tasks to have succeeded, but found 1 non-success(es).
[2016-10-03 09:52:27,414] {models.py:1032} WARNING - Dependencies not met for
<TaskInstance: sqlDag.Build_Task_6 2016-10-02 00:00:00 [None]>, dependency
'Trigger Rule' FAILED: Task's trigger rule 'all_success' requires all upstream
tasks to have succeeded, but found 1 non-success(es).
[2016-10-03 09:52:27,430] {models.py:1032} WARNING - Dependencies not met for
<TaskInstance: sqlDag.Build_Task_3 2016-10-02 00:00:00 [None]>, dependency
'Trigger Rule' FAILED: Task's trigger rule 'all_success' requires all upstream
tasks to have succeeded, but found 1 non-success(es).
[2016-10-03 09:52:27,444] {models.py:1032} WARNING - Dependencies not met for
<TaskInstance: sqlDag.Build_Task_8 2016-10-02 00:00:00 [None]>, dependency
'Trigger Rule' FAILED: Task's trigger rule 'all_success' requires all upstream
tasks to have succeeded, but found 2 non-success(es).
[2016-10-03 09:52:27,478] {models.py:1032} WARNING - Dependencies not met for
<TaskInstance: sqlDag.Build_Task_7 2016-10-02 00:00:00 [None]>, dependency
'Trigger Rule' FAILED: Task's trigger rule 'all_success' requires all upstream
tasks to have succeeded, but found 1 non-success(es).
[2016-10-03 09:52:27,514] {models.py:1032} WARNING - Dependencies not met for
<TaskInstance: sqlDag.Build_Task_9 2016-10-02 00:00:00 [None]>, dependency
'Trigger Rule' FAILED: Task's trigger rule 'all_success' requires all upstream
tasks to have succeeded, but found 1 non-success(es).
[2016-10-03 09:52:27,525] {models.py:1032} WARNING - Dependencies not met for
<TaskInstance: sqlDag.Build_Task_4 2016-10-02 00:00:00 [None]>, dependency
'Trigger Rule' FAILED: Task's trigger rule 'all_success' requires all upstream
tasks to have succeeded, but found 1 non-success(es).
[2016-10-03 09:52:27,536] {models.py:1035} INFO - Dependencies all met for
<TaskInstance: sqlDag.InitBuild 2016-10-02 00:00:00 [None]> [2016-10-03
09:52:27,548] {base_executor.py:50} INFO - Adding to queue: airflow run sqlDag
InitBuild 2016-10-02T00:00:00 --pickle 31 --local
[2016-10-03 09:52:27,561] {models.py:1032} WARNING - Dependencies not met for
<TaskInstance: sqlDag.Build_Task_1 2016-10-02 00:00:00 [None]>, dependency
'Trigger Rule' FAILED: Task's trigger rule 'all_success' requires all upstream
tasks to have succeeded, but found 1 non-success(es).
[2016-10-03 09:52:32,309] {celery_executor.py:78} INFO - [celery] queuing
(u'sqlDag', u'InitBuild', datetime.datetime(2016, 10, 2, 0, 0)) through celery,
queue=sql_queue_zzz
[2016-10-03 09:52:32,363] {jobs.py:1854} INFO - [backfill progress] | dag run 1
of 1 | tasks waiting: 10 | succeeded: 0 | kicked_off: 1 | failed: 0 | skipped:
0 | deadlocked: 0
[2016-10-03 09:52:32,379] {models.py:1032} WARNING - Dependencies not met for
<TaskInstance: sqlDag.Build_Task_5 2016-10-02 00:00:00 [None]>, dependency
'Trigger Rule' FAILED: Task's trigger rule 'all_success' requires all upstream
tasks to have succeeded, but found 2 non-success(es).
[2016-10-03 09:52:32,419] {models.py:1032} WARNING - Dependencies not met for
<TaskInstance: sqlDag.Build_Task_2 2016-10-02 00:00:00 [None]>, dependency
'Trigger Rule' FAILED: Task's trigger rule 'all_success' requires all upstream
tasks to have succeeded, but found 1 non-success(es).
[2016-10-03 09:52:32,462] {models.py:1032} WARNING - Dependencies not met for
<TaskInstance: sqlDag.Build_Task_6 2016-10-02 00:00:00 [None]>, dependency
'Trigger Rule' FAILED: Task's trigger rule 'all_success' requires all upstream
tasks to have succeeded, but found 1 non-success(es).
[2016-10-03 09:52:32,477] {models.py:1032} WARNING - Dependencies not met for
<TaskInstance: sqlDag.Build_Task_3 2016-10-02 00:00:00 [None]>, dependency
'Trigger Rule' FAILED: Task's trigger rule 'all_success' requires all upstream
tasks to have succeeded, but found 1 non-success(es).
[2016-10-03 09:52:32,504] {models.py:1032} WARNING - Dependencies not met for
<TaskInstance: sqlDag.Build_Task_8 2016-10-02 00:00:00 [None]>, dependency
'Trigger Rule' FAILED: Task's trigger rule 'all_success' requires all upstream
tasks to have succeeded, but found 2 non-success(es).
[2016-10-03 09:52:32,530] {models.py:1032} WARNING - Dependencies not met for
<TaskInstance: sqlDag.Build_Task_7 2016-10-02 00:00:00 [None]>, dependency
'Trigger Rule' FAILED: Task's trigger rule 'all_success' requires all upstream
tasks to have succeeded, but found 1 non-success(es).
[2016-10-03 09:52:32,552] {models.py:1032} WARNING - Dependencies not met for
<TaskInstance: sqlDag.Build_Task_9 2016-10-02 00:00:00 [None]>, dependency
'Trigger Rule' FAILED: Task's trigger rule 'all_success' requires all upstream
tasks to have succeeded, but found 1 non-success(es).
[2016-10-03 09:52:32,583] {models.py:1032} WARNING - Dependencies not met for
<TaskInstance: sqlDag.Build_Task_4 2016-10-02 00:00:00 [None]>, dependency
'Trigger Rule' FAILED: Task's trigger rule 'all_success' requires all upstream
tasks to have succeeded, but found 1 non-success(es).
[2016-10-03 09:52:32,604] {models.py:1035} INFO - Dependencies all met for
<TaskInstance: sqlDag.InitBuild 2016-10-02 00:00:00 [scheduled]>
[2016-10-03 09:52:32,634] {models.py:1032} WARNING - Dependencies not met for
<TaskInstance: sqlDag.Build_Task_1 2016-10-02 00:00:00 [None]>, dependency
'Trigger Rule' FAILED: Task's trigger rule 'all_success' requires all upstream
tasks to have succeeded, but found 1 non-success(es).
[2016-10-03 09:52:33,110] {__init__.py:50} INFO - Using executor CeleryExecutor
[2016-10-03 09:52:33,172] {driver.py:120} INFO - Generating grammar tables from
/usr/lib/python2.7/lib2to3/Grammar.txt
[2016-10-03 09:52:33,219] {driver.py:120} INFO - Generating grammar tables from
/usr/lib/python2.7/lib2to3/PatternGrammar.txt
Logging into: /usr/local/lib/airflow/logs/sqlDag/InitBuild/2016-10-02T00:00:00
[2016-10-03 09:52:34,103] {__init__.py:50} INFO - Using executor CeleryExecutor
[2016-10-03 09:52:34,234] {driver.py:120} INFO - Generating grammar tables from
/usr/lib/python2.7/lib2to3/Grammar.txt
[2016-10-03 09:52:34,269] {driver.py:120} INFO - Generating grammar tables from
/usr/lib/python2.7/lib2to3/PatternGrammar.txt
[2016-10-03 09:52:37,307] {jobs.py:1854} INFO - [backfill progress] | dag run 1
of 1 | tasks waiting: 10 | succeeded: 0 | kicked_off: 1 | failed: 0 | skipped:
0 | deadlocked: 0
[2016-10-03 09:52:37,316] {models.py:1032} WARNING - Dependencies not met for
<TaskInstance: sqlDag.Build_Task_5 2016-10-02 00:00:00 [None]>, dependency
'Trigger Rule' FAILED: Task's trigger rule 'all_success' requires all upstream
tasks to have succeeded, but found 2 non-success(es).
[2016-10-03 09:52:37,336] {models.py:1032} WARNING - Dependencies not met for
<TaskInstance: sqlDag.Build_Task_2 2016-10-02 00:00:00 [None]>, dependency
'Trigger Rule' FAILED: Task's trigger rule 'all_success' requires all upstream
tasks to have succeeded, but found 1 non-success(es).
[2016-10-03 09:52:37,345] {models.py:1032} WARNING - Dependencies not met for
<TaskInstance: sqlDag.Build_Task_6 2016-10-02 00:00:00 [None]>, dependency
'Trigger Rule' FAILED: Task's trigger rule 'all_success' requires all upstream
tasks to have succeeded, but found 1 non-success(es).
[2016-10-03 09:52:37,384] {models.py:1032} WARNING - Dependencies not met for
<TaskInstance: sqlDag.Build_Task_3 2016-10-02 00:00:00 [None]>, dependency
'Trigger Rule' FAILED: Task's trigger rule 'all_success' requires all upstream
tasks to have succeeded, but found 1 non-success(es).
[2016-10-03 09:52:37,458] {models.py:1032} WARNING - Dependencies not met for
<TaskInstance: sqlDag.Build_Task_8 2016-10-02 00:00:00 [None]>, dependency
'Trigger Rule' FAILED: Task's trigger rule 'all_success' requires all upstream
tasks to have succeeded, but found 2 non-success(es).
[2016-10-03 09:52:37,468] {models.py:1032} WARNING - Dependencies not met for
<TaskInstance: sqlDag.Build_Task_7 2016-10-02 00:00:00 [None]>, dependency
'Trigger Rule' FAILED: Task's trigger rule 'all_success' requires all upstream
tasks to have succeeded, but found 1 non-success(es).
[2016-10-03 09:52:37,483] {models.py:1032} WARNING - Dependencies not met for
<TaskInstance: sqlDag.Build_Task_9 2016-10-02 00:00:00 [None]>, dependency
'Trigger Rule' FAILED: Task's trigger rule 'all_success' requires all upstream
tasks to have succeeded, but found 1 non-success(es).
[2016-10-03 09:52:37,510] {models.py:1035} INFO - Dependencies all met for
<TaskInstance: sqlDag.Build_Task_4 2016-10-02 00:00:00 [None]>
[2016-10-03 09:52:37,515] {base_executor.py:50} INFO - Adding to queue: airflow
run sqlDag Build_Task_4 2016-10-02T00:00:00 --pickle 31 --local
[2016-10-03 09:52:37,613] {models.py:1035} INFO - Dependencies all met for
<TaskInstance: sqlDag.Build_Task_1 2016-10-02 00:00:00 [None]>
[2016-10-03 09:52:37,629] {base_executor.py:50} INFO - Adding to queue: airflow
run sqlDag Build_Task_1 2016-10-02T00:00:00 --pickle 31 --local
[2016-10-03 09:52:42,316] {celery_executor.py:78} INFO - [celery] queuing
(u'sqlDag', u'Build_Task_1', datetime.datetime(2016, 10, 2, 0, 0)) through
celery, queue=sql_queue_zzz
[2016-10-03 09:52:42,339] {celery_executor.py:78} INFO - [celery] queuing
(u'sqlDag', u'Build_Task_4', datetime.datetime(2016, 10, 2, 0, 0)) through
celery, queue=sql_queue_zzz
[2016-10-03 09:52:42,398] {jobs.py:1854} INFO - [backfill progress] | dag run 1
of 1 | tasks waiting: 9 | succeeded: 1 | kicked_off: 3 | failed: 0 | skipped: 0
| deadlocked: 0
[2016-10-03 09:52:42,451] {models.py:1032} WARNING - Dependencies not met for
<TaskInstance: sqlDag.Build_Task_5 2016-10-02 00:00:00 [None]>, dependency
'Trigger Rule' FAILED: Task's trigger rule 'all_success' requires all upstream
tasks to have succeeded, but found 2 non-success(es).
[2016-10-03 09:52:42,491] {models.py:1032} WARNING - Dependencies not met for
<TaskInstance: sqlDag.Build_Task_2 2016-10-02 00:00:00 [None]>, dependency
'Trigger Rule' FAILED: Task's trigger rule 'all_success' requires all upstream
tasks to have succeeded, but found 1 non-success(es).
[2016-10-03 09:52:42,530] {models.py:1032} WARNING - Dependencies not met for
<TaskInstance: sqlDag.Build_Task_6 2016-10-02 00:00:00 [None]>, dependency
'Trigger Rule' FAILED: Task's trigger rule 'all_success' requires all upstream
tasks to have succeeded, but found 1 non-success(es).
[2016-10-03 09:52:42,605] {models.py:1032} WARNING - Dependencies not met for
<TaskInstance: sqlDag.Build_Task_3 2016-10-02 00:00:00 [None]>, dependency
'Trigger Rule' FAILED: Task's trigger rule 'all_success' requires all upstream
tasks to have succeeded, but found 1 non-success(es).
[2016-10-03 09:52:42,655] {models.py:1032} WARNING - Dependencies not met for
<TaskInstance: sqlDag.Build_Task_8 2016-10-02 00:00:00 [None]>, dependency
'Trigger Rule' FAILED: Task's trigger rule 'all_success' requires all upstream
tasks to have succeeded, but found 2 non-success(es).
[2016-10-03 09:52:42,718] {models.py:1032} WARNING - Dependencies not met for
<TaskInstance: sqlDag.Build_Task_7 2016-10-02 00:00:00 [None]>, dependency
'Trigger Rule' FAILED: Task's trigger rule 'all_success' requires all upstream
tasks to have succeeded, but found 1 non-success(es).
[2016-10-03 09:52:42,751] {models.py:1032} WARNING - Dependencies not met for
<TaskInstance: sqlDag.Build_Task_9 2016-10-02 00:00:00 [None]>, dependency
'Trigger Rule' FAILED: Task's trigger rule 'all_success' requires all upstream
tasks to have succeeded, but found 1 non-success(es).
[2016-10-03 09:52:42,835] {models.py:1035} INFO - Dependencies all met for
<TaskInstance: sqlDag.Build_Task_4 2016-10-02 00:00:00 [scheduled]>
[2016-10-03 09:52:42,919] {models.py:1035} INFO - Dependencies all met for
<TaskInstance: sqlDag.Build_Task_1 2016-10-02 00:00:00 [scheduled]>
[2016-10-03 09:52:43,813] {__init__.py:50} INFO - Using executor CeleryExecutor
[2016-10-03 09:52:43,851] {__init__.py:50} INFO - Using executor CeleryExecutor
[2016-10-03 09:52:44,036] {driver.py:120} INFO - Generating grammar tables from
/usr/lib/python2.7/lib2to3/Grammar.txt
[2016-10-03 09:52:44,049] {driver.py:120} INFO - Generating grammar tables from
/usr/lib/python2.7/lib2to3/Grammar.txt
[2016-10-03 09:52:44,141] {driver.py:120} INFO - Generating grammar tables from
/usr/lib/python2.7/lib2to3/PatternGrammar.txt
[2016-10-03 09:52:44,146] {driver.py:120} INFO - Generating grammar tables from
/usr/lib/python2.7/lib2to3/PatternGrammar.txt
Logging into:
/usr/local/lib/airflow/logs/sqlDag/Build_Task_4/2016-10-02T00:00:00
Logging into:
/usr/local/lib/airflow/logs/sqlDag/Build_Task_1/2016-10-02T00:00:00
[2016-10-03 09:52:45,928] {__init__.py:50} INFO - Using executor CeleryExecutor
[2016-10-03 09:52:45,929] {__init__.py:50} INFO - Using executor CeleryExecutor
[2016-10-03 09:52:46,090] {driver.py:120} INFO - Generating grammar tables from
/usr/lib/python2.7/lib2to3/Grammar.txt
[2016-10-03 09:52:46,099] {driver.py:120} INFO - Generating grammar tables from
/usr/lib/python2.7/lib2to3/Grammar.txt
[2016-10-03 09:52:46,162] {driver.py:120} INFO - Generating grammar tables from
/usr/lib/python2.7/lib2to3/PatternGrammar.txt
[2016-10-03 09:52:46,166] {driver.py:120} INFO - Generating grammar tables from
/usr/lib/python2.7/lib2to3/PatternGrammar.txt
[2016-10-03 09:52:47 +0000] [2210] [INFO] Handling signal: ttin
[2016-10-03 09:52:47 +0000] [14515] [INFO] Booting worker with pid: 14515
[2016-10-03 09:52:47,310] {jobs.py:1854} INFO - [backfill progress] | dag run 1
of 1 | tasks waiting: 9 | succeeded: 1 | kicked_off: 3 | failed: 0 | skipped: 0
| deadlocked: 0
[2016-10-03 09:52:47,327] {models.py:1032} WARNING - Dependencies not met for
<TaskInstance: sqlDag.Build_Task_5 2016-10-02 00:00:00 [None]>, dependency
'Trigger Rule' FAILED: Task's trigger rule 'all_success' requires all upstream
tasks to have succeeded, but found 2 non-success(es).
[2016-10-03 09:52:47,359] {models.py:1032} WARNING - Dependencies not met for
<TaskInstance: sqlDag.Build_Task_2 2016-10-02 00:00:00 [None]>, dependency
'Trigger Rule' FAILED: Task's trigger rule 'all_success' requires all upstream
tasks to have succeeded, but found 1 non-success(es).
[2016-10-03 09:52:47,409] {models.py:1032} WARNING - Dependencies not met for
<TaskInstance: sqlDag.Build_Task_6 2016-10-02 00:00:00 [None]>, dependency
'Trigger Rule' FAILED: Task's trigger rule 'all_success' requires all upstream
tasks to have succeeded, but found 1 non-success(es).
[2016-10-03 09:52:47,429] {models.py:1032} WARNING - Dependencies not met for
<TaskInstance: sqlDag.Build_Task_3 2016-10-02 00:00:00 [None]>, dependency
'Trigger Rule' FAILED: Task's trigger rule 'all_success' requires all upstream
tasks to have succeeded, but found 1 non-success(es).
[2016-10-03 09:52:47,471] [14515] {models.py:168} INFO - Filling up the DagBag
from /usr/local/lib/airflow/dags
[2016-10-03 09:52:47,472] {models.py:1032} WARNING - Dependencies not met for
<TaskInstance: sqlDag.Build_Task_8 2016-10-02 00:00:00 [None]>, dependency
'Trigger Rule' FAILED: Task's trigger rule 'all_success' requires all upstream
tasks to have succeeded, but found 2 non-success(es).
[2016-10-03 09:52:47,487] {models.py:1032} WARNING - Dependencies not met for
<TaskInstance: sqlDag.Build_Task_7 2016-10-02 00:00:00 [None]>, dependency
'Trigger Rule' FAILED: Task's trigger rule 'all_success' requires all upstream
tasks to have succeeded, but found 1 non-success(es).
[2016-10-03 09:52:47,539] {models.py:1032} WARNING - Dependencies not met for
<TaskInstance: sqlDag.Build_Task_9 2016-10-02 00:00:00 [None]>, dependency
'Trigger Rule' FAILED: Task's trigger rule 'all_success' requires all upstream
tasks to have succeeded, but found 1 non-success(es).
[2016-10-03 09:52:47,555] {models.py:1032} WARNING - Dependencies not met for
<TaskInstance: sqlDag.Build_Task_4 2016-10-02 00:00:00 [running]>, dependency
'Task Instance Not Already Running' FAILED: Task is already running, it started
on 2016-10-03 09:52:46.849966.
[2016-10-03 09:52:47,566] {models.py:1032} WARNING - Dependencies not met for
<TaskInstance: sqlDag.Build_Task_1 2016-10-02 00:00:00 [running]>, dependency
'Task Instance Not Already Running' FAILED: Task is already running, it started
on 2016-10-03 09:52:46.810830.
[2016-10-03 09:52:48 +0000] [2210] [INFO] Handling signal: ttou
[2016-10-03 09:52:48 +0000] [14432] [INFO] Worker exiting (pid: 14432)
[2016-10-03 09:52:52,315] {jobs.py:1854} INFO - [backfill progress] | dag run 1
of 1 | tasks waiting: 9 | succeeded: 1 | kicked_off: 3 | failed: 0 | skipped: 0
| deadlocked: 0
[2016-10-03 09:52:52,341] {models.py:1032} WARNING - Dependencies not met for
<TaskInstance: sqlDag.Build_Task_5 2016-10-02 00:00:00 [None]>, dependency
'Trigger Rule' FAILED: Task's trigger rule 'all_success' requires all upstream
tasks to have succeeded, but found 2 non-success(es).
[2016-10-03 09:52:52,359] {models.py:1032} WARNING - Dependencies not met for
<TaskInstance: sqlDag.Build_Task_2 2016-10-02 00:00:00 [None]>, dependency
'Trigger Rule' FAILED: Task's trigger rule 'all_success' requires all upstream
tasks to have succeeded, but found 1 non-success(es).
[2016-10-03 09:52:52,382] {models.py:1032} WARNING - Dependencies not met for
<TaskInstance: sqlDag.Build_Task_6 2016-10-02 00:00:00 [None]>, dependency
'Trigger Rule' FAILED: Task's trigger rule 'all_success' requires all upstream
tasks to have succeeded, but found 1 non-success(es).
[2016-10-03 09:52:52,391] {models.py:1032} WARNING - Dependencies not met for
<TaskInstance: sqlDag.Build_Task_3 2016-10-02 00:00:00 [None]>, dependency
'Trigger Rule' FAILED: Task's trigger rule 'all_success' requires all upstream
tasks to have succeeded, but found 1 non-success(es).
[2016-10-03 09:52:52,417] {models.py:1032} WARNING - Dependencies not met for
<TaskInstance: sqlDag.Build_Task_8 2016-10-02 00:00:00 [None]>, dependency
'Trigger Rule' FAILED: Task's trigger rule 'all_success' requires all upstream
tasks to have succeeded, but found 2 non-success(es).
[2016-10-03 09:52:52,435] {models.py:1032} WARNING - Dependencies not met for
<TaskInstance: sqlDag.Build_Task_7 2016-10-02 00:00:00 [None]>, dependency
'Trigger Rule' FAILED: Task's trigger rule 'all_success' requires all upstream
tasks to have succeeded, but found 1 non-success(es).
[2016-10-03 09:52:52,450] {models.py:1032} WARNING - Dependencies not met for
<TaskInstance: sqlDag.Build_Task_9 2016-10-02 00:00:00 [None]>, dependency
'Trigger Rule' FAILED: Task's trigger rule 'all_success' requires all upstream
tasks to have succeeded, but found 1 non-success(es).
[2016-10-03 09:52:52,472] {models.py:1032} WARNING - Dependencies not met for
<TaskInstance: sqlDag.Build_Task_4 2016-10-02 00:00:00 [running]>, dependency
'Task Instance Not Already Running' FAILED: Task is already running, it started
on 2016-10-03 09:52:46.849966.
[2016-10-03 09:52:52,499] {models.py:1032} WARNING - Dependencies not met for
<TaskInstance: sqlDag.Build_Task_1 2016-10-02 00:00:00 [running]>, dependency
'Task Instance Not Already Running' FAILED: Task is already running, it started
on 2016-10-03 09:52:46.810830.
[2016-10-03 09:52:57,320] {jobs.py:1854} INFO - [backfill progress] | dag run 1
of 1 | tasks waiting: 9 | succeeded: 1 | kicked_off: 3 | failed: 0 | skipped: 0
| deadlocked: 0
[2016-10-03 09:52:57,327] {models.py:1032} WARNING - Dependencies not met for
<TaskInstance: sqlDag.Build_Task_5 2016-10-02 00:00:00 [None]>, dependency
'Trigger Rule' FAILED: Task's trigger rule 'all_success' requires all upstream
tasks to have succeeded, but found 2 non-success(es).
[2016-10-03 09:52:57,374] {models.py:1035} INFO - Dependencies all met for
<TaskInstance: sqlDag.Build_Task_2 2016-10-02 00:00:00 [None]>
[2016-10-03 09:52:57,403] {base_executor.py:50} INFO - Adding to queue: airflow
run sqlDag Build_Task_2 2016-10-02T00:00:00 --pickle 31 --local
[2016-10-03 09:52:57,426] {models.py:1032} WARNING - Dependencies not met for
<TaskInstance: sqlDag.Build_Task_6 2016-10-02 00:00:00 [None]>, dependency
'Trigger Rule' FAILED: Task's trigger rule 'all_success' requires all upstream
tasks to have succeeded, but found 1 non-success(es).
[2016-10-03 09:52:57,442] {models.py:1032} WARNING - Dependencies not met for
<TaskInstance: sqlDag.Build_Task_3 2016-10-02 00:00:00 [None]>, dependency
'Trigger Rule' FAILED: Task's trigger rule 'all_success' requires all upstream
tasks to have succeeded, but found 1 non-success(es).
[2016-10-03 09:52:57,454] {models.py:1032} WARNING - Dependencies not met for
<TaskInstance: sqlDag.Build_Task_8 2016-10-02 00:00:00 [None]>, dependency
'Trigger Rule' FAILED: Task's trigger rule 'all_success' requires all upstream
tasks to have succeeded, but found 2 non-success(es).
[2016-10-03 09:52:57,471] {models.py:1032} WARNING - Dependencies not met for
<TaskInstance: sqlDag.Build_Task_7 2016-10-02 00:00:00 [None]>, dependency
'Trigger Rule' FAILED: Task's trigger rule 'all_success' requires all upstream
tasks to have succeeded, but found 1 non-success(es).
[2016-10-03 09:52:57,493] {models.py:1032} WARNING - Dependencies not met for
<TaskInstance: sqlDag.Build_Task_9 2016-10-02 00:00:00 [None]>, dependency
'Trigger Rule' FAILED: Task's trigger rule 'all_success' requires all upstream
tasks to have succeeded, but found 1 non-success(es).
[2016-10-03 09:52:57,499] {models.py:1032} WARNING - Dependencies not met for
<TaskInstance: sqlDag.Build_Task_4 2016-10-02 00:00:00 [running]>, dependency
'Task Instance Not Already Running' FAILED: Task is already running, it started
on 2016-10-03 09:52:46.849966.
[2016-10-03 09:53:02,327] {celery_executor.py:78} INFO - [celery] queuing
(u'sqlDag', u'Build_Task_2', datetime.datetime(2016, 10, 2, 0, 0)) through
celery, queue=sql_queue_zzz
[2016-10-03 09:53:02,341] {jobs.py:1854} INFO - [backfill progress] | dag run 1
of 1 | tasks waiting: 8 | succeeded: 2 | kicked_off: 4 | failed: 0 | skipped: 0
| deadlocked: 0
[2016-10-03 09:53:02,359] {models.py:1032} WARNING - Dependencies not met for
<TaskInstance: sqlDag.Build_Task_5 2016-10-02 00:00:00 [None]>, dependency
'Trigger Rule' FAILED: Task's trigger rule 'all_success' requires all upstream
tasks to have succeeded, but found 1 non-success(es).
[2016-10-03 09:53:02,416] {models.py:1035} INFO - Dependencies all met for
<TaskInstance: sqlDag.Build_Task_2 2016-10-02 00:00:00 [scheduled]>
[2016-10-03 09:53:02,432] {models.py:1032} WARNING - Dependencies not met for
<TaskInstance: sqlDag.Build_Task_6 2016-10-02 00:00:00 [None]>, dependency
'Trigger Rule' FAILED: Task's trigger rule 'all_success' requires all upstream
tasks to have succeeded, but found 1 non-success(es).
[2016-10-03 09:53:02,451] {models.py:1032} WARNING - Dependencies not met for
<TaskInstance: sqlDag.Build_Task_3 2016-10-02 00:00:00 [None]>, dependency
'Trigger Rule' FAILED: Task's trigger rule 'all_success' requires all upstream
tasks to have succeeded, but found 1 non-success(es).
[2016-10-03 09:53:02,479] {models.py:1032} WARNING - Dependencies not met for
<TaskInstance: sqlDag.Build_Task_8 2016-10-02 00:00:00 [None]>, dependency
'Trigger Rule' FAILED: Task's trigger rule 'all_success' requires all upstream
tasks to have succeeded, but found 2 non-success(es).
[2016-10-03 09:53:02,527] {models.py:1032} WARNING - Dependencies not met for
<TaskInstance: sqlDag.Build_Task_7 2016-10-02 00:00:00 [None]>, dependency
'Trigger Rule' FAILED: Task's trigger rule 'all_success' requires all upstream
tasks to have succeeded, but found 1 non-success(es).
[2016-10-03 09:53:02,564] {models.py:1032} WARNING - Dependencies not met for
<TaskInstance: sqlDag.Build_Task_9 2016-10-02 00:00:00 [None]>, dependency
'Trigger Rule' FAILED: Task's trigger rule 'all_success' requires all upstream
tasks to have succeeded, but found 1 non-success(es).
[2016-10-03 09:53:03,014] {__init__.py:50} INFO - Using executor CeleryExecutor
[2016-10-03 09:53:03,118] {driver.py:120} INFO - Generating grammar tables from
/usr/lib/python2.7/lib2to3/Grammar.txt
[2016-10-03 09:53:03,152] {driver.py:120} INFO - Generating grammar tables from
/usr/lib/python2.7/lib2to3/PatternGrammar.txt
Logging into:
/usr/local/lib/airflow/logs/sqlDag/Build_Task_2/2016-10-02T00:00:00
[2016-10-03 09:53:03,974] {__init__.py:50} INFO - Using executor CeleryExecutor
[2016-10-03 09:53:04,040] {driver.py:120} INFO - Generating grammar tables from
/usr/lib/python2.7/lib2to3/Grammar.txt
[2016-10-03 09:53:04,094] {driver.py:120} INFO - Generating grammar tables from
/usr/lib/python2.7/lib2to3/PatternGrammar.txt
[2016-10-03 09:53:07,338] {jobs.py:1854} INFO - [backfill progress] | dag run 1
of 1 | tasks waiting: 7 | succeeded: 3 | kicked_off: 4 | failed: 0 | skipped: 0
| deadlocked: 0
[2016-10-03 09:53:07,346] {models.py:1032} WARNING - Dependencies not met for
<TaskInstance: sqlDag.Build_Task_5 2016-10-02 00:00:00 [None]>, dependency
'Trigger Rule' FAILED: Task's trigger rule 'all_success' requires all upstream
tasks to have succeeded, but found 1 non-success(es).
[2016-10-03 09:53:07,352] {models.py:1032} WARNING - Dependencies not met for
<TaskInstance: sqlDag.Build_Task_2 2016-10-02 00:00:00 [running]>, dependency
'Task Instance Not Already Running' FAILED: Task is already running, it started
on 2016-10-03 09:53:04.436914.
[2016-10-03 09:53:07,391] {models.py:1032} WARNING - Dependencies not met for
<TaskInstance: sqlDag.Build_Task_6 2016-10-02 00:00:00 [None]>, dependency
'Trigger Rule' FAILED: Task's trigger rule 'all_success' requires all upstream
tasks to have succeeded, but found 1 non-success(es).
[2016-10-03 09:53:07,436] {models.py:1032} WARNING - Dependencies not met for
<TaskInstance: sqlDag.Build_Task_3 2016-10-02 00:00:00 [None]>, dependency
'Trigger Rule' FAILED: Task's trigger rule 'all_success' requires all upstream
tasks to have succeeded, but found 1 non-success(es).
[2016-10-03 09:53:07,444] {models.py:1032} WARNING - Dependencies not met for
<TaskInstance: sqlDag.Build_Task_8 2016-10-02 00:00:00 [None]>, dependency
'Trigger Rule' FAILED: Task's trigger rule 'all_success' requires all upstream
tasks to have succeeded, but found 2 non-success(es).
[2016-10-03 09:53:07,458] {models.py:1032} WARNING - Dependencies not met for
<TaskInstance: sqlDag.Build_Task_7 2016-10-02 00:00:00 [None]>, dependency
'Trigger Rule' FAILED: Task's trigger rule 'all_success' requires all upstream
tasks to have succeeded, but found 1 non-success(es).
[2016-10-03 09:53:07,488] {models.py:1032} WARNING - Dependencies not met for
<TaskInstance: sqlDag.Build_Task_9 2016-10-02 00:00:00 [None]>, dependency
'Trigger Rule' FAILED: Task's trigger rule 'all_success' requires all upstream
tasks to have succeeded, but found 1 non-success(es).
[2016-10-03 09:53:12,340] {jobs.py:1854} INFO - [backfill progress] | dag run 1
of 1 | tasks waiting: 7 | succeeded: 3 | kicked_off: 4 | failed: 0 | skipped: 0
| deadlocked: 0
[2016-10-03 09:53:12,349] {models.py:1032} WARNING - Dependencies not met for
<TaskInstance: sqlDag.Build_Task_5 2016-10-02 00:00:00 [None]>, dependency
'Trigger Rule' FAILED: Task's trigger rule 'all_success' requires all upstream
tasks to have succeeded, but found 1 non-success(es).
[2016-10-03 09:53:12,367] {models.py:1032} WARNING - Dependencies not met for
<TaskInstance: sqlDag.Build_Task_2 2016-10-02 00:00:00 [running]>, dependency
'Task Instance Not Already Running' FAILED: Task is already running, it started
on 2016-10-03 09:53:04.436914.
[2016-10-03 09:53:12,382] {models.py:1032} WARNING - Dependencies not met for
<TaskInstance: sqlDag.Build_Task_6 2016-10-02 00:00:00 [None]>, dependency
'Trigger Rule' FAILED: Task's trigger rule 'all_success' requires all upstream
tasks to have succeeded, but found 1 non-success(es).
[2016-10-03 09:53:12,532] {models.py:1032} WARNING - Dependencies not met for
<TaskInstance: sqlDag.Build_Task_3 2016-10-02 00:00:00 [None]>, dependency
'Trigger Rule' FAILED: Task's trigger rule 'all_success' requires all upstream
tasks to have succeeded, but found 1 non-success(es).
[2016-10-03 09:53:12,674] {models.py:1032} WARNING - Dependencies not met for
<TaskInstance: sqlDag.Build_Task_8 2016-10-02 00:00:00 [None]>, dependency
'Trigger Rule' FAILED: Task's trigger rule 'all_success' requires all upstream
tasks to have succeeded, but found 2 non-success(es).
[2016-10-03 09:53:12,895] {models.py:1032} WARNING - Dependencies not met for
<TaskInstance: sqlDag.Build_Task_7 2016-10-02 00:00:00 [None]>, dependency
'Trigger Rule' FAILED: Task's trigger rule 'all_success' requires all upstream
tasks to have succeeded, but found 1 non-success(es).
[2016-10-03 09:53:12,967] {models.py:1032} WARNING - Dependencies not met for
<TaskInstance: sqlDag.Build_Task_9 2016-10-02 00:00:00 [None]>, dependency
'Trigger Rule' FAILED: Task's trigger rule 'all_success' requires all upstream
tasks to have succeeded, but found 1 non-success(es).
[2016-10-03 09:53:17,340] {jobs.py:1854} INFO - [backfill progress] | dag run 1
of 1 | tasks waiting: 7 | succeeded: 3 | kicked_off: 4 | failed: 0 | skipped: 0
| deadlocked: 0
[2016-10-03 09:53:17,348] {models.py:1032} WARNING - Dependencies not met for
<TaskInstance: sqlDag.Build_Task_5 2016-10-02 00:00:00 [None]>, dependency
'Trigger Rule' FAILED: Task's trigger rule 'all_success' requires all upstream
tasks to have succeeded, but found 1 non-success(es).
[2016-10-03 09:53:17,375] {models.py:1035} INFO - Dependencies all met for
<TaskInstance: sqlDag.Build_Task_6 2016-10-02 00:00:00 [None]>
[2016-10-03 09:53:17,403] {base_executor.py:50} INFO - Adding to queue: airflow
run sqlDag Build_Task_6 2016-10-02T00:00:00 --pickle 31 --local
[2016-10-03 09:53:17,459] {models.py:1035} INFO - Dependencies all met for
<TaskInstance: sqlDag.Build_Task_3 2016-10-02 00:00:00 [None]>
[2016-10-03 09:53:17,465] {base_executor.py:50} INFO - Adding to queue: airflow
run sqlDag Build_Task_3 2016-10-02T00:00:00 --pickle 31 --local
[2016-10-03 09:53:17,482] {models.py:1032} WARNING - Dependencies not met for
<TaskInstance: sqlDag.Build_Task_8 2016-10-02 00:00:00 [None]>, dependency
'Trigger Rule' FAILED: Task's trigger rule 'all_success' requires all upstream
tasks to have succeeded, but found 2 non-success(es).
[2016-10-03 09:53:17,508] {models.py:1032} WARNING - Dependencies not met for
<TaskInstance: sqlDag.Build_Task_7 2016-10-02 00:00:00 [None]>, dependency
'Trigger Rule' FAILED: Task's trigger rule 'all_success' requires all upstream
tasks to have succeeded, but found 1 non-success(es).
[2016-10-03 09:53:17,532] {models.py:1032} WARNING - Dependencies not met for
<TaskInstance: sqlDag.Build_Task_9 2016-10-02 00:00:00 [None]>, dependency
'Trigger Rule' FAILED: Task's trigger rule 'all_success' requires all upstream
tasks to have succeeded, but found 1 non-success(es).
[2016-10-03 09:53:18 +0000] [2210] [INFO] Handling signal: ttin
[2016-10-03 09:53:18 +0000] [14540] [INFO] Booting worker with pid: 14540
[2016-10-03 09:53:18,910] [14540] {models.py:168} INFO - Filling up the DagBag
from /usr/local/lib/airflow/dags
[2016-10-03 09:53:19 +0000] [2210] [INFO] Handling signal: ttou
[2016-10-03 09:53:19 +0000] [14441] [INFO] Worker exiting (pid: 14441)
[2016-10-03 09:53:22,344] {celery_executor.py:78} INFO - [celery] queuing
(u'sqlDag', u'Build_Task_3', datetime.datetime(2016, 10, 2, 0, 0)) through
celery, queue=sql_queue_zzz
[2016-10-03 09:53:22,350] {celery_executor.py:78} INFO - [celery] queuing
(u'sqlDag', u'Build_Task_6', datetime.datetime(2016, 10, 2, 0, 0)) through
celery, queue=sql_queue_zzz
[2016-10-03 09:53:22,407] {jobs.py:1854} INFO - [backfill progress] | dag run 1
of 1 | tasks waiting: 6 | succeeded: 4 | kicked_off: 6 | failed: 0 | skipped: 0
| deadlocked: 0
[2016-10-03 09:53:22,427] {models.py:1032} WARNING - Dependencies not met for
<TaskInstance: sqlDag.Build_Task_5 2016-10-02 00:00:00 [None]>, dependency
'Trigger Rule' FAILED: Task's trigger rule 'all_success' requires all upstream
tasks to have succeeded, but found 1 non-success(es).
[2016-10-03 09:53:22,493] {models.py:1035} INFO - Dependencies all met for
<TaskInstance: sqlDag.Build_Task_6 2016-10-02 00:00:00 [scheduled]>
[2016-10-03 09:53:22,533] {models.py:1035} INFO - Dependencies all met for
<TaskInstance: sqlDag.Build_Task_3 2016-10-02 00:00:00 [scheduled]>
[2016-10-03 09:53:22,559] {models.py:1032} WARNING - Dependencies not met for
<TaskInstance: sqlDag.Build_Task_8 2016-10-02 00:00:00 [None]>, dependency
'Trigger Rule' FAILED: Task's trigger rule 'all_success' requires all upstream
tasks to have succeeded, but found 2 non-success(es).
[2016-10-03 09:53:22,590] {models.py:1032} WARNING - Dependencies not met for
<TaskInstance: sqlDag.Build_Task_7 2016-10-02 00:00:00 [None]>, dependency
'Trigger Rule' FAILED: Task's trigger rule 'all_success' requires all upstream
tasks to have succeeded, but found 1 non-success(es).
[2016-10-03 09:53:22,643] {models.py:1032} WARNING - Dependencies not met for
<TaskInstance: sqlDag.Build_Task_9 2016-10-02 00:00:00 [None]>, dependency
'Trigger Rule' FAILED: Task's trigger rule 'all_success' requires all upstream
tasks to have succeeded, but found 1 non-success(es).
[2016-10-03 09:53:23,678] {__init__.py:50} INFO - Using executor CeleryExecutor
[2016-10-03 09:53:23,679] {__init__.py:50} INFO - Using executor CeleryExecutor
[2016-10-03 09:53:23,834] {driver.py:120} INFO - Generating grammar tables from
/usr/lib/python2.7/lib2to3/Grammar.txt
[2016-10-03 09:53:23,839] {driver.py:120} INFO - Generating grammar tables from
/usr/lib/python2.7/lib2to3/Grammar.txt
[2016-10-03 09:53:23,881] {driver.py:120} INFO - Generating grammar tables from
/usr/lib/python2.7/lib2to3/PatternGrammar.txt
[2016-10-03 09:53:23,887] {driver.py:120} INFO - Generating grammar tables from
/usr/lib/python2.7/lib2to3/PatternGrammar.txt
Logging into:
/usr/local/lib/airflow/logs/sqlDag/Build_Task_3/2016-10-02T00:00:00
Logging into:
/usr/local/lib/airflow/logs/sqlDag/Build_Task_6/2016-10-02T00:00:00
[2016-10-03 09:53:25,660] {__init__.py:50} INFO - Using executor CeleryExecutor
[2016-10-03 09:53:25,680] {__init__.py:50} INFO - Using executor CeleryExecutor
[2016-10-03 09:53:25,850] {driver.py:120} INFO - Generating grammar tables from
/usr/lib/python2.7/lib2to3/Grammar.txt
[2016-10-03 09:53:25,901] {driver.py:120} INFO - Generating grammar tables from
/usr/lib/python2.7/lib2to3/Grammar.txt
[2016-10-03 09:53:25,943] {driver.py:120} INFO - Generating grammar tables from
/usr/lib/python2.7/lib2to3/PatternGrammar.txt
[2016-10-03 09:53:25,963] {driver.py:120} INFO - Generating grammar tables from
/usr/lib/python2.7/lib2to3/PatternGrammar.txt
[2016-10-03 09:53:27,348] {jobs.py:1854} INFO - [backfill progress] | dag run 1
of 1 | tasks waiting: 6 | succeeded: 4 | kicked_off: 6 | failed: 0 | skipped: 0
| deadlocked: 0
[2016-10-03 09:53:27,357] {models.py:1032} WARNING - Dependencies not met for
<TaskInstance: sqlDag.Build_Task_5 2016-10-02 00:00:00 [None]>, dependency
'Trigger Rule' FAILED: Task's trigger rule 'all_success' requires all upstream
tasks to have succeeded, but found 1 non-success(es).
[2016-10-03 09:53:27,362] {models.py:1032} WARNING - Dependencies not met for
<TaskInstance: sqlDag.Build_Task_6 2016-10-02 00:00:00 [running]>, dependency
'Task Instance Not Already Running' FAILED: Task is already running, it started
on 2016-10-03 09:53:26.733119.
[2016-10-03 09:53:27,384] {models.py:1032} WARNING - Dependencies not met for
<TaskInstance: sqlDag.Build_Task_3 2016-10-02 00:00:00 [running]>, dependency
'Task Instance Not Already Running' FAILED: Task is already running, it started
on 2016-10-03 09:53:26.664686.
[2016-10-03 09:53:27,402] {models.py:1032} WARNING - Dependencies not met for
<TaskInstance: sqlDag.Build_Task_8 2016-10-02 00:00:00 [None]>, dependency
'Trigger Rule' FAILED: Task's trigger rule 'all_success' requires all upstream
tasks to have succeeded, but found 2 non-success(es).
[2016-10-03 09:53:27,423] {models.py:1032} WARNING - Dependencies not met for
<TaskInstance: sqlDag.Build_Task_7 2016-10-02 00:00:00 [None]>, dependency
'Trigger Rule' FAILED: Task's trigger rule 'all_success' requires all upstream
tasks to have succeeded, but found 1 non-success(es).
[2016-10-03 09:53:27,457] {models.py:1032} WARNING - Dependencies not met for
<TaskInstance: sqlDag.Build_Task_9 2016-10-02 00:00:00 [None]>, dependency
'Trigger Rule' FAILED: Task's trigger rule 'all_success' requires all upstream
tasks to have succeeded, but found 1 non-success(es).
[2016-10-03 09:53:32,357] {jobs.py:1854} INFO - [backfill progress] | dag run 1
of 1 | tasks waiting: 6 | succeeded: 4 | kicked_off: 6 | failed: 0 | skipped: 0
| deadlocked: 0
[2016-10-03 09:53:32,375] {models.py:1032} WARNING - Dependencies not met for
<TaskInstance: sqlDag.Build_Task_5 2016-10-02 00:00:00 [None]>, dependency
'Trigger Rule' FAILED: Task's trigger rule 'all_success' requires all upstream
tasks to have succeeded, but found 1 non-success(es).
[2016-10-03 09:53:32,387] {models.py:1032} WARNING - Dependencies not met for
<TaskInstance: sqlDag.Build_Task_6 2016-10-02 00:00:00 [running]>, dependency
'Task Instance Not Already Running' FAILED: Task is already running, it started
on 2016-10-03 09:53:26.733119.
[2016-10-03 09:53:32,396] {models.py:1032} WARNING - Dependencies not met for
<TaskInstance: sqlDag.Build_Task_3 2016-10-02 00:00:00 [running]>, dependency
'Task Instance Not Already Running' FAILED: Task is already running, it started
on 2016-10-03 09:53:26.664686.
[2016-10-03 09:53:32,420] {models.py:1032} WARNING - Dependencies not met for
<TaskInstance: sqlDag.Build_Task_8 2016-10-02 00:00:00 [None]>, dependency
'Trigger Rule' FAILED: Task's trigger rule 'all_success' requires all upstream
tasks to have succeeded, but found 2 non-success(es).
[2016-10-03 09:53:32,437] {models.py:1032} WARNING - Dependencies not met for
<TaskInstance: sqlDag.Build_Task_7 2016-10-02 00:00:00 [None]>, dependency
'Trigger Rule' FAILED: Task's trigger rule 'all_success' requires all upstream
tasks to have succeeded, but found 1 non-success(es).
[2016-10-03 09:53:32,447] {models.py:1032} WARNING - Dependencies not met for
<TaskInstance: sqlDag.Build_Task_9 2016-10-02 00:00:00 [None]>, dependency
'Trigger Rule' FAILED: Task's trigger rule 'all_success' requires all upstream
tasks to have succeeded, but found 1 non-success(es).
[2016-10-03 09:53:37,364] {jobs.py:1757} INFO - Executor state: success task
<TaskInstance: sqlDag.Build_Task_3 2016-10-02 00:00:00 [success]>
[2016-10-03 09:53:37,365] {jobs.py:1788} INFO - Task instance (u'sqlDag',
u'Build_Task_3', datetime.datetime(2016, 10, 2, 0, 0)) succeeded
[2016-10-03 09:53:37,370] {jobs.py:1757} INFO - Executor state: success task
<TaskInstance: sqlDag.Build_Task_6 2016-10-02 00:00:00 [success]>
[2016-10-03 09:53:37,371] {jobs.py:1788} INFO - Task instance (u'sqlDag',
u'Build_Task_6', datetime.datetime(2016, 10, 2, 0, 0)) succeeded
[2016-10-03 09:53:37,372] {jobs.py:1854} INFO - [backfill progress] | dag run 1
of 1 | tasks waiting: 4 | succeeded: 6 | kicked_off: 6 | failed: 0 | skipped: 0
| deadlocked: 0
[2016-10-03 09:53:37,434] {models.py:1035} INFO - Dependencies all met for
<TaskInstance: sqlDag.Build_Task_5 2016-10-02 00:00:00 [None]>
[2016-10-03 09:53:37,443] {base_executor.py:50} INFO - Adding to queue: airflow
run sqlDag Build_Task_5 2016-10-02T00:00:00 --pickle 31 --local
[2016-10-03 09:53:37,454] {models.py:1032} WARNING - Dependencies not met for
<TaskInstance: sqlDag.Build_Task_8 2016-10-02 00:00:00 [None]>, dependency
'Trigger Rule' FAILED: Task's trigger rule 'all_success' requires all upstream
tasks to have succeeded, but found 2 non-success(es).
[2016-10-03 09:53:37,474] {models.py:1035} INFO - Dependencies all met for
<TaskInstance: sqlDag.Build_Task_7 2016-10-02 00:00:00 [None]>
[2016-10-03 09:53:37,487] {base_executor.py:50} INFO - Adding to queue: airflow
run sqlDag Build_Task_7 2016-10-02T00:00:00 --pickle 31 --local
[2016-10-03 09:53:37,504] {models.py:1032} WARNING - Dependencies not met for
<TaskInstance: sqlDag.Build_Task_9 2016-10-02 00:00:00 [None]>, dependency
'Trigger Rule' FAILED: Task's trigger rule 'all_success' requires all upstream
tasks to have succeeded, but found 1 non-success(es).
[2016-10-03 09:53:42,360] {celery_executor.py:78} INFO - [celery] queuing
(u'sqlDag', u'Build_Task_5', datetime.datetime(2016, 10, 2, 0, 0)) through
celery, queue=sql_queue_zzz
[2016-10-03 09:53:42,370] {celery_executor.py:78} INFO - [celery] queuing
(u'sqlDag', u'Build_Task_7', datetime.datetime(2016, 10, 2, 0, 0)) through
celery, queue=sql_queue_zzz
[2016-10-03 09:53:42,412] {jobs.py:1854} INFO - [backfill progress] | dag run 1
of 1 | tasks waiting: 4 | succeeded: 6 | kicked_off: 8 | failed: 0 | skipped: 0
| deadlocked: 0
[2016-10-03 09:53:42,449] {models.py:1035} INFO - Dependencies all met for
<TaskInstance: sqlDag.Build_Task_5 2016-10-02 00:00:00 [scheduled]>
[2016-10-03 09:53:42,523] {models.py:1032} WARNING - Dependencies not met for
<TaskInstance: sqlDag.Build_Task_8 2016-10-02 00:00:00 [None]>, dependency
'Trigger Rule' FAILED: Task's trigger rule 'all_success' requires all upstream
tasks to have succeeded, but found 2 non-success(es).
[2016-10-03 09:53:42,548] {models.py:1035} INFO - Dependencies all met for
<TaskInstance: sqlDag.Build_Task_7 2016-10-02 00:00:00 [scheduled]>
[2016-10-03 09:53:42,583] {models.py:1032} WARNING - Dependencies not met for
<TaskInstance: sqlDag.Build_Task_9 2016-10-02 00:00:00 [None]>, dependency
'Trigger Rule' FAILED: Task's trigger rule 'all_success' requires all upstream
tasks to have succeeded, but found 1 non-success(es).
[2016-10-03 09:53:43,608] {__init__.py:50} INFO - Using executor CeleryExecutor
[2016-10-03 09:53:43,629] {__init__.py:50} INFO - Using executor CeleryExecutor
[2016-10-03 09:53:43,828] {driver.py:120} INFO - Generating grammar tables from
/usr/lib/python2.7/lib2to3/Grammar.txt
[2016-10-03 09:53:43,869] {driver.py:120} INFO - Generating grammar tables from
/usr/lib/python2.7/lib2to3/Grammar.txt
[2016-10-03 09:53:43,895] {driver.py:120} INFO - Generating grammar tables from
/usr/lib/python2.7/lib2to3/PatternGrammar.txt
[2016-10-03 09:53:43,955] {driver.py:120} INFO - Generating grammar tables from
/usr/lib/python2.7/lib2to3/PatternGrammar.txt
Logging into:
/usr/local/lib/airflow/logs/sqlDag/Build_Task_5/2016-10-02T00:00:00
Logging into:
/usr/local/lib/airflow/logs/sqlDag/Build_Task_7/2016-10-02T00:00:00
[2016-10-03 09:53:45,575] {__init__.py:50} INFO - Using executor CeleryExecutor
[2016-10-03 09:53:45,682] {__init__.py:50} INFO - Using executor CeleryExecutor
[2016-10-03 09:53:45,818] {driver.py:120} INFO - Generating grammar tables from
/usr/lib/python2.7/lib2to3/Grammar.txt
[2016-10-03 09:53:45,896] {driver.py:120} INFO - Generating grammar tables from
/usr/lib/python2.7/lib2to3/PatternGrammar.txt
[2016-10-03 09:53:45,951] {driver.py:120} INFO - Generating grammar tables from
/usr/lib/python2.7/lib2to3/Grammar.txt
[2016-10-03 09:53:46,012] {driver.py:120} INFO - Generating grammar tables from
/usr/lib/python2.7/lib2to3/PatternGrammar.txt
[2016-10-03 09:53:47,365] {jobs.py:1854} INFO - [backfill progress] | dag run 1
of 1 | tasks waiting: 4 | succeeded: 6 | kicked_off: 8 | failed: 0 | skipped: 0
| deadlocked: 0
[2016-10-03 09:53:47,381] {models.py:1032} WARNING - Dependencies not met for
<TaskInstance: sqlDag.Build_Task_5 2016-10-02 00:00:00 [running]>, dependency
'Task Instance Not Already Running' FAILED: Task is already running, it started
on 2016-10-03 09:53:46.596860.
[2016-10-03 09:53:47,421] {models.py:1032} WARNING - Dependencies not met for
<TaskInstance: sqlDag.Build_Task_8 2016-10-02 00:00:00 [None]>, dependency
'Trigger Rule' FAILED: Task's trigger rule 'all_success' requires all upstream
tasks to have succeeded, but found 2 non-success(es).
[2016-10-03 09:53:47,432] {models.py:1032} WARNING - Dependencies not met for
<TaskInstance: sqlDag.Build_Task_7 2016-10-02 00:00:00 [running]>, dependency
'Task Instance Not Already Running' FAILED: Task is already running, it started
on 2016-10-03 09:53:46.700889.
[2016-10-03 09:53:47,454] {models.py:1032} WARNING - Dependencies not met for
<TaskInstance: sqlDag.Build_Task_9 2016-10-02 00:00:00 [None]>, dependency
'Trigger Rule' FAILED: Task's trigger rule 'all_success' requires all upstream
tasks to have succeeded, but found 1 non-success(es).
[2016-10-03 09:53:49 +0000] [2210] [INFO] Handling signal: ttin
[2016-10-03 09:53:49 +0000] [14605] [INFO] Booting worker with pid: 14605
[2016-10-03 09:53:50,284] [14605] {models.py:168} INFO - Filling up the DagBag
from /usr/local/lib/airflow/dags
[2016-10-03 09:53:51 +0000] [2210] [INFO] Handling signal: ttou
[2016-10-03 09:53:51 +0000] [14449] [INFO] Worker exiting (pid: 14449)
[2016-10-03 09:53:52,370] {jobs.py:1854} INFO - [backfill progress] | dag run 1
of 1 | tasks waiting: 4 | succeeded: 6 | kicked_off: 8 | failed: 0 | skipped: 0
| deadlocked: 0
[2016-10-03 09:53:52,410] {models.py:1032} WARNING - Dependencies not met for
<TaskInstance: sqlDag.Build_Task_8 2016-10-02 00:00:00 [None]>, dependency
'Trigger Rule' FAILED: Task's trigger rule 'all_success' requires all upstream
tasks to have succeeded, but found 1 non-success(es).
[2016-10-03 09:53:52,425] {models.py:1032} WARNING - Dependencies not met for
<TaskInstance: sqlDag.Build_Task_7 2016-10-02 00:00:00 [running]>, dependency
'Task Instance Not Already Running' FAILED: Task is already running, it started
on 2016-10-03 09:53:46.700889.
[2016-10-03 09:53:52,468] {models.py:1032} WARNING - Dependencies not met for
<TaskInstance: sqlDag.Build_Task_9 2016-10-02 00:00:00 [None]>, dependency
'Trigger Rule' FAILED: Task's trigger rule 'all_success' requires all upstream
tasks to have succeeded, but found 1 non-success(es).
[2016-10-03 09:53:57,376] {jobs.py:1757} INFO - Executor state: success task
<TaskInstance: sqlDag.Build_Task_7 2016-10-02 00:00:00 [success]>
[2016-10-03 09:53:57,376] {jobs.py:1788} INFO - Task instance (u'sqlDag',
u'Build_Task_7', datetime.datetime(2016, 10, 2, 0, 0)) succeeded
[2016-10-03 09:53:57,377] {jobs.py:1854} INFO - [backfill progress] | dag run 1
of 1 | tasks waiting: 2 | succeeded: 8 | kicked_off: 8 | failed: 0 | skipped: 0
| deadlocked: 0
[2016-10-03 09:53:57,387] {models.py:1035} INFO - Dependencies all met for
<TaskInstance: sqlDag.Build_Task_8 2016-10-02 00:00:00 [None]>
[2016-10-03 09:53:57,391] {base_executor.py:50} INFO - Adding to queue: airflow
run sqlDag Build_Task_8 2016-10-02T00:00:00 --pickle 31 --local
[2016-10-03 09:53:57,407] {models.py:1032} WARNING - Dependencies not met for
<TaskInstance: sqlDag.Build_Task_9 2016-10-02 00:00:00 [None]>, dependency
'Trigger Rule' FAILED: Task's trigger rule 'all_success' requires all upstream
tasks to have succeeded, but found 1 non-success(es).
[2016-10-03 09:54:02,377] {celery_executor.py:78} INFO - [celery] queuing
(u'sqlDag', u'Build_Task_8', datetime.datetime(2016, 10, 2, 0, 0)) through
celery, queue=sql_queue_zzz
[2016-10-03 09:54:02,392] {jobs.py:1854} INFO - [backfill progress] | dag run 1
of 1 | tasks waiting: 2 | succeeded: 8 | kicked_off: 9 | failed: 0 | skipped: 0
| deadlocked: 0
[2016-10-03 09:54:02,432] {models.py:1035} INFO - Dependencies all met for
<TaskInstance: sqlDag.Build_Task_8 2016-10-02 00:00:00 [scheduled]>
[2016-10-03 09:54:02,467] {models.py:1032} WARNING - Dependencies not met for
<TaskInstance: sqlDag.Build_Task_9 2016-10-02 00:00:00 [None]>, dependency
'Trigger Rule' FAILED: Task's trigger rule 'all_success' requires all upstream
tasks to have succeeded, but found 1 non-success(es).
[2016-10-03 09:54:02,982] {__init__.py:50} INFO - Using executor CeleryExecutor
[2016-10-03 09:54:03,072] {driver.py:120} INFO - Generating grammar tables from
/usr/lib/python2.7/lib2to3/Grammar.txt
[2016-10-03 09:54:03,117] {driver.py:120} INFO - Generating grammar tables from
/usr/lib/python2.7/lib2to3/PatternGrammar.txt
Logging into:
/usr/local/lib/airflow/logs/sqlDag/Build_Task_8/2016-10-02T00:00:00
[2016-10-03 09:54:03,988] {__init__.py:50} INFO - Using executor CeleryExecutor
[2016-10-03 09:54:04,070] {driver.py:120} INFO - Generating grammar tables from
/usr/lib/python2.7/lib2to3/Grammar.txt
[2016-10-03 09:54:04,105] {driver.py:120} INFO - Generating grammar tables from
/usr/lib/python2.7/lib2to3/PatternGrammar.txt
[2016-10-03 09:54:07,384] {jobs.py:1854} INFO - [backfill progress] | dag run 1
of 1 | tasks waiting: 2 | succeeded: 8 | kicked_off: 9 | failed: 0 | skipped: 0
| deadlocked: 0
[2016-10-03 09:54:07,392] {models.py:1032} WARNING - Dependencies not met for
<TaskInstance: sqlDag.Build_Task_8 2016-10-02 00:00:00 [running]>, dependency
'Task Instance Not Already Running' FAILED: Task is already running, it started
on 2016-10-03 09:54:04.458567.
[2016-10-03 09:54:07,431] {models.py:1032} WARNING - Dependencies not met for
<TaskInstance: sqlDag.Build_Task_9 2016-10-02 00:00:00 [None]>, dependency
'Trigger Rule' FAILED: Task's trigger rule 'all_success' requires all upstream
tasks to have succeeded, but found 1 non-success(es).
[2016-10-03 09:54:12,387] {jobs.py:1854} INFO - [backfill progress] | dag run 1
of 1 | tasks waiting: 2 | succeeded: 8 | kicked_off: 9 | failed: 0 | skipped: 0
| deadlocked: 0
[2016-10-03 09:54:12,400] {models.py:1035} INFO - Dependencies all met for
<TaskInstance: sqlDag.Build_Task_9 2016-10-02 00:00:00 [None]>
[2016-10-03 09:54:12,412] {base_executor.py:50} INFO - Adding to queue: airflow
run sqlDag Build_Task_9 2016-10-02T00:00:00 --pickle 31 --local
[2016-10-03 09:54:17,394] {celery_executor.py:78} INFO - [celery] queuing
(u'sqlDag', u'Build_Task_9', datetime.datetime(2016, 10, 2, 0, 0)) through
celery, queue=sql_queue_zzz
[2016-10-03 09:54:17,417] {jobs.py:1854} INFO - [backfill progress] | dag run 1
of 1 | tasks waiting: 1 | succeeded: 9 | kicked_off: 10 | failed: 0 | skipped:
0 | deadlocked: 0
[2016-10-03 09:54:17,458] {models.py:1035} INFO - Dependencies all met for
<TaskInstance: sqlDag.Build_Task_9 2016-10-02 00:00:00 [scheduled]>
[2016-10-03 09:54:18,039] {__init__.py:50} INFO - Using executor CeleryExecutor
[2016-10-03 09:54:18,179] {driver.py:120} INFO - Generating grammar tables from
/usr/lib/python2.7/lib2to3/Grammar.txt
[2016-10-03 09:54:18,213] {driver.py:120} INFO - Generating grammar tables from
/usr/lib/python2.7/lib2to3/PatternGrammar.txt
Logging into:
/usr/local/lib/airflow/logs/sqlDag/Build_Task_9/2016-10-02T00:00:00
[2016-10-03 09:54:19,245] {__init__.py:50} INFO - Using executor CeleryExecutor
[2016-10-03 09:54:19,446] {driver.py:120} INFO - Generating grammar tables from
/usr/lib/python2.7/lib2to3/Grammar.txt
[2016-10-03 09:54:19,538] {driver.py:120} INFO - Generating grammar tables from
/usr/lib/python2.7/lib2to3/PatternGrammar.txt
[2016-10-03 09:54:21 +0000] [2210] [INFO] Handling signal: ttin
[2016-10-03 09:54:21 +0000] [14642] [INFO] Booting worker with pid: 14642
[2016-10-03 09:54:21,551] [14642] {models.py:168} INFO - Filling up the DagBag
from /usr/local/lib/airflow/dags
[2016-10-03 09:54:22,400] {jobs.py:1854} INFO - [backfill progress] | dag run 1
of 1 | tasks waiting: 1 | succeeded: 9 | kicked_off: 10 | failed: 0 | skipped:
0 | deadlocked: 0
[2016-10-03 09:54:22,403] {models.py:1032} WARNING - Dependencies not met for
<TaskInstance: sqlDag.Build_Task_9 2016-10-02 00:00:00 [running]>, dependency
'Task Instance Not Already Running' FAILED: Task is already running, it started
on 2016-10-03 09:54:19.824814.
[2016-10-03 09:54:22 +0000] [2210] [INFO] Handling signal: ttou
[2016-10-03 09:54:22 +0000] [14459] [INFO] Worker exiting (pid: 14459)
[2016-10-03 09:54:27,405] {jobs.py:1854} INFO - [backfill progress] | dag run 1
of 1 | tasks waiting: 1 | succeeded: 9 | kicked_off: 10 | failed: 0 | skipped:
0 | deadlocked: 0
[2016-10-03 09:54:27,408] {models.py:1032} WARNING - Dependencies not met for
<TaskInstance: sqlDag.Build_Task_9 2016-10-02 00:00:00 [running]>, dependency
'Task Instance Not Already Running' FAILED: Task is already running, it started
on 2016-10-03 09:54:19.824814.
[2016-10-03 09:54:32,409] {jobs.py:1854} INFO - [backfill progress] | dag run 1
of 1 | tasks waiting: 1 | succeeded: 9 | kicked_off: 10 | failed: 0 | skipped:
0 | deadlocked: 0
[2016-10-03 09:54:37,413] {jobs.py:1854} INFO - [backfill progress] | dag run 1
of 1 | tasks waiting: 0 | succeeded: 10 | kicked_off: 10 | failed: 0 | skipped:
0 | deadlocked: 0
[2016-10-03 09:54:37,416] {models.py:3730} INFO - Updating state for <DagRun
sqlDag @ 2016-10-02 00:00:00: backfill_2016-10-02T00:00:00, externally
triggered: False> considering 10 task(s)
[2016-10-03 09:54:37,420] {models.py:3771} INFO - Marking run <DagRun sqlDag @
2016-10-02 00:00:00: backfill_2016-10-02T00:00:00, externally triggered: False>
successful
[2016-10-03 09:54:37,441] {jobs.py:1896} INFO - Backfill done. Exiting.
