[incubator-airflow] Git Push Summary

Repository: incubator-airflow Updated Tags: refs/tags/1.8.1 [created] 2b811c445

[incubator-airflow] Git Push Summary

Repository: incubator-airflow Updated Tags: refs/tags/1.8.1 2b811c445 -> 0d8509e7e

incubator-airflow git commit: [AIRFLOW-1181] Add delete and list functionality to gcs_hook

Repository: incubator-airflow Updated Branches: refs/heads/master ac9ccb151 -> 24f73c032 [AIRFLOW-1181] Add delete and list functionality to gcs_hook Closes #2281 from mattuuh7/gcs-delete-list Project: http://git-wip-us.apache.org/repos/asf/incubator-airflow/repo Commit:

[jira] [Created] (AIRFLOW-1184) Contrib Spark Submit Hook does not split argument and argument value

Vianney FOUCAULT created AIRFLOW-1184: - Summary: Contrib Spark Submit Hook does not split argument and argument value Key: AIRFLOW-1184 URL: https://issues.apache.org/jira/browse/AIRFLOW-1184

[jira] [Closed] (AIRFLOW-1183) How to pass Spark URL for standalone cluster?

[ https://issues.apache.org/jira/browse/AIRFLOW-1183?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] sam sen closed AIRFLOW-1183. Resolution: Fixed > How to pass Spark URL for standalone cluster? >

[jira] [Commented] (AIRFLOW-1138) Add licenses to files in scripts directory

[ https://issues.apache.org/jira/browse/AIRFLOW-1138?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16003135#comment-16003135 ] ASF subversion and git services commented on AIRFLOW-1138: -- Commit

[jira] [Commented] (AIRFLOW-1142) SubDAG Tasks Not Executed Even Though All Dependencies Met

[ https://issues.apache.org/jira/browse/AIRFLOW-1142?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16003137#comment-16003137 ] ASF subversion and git services commented on AIRFLOW-1142: -- Commit

[01/36] incubator-airflow git commit: [AIRFLOW-974] Fix mkdirs race condition

Repository: incubator-airflow Updated Branches: refs/heads/v1-8-stable 8e7a55836 -> 0d8509e7e [AIRFLOW-974] Fix mkdirs race condition mkdirs congtained a race condition for when if the directory is created between the os.path.exists and the os.makedirs calls, the os.makedirs will fail with an

[25/36] incubator-airflow git commit: [AIRFLOW-1124] Do not set all tasks to scheduled in backfill

[AIRFLOW-1124] Do not set all tasks to scheduled in backfill Backfill is supposed to fill in the blanks and not to reschedule all tasks. This fixes a regression from 1.8.0. Closes #2247 from bolkedebruin/AIRFLOW-1124 (cherry picked from commit 0406462dc91427793ba40d0f05f321e85dbc6f19)

[36/36] incubator-airflow git commit: Merge branch 'v1-8-stable' of https://git-wip-us.apache.org/repos/asf/incubator-airflow into v1-8-stable

Merge branch 'v1-8-stable' of https://git-wip-us.apache.org/repos/asf/incubator-airflow into v1-8-stable Project: http://git-wip-us.apache.org/repos/asf/incubator-airflow/repo Commit: http://git-wip-us.apache.org/repos/asf/incubator-airflow/commit/0d8509e7 Tree:

[15/36] incubator-airflow git commit: [AIRFLOW-1050] Do not count up_for_retry as not ready

[AIRFLOW-1050] Do not count up_for_retry as not ready up_for_retry tasks were incorrectly counted towards not_ready therefore marking a dag run deadlocked instead of retrying. Closes #2225 from bolkedebruin/AIRFLOW-1050 (cherry picked from commit 35e43f5067f4741640278b765c0e54e4fd45ffa3)

[17/36] incubator-airflow git commit: [AIRFLOW-1035] Use binary exponential backoff

[AIRFLOW-1035] Use binary exponential backoff Closes #2196 from IvanVergiliev/exponential- backoff (cherry picked from commit 4ec932b551774bb394c5770c4d2660f565a4c592) Signed-off-by: Bolke de Bruin Project: http://git-wip-us.apache.org/repos/asf/incubator-airflow/repo Commit:

[24/36] incubator-airflow git commit: [AIRFLOW-1120] Update version view to include Apache prefix

[AIRFLOW-1120] Update version view to include Apache prefix Closes #2244 from criccomini/AIRFLOW-1120 (cherry picked from commit 6684597d951cb9f2fea24576a3d19534d67c89ea) Project: http://git-wip-us.apache.org/repos/asf/incubator-airflow/repo Commit:

[26/36] incubator-airflow git commit: [AIRFLOW-1121][AIRFLOW-1004] Fix `airflow webserver --pid` to write out pid file

[AIRFLOW-1121][AIRFLOW-1004] Fix `airflow webserver --pid` to write out pid file After AIRFLOW-1004, --pid option is no longer honored and the pid file is not being written out. This PR fixes it. Closes #2249 from sekikn/AIRFLOW-1121 (cherry picked from commit

[11/36] incubator-airflow git commit: [AIRFLOW-111] Include queued tasks in scheduler concurrency check

[AIRFLOW-111] Include queued tasks in scheduler concurrency check The concurrency argument in dags appears to not be obeyed because the scheduler does not check the concurrency properly when checking tasks. The tasks do not run, but this leads to a lot of scheduler churn. Closes #2214 from

[13/36] incubator-airflow git commit: [AIRFLOW-969] Catch bad python_callable argument

[AIRFLOW-969] Catch bad python_callable argument Checks for callable when Operator is created, not when it is run. * added initial PythonOperator unit test, testing run * python_callable must be callable; added unit test Closes #2142 from abloomston/python-callable (cherry picked from commit

[31/36] incubator-airflow git commit: [AIRFLOW-492] Make sure stat updates cannot fail a task

[AIRFLOW-492] Make sure stat updates cannot fail a task Previously a failed commit into the db for the statistics could also fail a task. Secondly, the ui could display out of date statistics. This patch reworks DagStat so that failure to update the statistics does not propagate. Next to that,

[16/36] incubator-airflow git commit: [AIRFLOW-1085] Enhance the SparkSubmitOperator

[AIRFLOW-1085] Enhance the SparkSubmitOperator - Allow the Spark home to be set on per connection basis to obviate the need for the spark-submit to be on the PATH, and allows different versions of Spark to be easily used. - Enable the use of the --driver-memory parameter on the spark-submit

[34/36] incubator-airflow git commit: [AIRFLOW-970] Load latest_runs on homepage async

[AIRFLOW-970] Load latest_runs on homepage async The latest_runs column on the homepage loads synchronously with an n+1 query. Homepage loads will be significantly faster if this happens asynchronously and as a batch. Closes #2144 from saguziel/aguziel-latest-run- async (cherry picked from

[07/36] incubator-airflow git commit: [AIRFLOW-1054] Fix broken import in test_dag

[AIRFLOW-1054] Fix broken import in test_dag Closes #2201 from r39132/fix_broken_import_on_test_dag (cherry picked from commit c64e876bd50eeb6c9e2600ac9d832c05eb5e9640) Project: http://git-wip-us.apache.org/repos/asf/incubator-airflow/repo Commit:

[jira] [Commented] (AIRFLOW-1181) Enable delete and list function for Google Cloud Storage Hook

[ https://issues.apache.org/jira/browse/AIRFLOW-1181?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16003164#comment-16003164 ] ASF subversion and git services commented on AIRFLOW-1181: -- Commit

incubator-airflow git commit: [AIRFLOW-1179] Fix Pandas 0.2x breaking Google BigQuery change

Repository: incubator-airflow Updated Branches: refs/heads/master 4284e6485 -> ac9ccb151 [AIRFLOW-1179] Fix Pandas 0.2x breaking Google BigQuery change Closes #2279 from NielsZeilemaker/AIRFLOW-1179 Project: http://git-wip-us.apache.org/repos/asf/incubator-airflow/repo Commit:

[jira] [Commented] (AIRFLOW-1179) Pandas 0.20 broke Google BigQuery hook

[ https://issues.apache.org/jira/browse/AIRFLOW-1179?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16003006#comment-16003006 ] ASF subversion and git services commented on AIRFLOW-1179: -- Commit

[jira] [Commented] (AIRFLOW-1179) Pandas 0.20 broke Google BigQuery hook

[ https://issues.apache.org/jira/browse/AIRFLOW-1179?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16003005#comment-16003005 ] ASF subversion and git services commented on AIRFLOW-1179: -- Commit

[jira] [Closed] (AIRFLOW-1179) Pandas 0.20 broke Google BigQuery hook

[ https://issues.apache.org/jira/browse/AIRFLOW-1179?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chris Riccomini closed AIRFLOW-1179. Resolution: Fixed Fix Version/s: 1.9.0 > Pandas 0.20 broke Google BigQuery hook >

[jira] [Closed] (AIRFLOW-1181) Enable delete and list function for Google Cloud Storage Hook

[ https://issues.apache.org/jira/browse/AIRFLOW-1181?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chris Riccomini closed AIRFLOW-1181. Resolution: Fixed Fix Version/s: 1.9.0 > Enable delete and list function for

[jira] [Created] (AIRFLOW-1182) Contrib Spark Submit operator should template fields

Vianney FOUCAULT created AIRFLOW-1182: - Summary: Contrib Spark Submit operator should template fields Key: AIRFLOW-1182 URL: https://issues.apache.org/jira/browse/AIRFLOW-1182 Project: Apache

[20/36] incubator-airflow git commit: [AIRFLOW-1004][AIRFLOW-276] Fix `airflow webserver -D` to run in background

[AIRFLOW-1004][AIRFLOW-276] Fix `airflow webserver -D` to run in background AIRFLOW-276 introduced a monitor process for gunicorn to find new files in the dag folder, but it also changed `airflow webserver -D`'s behavior to run in foreground. This PR fixes that by running the monitor as a daemon

[jira] [Commented] (AIRFLOW-1127) Move license notices to LICENSE instead of NOTICE

[ https://issues.apache.org/jira/browse/AIRFLOW-1127?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16003133#comment-16003133 ] ASF subversion and git services commented on AIRFLOW-1127: -- Commit

[14/36] incubator-airflow git commit: [AIRFLOW-1033][AIFRLOW-1033] Fix ti_deps for no schedule dags

[AIRFLOW-1033][AIFRLOW-1033] Fix ti_deps for no schedule dags DAGs that did not have a schedule (None or @once) make the dependency checker raise an exception as the previous schedule will not exist. Also activates all ti_deps tests. Closes #2220 from bolkedebruin/AIRFLOW-1033 (cherry picked

[19/36] incubator-airflow git commit: [AIRFLOW-1001] Fix landing times if there is no following schedule

[AIRFLOW-1001] Fix landing times if there is no following schedule @once does not have a following schedule. This was not checked for and therefore the landing times page could bork. Closes #2213 from bolkedebruin/AIRFLOW-1001 (cherry picked from commit 0371df4f1bd78e220e591d5cb23630d6a062f109)

[33/36] incubator-airflow git commit: [AIRFLOW-XXX] Fix merge issue with test/models.py by adding execution_date

[AIRFLOW-XXX] Fix merge issue with test/models.py by adding execution_date Project: http://git-wip-us.apache.org/repos/asf/incubator-airflow/repo Commit: http://git-wip-us.apache.org/repos/asf/incubator-airflow/commit/d61af623 Tree:

[jira] [Commented] (AIRFLOW-1138) Add licenses to files in scripts directory

[ https://issues.apache.org/jira/browse/AIRFLOW-1138?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16003136#comment-16003136 ] ASF subversion and git services commented on AIRFLOW-1138: -- Commit

[12/36] incubator-airflow git commit: Merge pull request #2195 from bolkedebruin/AIRFLOW-719

Merge pull request #2195 from bolkedebruin/AIRFLOW-719 (cherry picked from commit 4a6bef69d1817a5fc3ddd6ffe14c2578eaa49cf0) Signed-off-by: Bolke de Bruin Project: http://git-wip-us.apache.org/repos/asf/incubator-airflow/repo Commit:

[10/36] incubator-airflow git commit: [AIRFLOW-1030][AIRFLOW-1] Fix hook import for HttpSensor

[AIRFLOW-1030][AIRFLOW-1] Fix hook import for HttpSensor Closes #2180 from pdambrauskas/fix/http_hook_import (cherry picked from commit f2dae7d15623e2534e7c0dab3b5a7e02d4cff81d) Signed-off-by: Bolke de Bruin Project:

[32/36] incubator-airflow git commit: [AIRFLOW-XXX] Set version to 1.8.1

[AIRFLOW-XXX] Set version to 1.8.1 Project: http://git-wip-us.apache.org/repos/asf/incubator-airflow/repo Commit: http://git-wip-us.apache.org/repos/asf/incubator-airflow/commit/0a105eed Tree: http://git-wip-us.apache.org/repos/asf/incubator-airflow/tree/0a105eed Diff:

[09/36] incubator-airflow git commit: [AIRFLOW-1062] Fix DagRun#find to return correct result

[AIRFLOW-1062] Fix DagRun#find to return correct result DagRun#find returns wrong result if external_trigger=False is specified, because adding filter is skipped on that condition. This PR fixes it. Closes #2210 from sekikn/AIRFLOW-1062 (cherry picked from commit

[23/36] incubator-airflow git commit: [AIRFLOW-XXX] Set 1.8.1 version

[AIRFLOW-XXX] Set 1.8.1 version Project: http://git-wip-us.apache.org/repos/asf/incubator-airflow/repo Commit: http://git-wip-us.apache.org/repos/asf/incubator-airflow/commit/58a0ee78 Tree: http://git-wip-us.apache.org/repos/asf/incubator-airflow/tree/58a0ee78 Diff:

[35/36] incubator-airflow git commit: Merge branch 'v1-8-test' into v1-8-stable

Merge branch 'v1-8-test' into v1-8-stable Project: http://git-wip-us.apache.org/repos/asf/incubator-airflow/repo Commit: http://git-wip-us.apache.org/repos/asf/incubator-airflow/commit/2b811c44 Tree: http://git-wip-us.apache.org/repos/asf/incubator-airflow/tree/2b811c44 Diff:

[05/36] incubator-airflow git commit: [AIRFLOW-832] Let debug server run without SSL

[AIRFLOW-832] Let debug server run without SSL Closes #2051 from gsakkis/fix-debug-server (cherry picked from commit b0ae70d3a8e935dc9266b6853683ae5375a7390b) Project: http://git-wip-us.apache.org/repos/asf/incubator-airflow/repo Commit:

[02/36] incubator-airflow git commit: [AIRFLOW-989] Do not mark dag run successful if unfinished tasks

[AIRFLOW-989] Do not mark dag run successful if unfinished tasks Dag runs could be marked successful if all root tasks were successful, even if some tasks did not run yet, ie. in case of clearing. Now we consider unfinished_tasks, before marking successful. Closes #2154 from

[08/36] incubator-airflow git commit: [AIRFLOW-1011] Fix bug in BackfillJob._execute() for SubDAGs

[AIRFLOW-1011] Fix bug in BackfillJob._execute() for SubDAGs BackfillJob._execute() checks that the next run date is less than or equal to the end date before creating a DAG run and task instances. For SubDAGs, the next run date is not relevant, i.e. schedule_interval can be anything other than

[21/36] incubator-airflow git commit: [AIRFLOW-1030][AIRFLOW-1] Fix hook import for HttpSensor

[AIRFLOW-1030][AIRFLOW-1] Fix hook import for HttpSensor Closes #2180 from pdambrauskas/fix/http_hook_import (cherry picked from commit f2dae7d15623e2534e7c0dab3b5a7e02d4cff81d) Project: http://git-wip-us.apache.org/repos/asf/incubator-airflow/repo Commit:

[04/36] incubator-airflow git commit: [AIRFLOW-906] Update Code icon from lightning bolt to file

[AIRFLOW-906] Update Code icon from lightning bolt to file Lightning bolts are not a visual metaphor for code or files. Since Glyphicon doesn't have a code icon (<>, for instance), we should use its file icon. Dear Airflow Maintainers, Please accept this PR that addresses the following issues:

[jira] [Commented] (AIRFLOW-492) Insert into dag_stats table results into failed task while task itself succeeded

[ https://issues.apache.org/jira/browse/AIRFLOW-492?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16003140#comment-16003140 ] ASF subversion and git services commented on AIRFLOW-492: - Commit

[jira] [Commented] (AIRFLOW-1142) SubDAG Tasks Not Executed Even Though All Dependencies Met

[ https://issues.apache.org/jira/browse/AIRFLOW-1142?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16003138#comment-16003138 ] ASF subversion and git services commented on AIRFLOW-1142: -- Commit

[jira] [Commented] (AIRFLOW-492) Insert into dag_stats table results into failed task while task itself succeeded

[ https://issues.apache.org/jira/browse/AIRFLOW-492?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16003139#comment-16003139 ] ASF subversion and git services commented on AIRFLOW-492: - Commit

[jira] [Commented] (AIRFLOW-1127) Move license notices to LICENSE instead of NOTICE

[ https://issues.apache.org/jira/browse/AIRFLOW-1127?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16003134#comment-16003134 ] ASF subversion and git services commented on AIRFLOW-1127: -- Commit

[jira] [Commented] (AIRFLOW-970) Latest runs on homepage should load async and in batch

[ https://issues.apache.org/jira/browse/AIRFLOW-970?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16003141#comment-16003141 ] ASF subversion and git services commented on AIRFLOW-970: - Commit

[22/36] incubator-airflow git commit: [AIRFLOW-1000] Rebrand distribution to Apache Airflow

[AIRFLOW-1000] Rebrand distribution to Apache Airflow Per Apache requirements Airflow should be branded Apache Airflow. It is impossible to provide a forward compatible automatic update path and users will be required to manually upgrade. Closes #2172 from bolkedebruin/AIRFLOW-1000 (cherry

[jira] [Updated] (AIRFLOW-1183) How to pass Spark URL for standalone cluster?

[ https://issues.apache.org/jira/browse/AIRFLOW-1183?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] sam sen updated AIRFLOW-1183: - Description: How can I pass my Spark URL? When I look in the logs I see `--master` is pointed to

[jira] [Updated] (AIRFLOW-1183) How to pass Spark URL for standalone cluster?

[ https://issues.apache.org/jira/browse/AIRFLOW-1183?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] sam sen updated AIRFLOW-1183: - Description: How can I pass my Spark URL? When I look in the logs I see `--master` is pointed to

[jira] [Created] (AIRFLOW-1183) How to pass Spark URL for standalone cluster?

sam sen created AIRFLOW-1183: Summary: How to pass Spark URL for standalone cluster? Key: AIRFLOW-1183 URL: https://issues.apache.org/jira/browse/AIRFLOW-1183 Project: Apache Airflow Issue Type:

[jira] [Updated] (AIRFLOW-1183) How to pass Spark URL for standalone cluster?

[ https://issues.apache.org/jira/browse/AIRFLOW-1183?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] sam sen updated AIRFLOW-1183: - Description: How can I pass my Spark URL? When I look in the logs I see `--master` is pointed to

[jira] [Updated] (AIRFLOW-1183) How to pass Spark URL for standalone cluster?

[ https://issues.apache.org/jira/browse/AIRFLOW-1183?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] sam sen updated AIRFLOW-1183: - Description: How can I pass my Spark URL? When I look in the logs I see `--master` is pointed to

[jira] [Updated] (AIRFLOW-1183) How to pass Spark URL for standalone cluster?

[ https://issues.apache.org/jira/browse/AIRFLOW-1183?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] sam sen updated AIRFLOW-1183: - Description: How can I pass my Spark URL? When I look in the logs I see `--master` is pointed to

[jira] [Updated] (AIRFLOW-1183) How to pass Spark URL for standalone cluster?

[ https://issues.apache.org/jira/browse/AIRFLOW-1183?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] sam sen updated AIRFLOW-1183: - Description: How can I pass my Spark URL? When I look in the logs I see `--master` is pointed to

[jira] [Updated] (AIRFLOW-1183) How to pass Spark URL for standalone cluster?

[ https://issues.apache.org/jira/browse/AIRFLOW-1183?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] sam sen updated AIRFLOW-1183: - Description: How can I pass my Spark URL? When I look in the logs I see `--master` is pointed to

[jira] [Updated] (AIRFLOW-1183) How to pass Spark URL for standalone cluster?

[ https://issues.apache.org/jira/browse/AIRFLOW-1183?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] sam sen updated AIRFLOW-1183: - Description: How can I pass my Spark URL? When I look in the logs I see `--master` is pointed to

[jira] [Updated] (AIRFLOW-1183) How to pass Spark URL for standalone cluster?

[ https://issues.apache.org/jira/browse/AIRFLOW-1183?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] sam sen updated AIRFLOW-1183: - Description: How can I pass my Spark URL? When I look in the logs I see `--master` is pointed to

[jira] [Updated] (AIRFLOW-1183) How to pass Spark URL for standalone cluster?

[ https://issues.apache.org/jira/browse/AIRFLOW-1183?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] sam sen updated AIRFLOW-1183: - Description: How can I pass my Spark URL? When I look in the logs I see `--master` is pointed to

[jira] [Updated] (AIRFLOW-1183) How to pass Spark URL for standalone cluster?

[ https://issues.apache.org/jira/browse/AIRFLOW-1183?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] sam sen updated AIRFLOW-1183: - Description: How can I pass my Spark URL? When I look in the logs I see `--master` is pointed to