[jira] [Commented] (AIRFLOW-2812) Fix error in Updating.md for upgrading to 1.10

2018-07-28 Thread ASF subversion and git services (JIRA)


[ 
https://issues.apache.org/jira/browse/AIRFLOW-2812?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16560711#comment-16560711
 ] 

ASF subversion and git services commented on AIRFLOW-2812:
--

Commit 3b8c4ff7a02bb7914ffab8edc810a69e77cca0e9 in incubator-airflow's branch 
refs/heads/v1-10-stable from [~nrhvyc]
[ https://git-wip-us.apache.org/repos/asf?p=incubator-airflow.git;h=3b8c4ff ]

[AIRFLOW-2812] Fix error in Updating.md for upgrading to 1.10

Closes #3654 from nrhvyc/AIRFLOW-2812


> Fix error in Updating.md for upgrading to 1.10
> --
>
> Key: AIRFLOW-2812
> URL: https://issues.apache.org/jira/browse/AIRFLOW-2812
> Project: Apache Airflow
>  Issue Type: Bug
>Affects Versions: 1.10
>Reporter: Nick Hughes
>Assignee: Nick Hughes
>Priority: Minor
>
> Error in Updating.md directions under Logging Configuration having too many 
> brackets which results in errors:
>  * {{log_filename_template}}
>  * {{log_processor_filename_template}}{{}}



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Commented] (AIRFLOW-2812) Fix error in Updating.md for upgrading to 1.10

2018-07-28 Thread ASF subversion and git services (JIRA)


[ 
https://issues.apache.org/jira/browse/AIRFLOW-2812?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16560712#comment-16560712
 ] 

ASF subversion and git services commented on AIRFLOW-2812:
--

Commit 3b8c4ff7a02bb7914ffab8edc810a69e77cca0e9 in incubator-airflow's branch 
refs/heads/v1-10-stable from [~nrhvyc]
[ https://git-wip-us.apache.org/repos/asf?p=incubator-airflow.git;h=3b8c4ff ]

[AIRFLOW-2812] Fix error in Updating.md for upgrading to 1.10

Closes #3654 from nrhvyc/AIRFLOW-2812


> Fix error in Updating.md for upgrading to 1.10
> --
>
> Key: AIRFLOW-2812
> URL: https://issues.apache.org/jira/browse/AIRFLOW-2812
> Project: Apache Airflow
>  Issue Type: Bug
>Affects Versions: 1.10
>Reporter: Nick Hughes
>Assignee: Nick Hughes
>Priority: Minor
>
> Error in Updating.md directions under Logging Configuration having too many 
> brackets which results in errors:
>  * {{log_filename_template}}
>  * {{log_processor_filename_template}}{{}}



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


incubator-airflow git commit: [AIRFLOW-2812] Fix error in Updating.md for upgrading to 1.10

2018-07-28 Thread bolke
Repository: incubator-airflow
Updated Branches:
  refs/heads/v1-10-stable 64dacb1ca -> 3b8c4ff7a


[AIRFLOW-2812] Fix error in Updating.md for upgrading to 1.10

Closes #3654 from nrhvyc/AIRFLOW-2812


Project: http://git-wip-us.apache.org/repos/asf/incubator-airflow/repo
Commit: http://git-wip-us.apache.org/repos/asf/incubator-airflow/commit/3b8c4ff7
Tree: http://git-wip-us.apache.org/repos/asf/incubator-airflow/tree/3b8c4ff7
Diff: http://git-wip-us.apache.org/repos/asf/incubator-airflow/diff/3b8c4ff7

Branch: refs/heads/v1-10-stable
Commit: 3b8c4ff7a02bb7914ffab8edc810a69e77cca0e9
Parents: 64dacb1
Author: Nick Hughes 
Authored: Sat Jul 28 12:55:47 2018 +0200
Committer: Bolke de Bruin 
Committed: Sat Jul 28 12:55:47 2018 +0200

--
 UPDATING.md | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/incubator-airflow/blob/3b8c4ff7/UPDATING.md
--
diff --git a/UPDATING.md b/UPDATING.md
index c341b38..80e5858 100644
--- a/UPDATING.md
+++ b/UPDATING.md
@@ -86,8 +86,8 @@ With Airflow 1.9 or lower, `FILENAME_TEMPLATE`, 
`PROCESSOR_FILENAME_TEMPLATE`, `
 ```
 [core]
 fab_logging_level = WARN
-log_filename_template =  ti.dag_id / ti.task_id / ts 
/ try_number .log
-log_processor_filename_template =  filename .log
+log_filename_template = {{ ti.dag_id }}/{{ ti.task_id }}/{{ ts }}/{{ 
try_number }}.log
+log_processor_filename_template = {{ filename }}.log
 
 [elasticsearch]
 elasticsearch_log_id_template = 
{{dag_id}}-{{task_id}}-{{execution_date}}-{{try_number}}



[jira] [Closed] (AIRFLOW-2812) Fix error in Updating.md for upgrading to 1.10

2018-07-28 Thread Bolke de Bruin (JIRA)


 [ 
https://issues.apache.org/jira/browse/AIRFLOW-2812?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Bolke de Bruin closed AIRFLOW-2812.
---
   Resolution: Fixed
Fix Version/s: 1.10.0

> Fix error in Updating.md for upgrading to 1.10
> --
>
> Key: AIRFLOW-2812
> URL: https://issues.apache.org/jira/browse/AIRFLOW-2812
> Project: Apache Airflow
>  Issue Type: Bug
>Affects Versions: 1.10
>Reporter: Nick Hughes
>Assignee: Nick Hughes
>Priority: Minor
> Fix For: 1.10.0
>
>
> Error in Updating.md directions under Logging Configuration having too many 
> brackets which results in errors:
>  * {{log_filename_template}}
>  * {{log_processor_filename_template}}{{}}



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


incubator-airflow git commit: [AIRFLOW-2812] Fix error in Updating.md for upgrading to 1.10

2018-07-28 Thread bolke
Repository: incubator-airflow
Updated Branches:
  refs/heads/v1-10-test fc00875ed -> b5ba06dd5


[AIRFLOW-2812] Fix error in Updating.md for upgrading to 1.10

Closes #3654 from nrhvyc/AIRFLOW-2812

(cherry picked from commit 3b8c4ff7a02bb7914ffab8edc810a69e77cca0e9)
Signed-off-by: Bolke de Bruin 


Project: http://git-wip-us.apache.org/repos/asf/incubator-airflow/repo
Commit: http://git-wip-us.apache.org/repos/asf/incubator-airflow/commit/b5ba06dd
Tree: http://git-wip-us.apache.org/repos/asf/incubator-airflow/tree/b5ba06dd
Diff: http://git-wip-us.apache.org/repos/asf/incubator-airflow/diff/b5ba06dd

Branch: refs/heads/v1-10-test
Commit: b5ba06dd57433d0b116e1b64f95093e078a41495
Parents: fc00875
Author: Nick Hughes 
Authored: Sat Jul 28 12:55:47 2018 +0200
Committer: Bolke de Bruin 
Committed: Sat Jul 28 12:56:09 2018 +0200

--
 UPDATING.md | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/incubator-airflow/blob/b5ba06dd/UPDATING.md
--
diff --git a/UPDATING.md b/UPDATING.md
index c341b38..80e5858 100644
--- a/UPDATING.md
+++ b/UPDATING.md
@@ -86,8 +86,8 @@ With Airflow 1.9 or lower, `FILENAME_TEMPLATE`, 
`PROCESSOR_FILENAME_TEMPLATE`, `
 ```
 [core]
 fab_logging_level = WARN
-log_filename_template =  ti.dag_id / ti.task_id / ts 
/ try_number .log
-log_processor_filename_template =  filename .log
+log_filename_template = {{ ti.dag_id }}/{{ ti.task_id }}/{{ ts }}/{{ 
try_number }}.log
+log_processor_filename_template = {{ filename }}.log
 
 [elasticsearch]
 elasticsearch_log_id_template = 
{{dag_id}}-{{task_id}}-{{execution_date}}-{{try_number}}



[jira] [Commented] (AIRFLOW-2812) Fix error in Updating.md for upgrading to 1.10

2018-07-28 Thread ASF subversion and git services (JIRA)


[ 
https://issues.apache.org/jira/browse/AIRFLOW-2812?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16560714#comment-16560714
 ] 

ASF subversion and git services commented on AIRFLOW-2812:
--

Commit b5ba06dd57433d0b116e1b64f95093e078a41495 in incubator-airflow's branch 
refs/heads/v1-10-test from [~nrhvyc]
[ https://git-wip-us.apache.org/repos/asf?p=incubator-airflow.git;h=b5ba06d ]

[AIRFLOW-2812] Fix error in Updating.md for upgrading to 1.10

Closes #3654 from nrhvyc/AIRFLOW-2812

(cherry picked from commit 3b8c4ff7a02bb7914ffab8edc810a69e77cca0e9)
Signed-off-by: Bolke de Bruin 


> Fix error in Updating.md for upgrading to 1.10
> --
>
> Key: AIRFLOW-2812
> URL: https://issues.apache.org/jira/browse/AIRFLOW-2812
> Project: Apache Airflow
>  Issue Type: Bug
>Affects Versions: 1.10
>Reporter: Nick Hughes
>Assignee: Nick Hughes
>Priority: Minor
>
> Error in Updating.md directions under Logging Configuration having too many 
> brackets which results in errors:
>  * {{log_filename_template}}
>  * {{log_processor_filename_template}}{{}}



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


incubator-airflow git commit: [AIRFLOW-XXX] Include license folder for binary

2018-07-28 Thread bolke
Repository: incubator-airflow
Updated Branches:
  refs/heads/v1-10-test d8869a261 -> ca0eb6f87


[AIRFLOW-XXX] Include license folder for binary

(cherry picked from commit 27fde3840c47fbf5e4d7c429ed4a94de22f9f80c)
Signed-off-by: Bolke de Bruin 


Project: http://git-wip-us.apache.org/repos/asf/incubator-airflow/repo
Commit: http://git-wip-us.apache.org/repos/asf/incubator-airflow/commit/ca0eb6f8
Tree: http://git-wip-us.apache.org/repos/asf/incubator-airflow/tree/ca0eb6f8
Diff: http://git-wip-us.apache.org/repos/asf/incubator-airflow/diff/ca0eb6f8

Branch: refs/heads/v1-10-test
Commit: ca0eb6f87f11c6ce17e3a6d0336f9f123d17a411
Parents: d8869a2
Author: Bolke de Bruin 
Authored: Sat Jul 21 19:41:43 2018 +0200
Committer: Bolke de Bruin 
Committed: Sat Jul 28 13:02:38 2018 +0200

--
 MANIFEST.in | 1 +
 1 file changed, 1 insertion(+)
--


http://git-wip-us.apache.org/repos/asf/incubator-airflow/blob/ca0eb6f8/MANIFEST.in
--
diff --git a/MANIFEST.in b/MANIFEST.in
index c04b806..b069194 100644
--- a/MANIFEST.in
+++ b/MANIFEST.in
@@ -21,6 +21,7 @@ include LICENSE
 include DISCLAIMER
 include CHANGELOG.txt
 include README.md
+graft licenses/
 graft airflow/www/templates
 graft airflow/www/static
 graft airflow/www_rbac/static



incubator-airflow git commit: [AIRFLOW-XXX] Include license folder for binary

2018-07-28 Thread bolke
Repository: incubator-airflow
Updated Branches:
  refs/heads/v1-10-stable 3b7e71f82 -> 90e89f0d8


[AIRFLOW-XXX] Include license folder for binary

(cherry picked from commit 27fde3840c47fbf5e4d7c429ed4a94de22f9f80c)
Signed-off-by: Bolke de Bruin 


Project: http://git-wip-us.apache.org/repos/asf/incubator-airflow/repo
Commit: http://git-wip-us.apache.org/repos/asf/incubator-airflow/commit/90e89f0d
Tree: http://git-wip-us.apache.org/repos/asf/incubator-airflow/tree/90e89f0d
Diff: http://git-wip-us.apache.org/repos/asf/incubator-airflow/diff/90e89f0d

Branch: refs/heads/v1-10-stable
Commit: 90e89f0d89e8448748a31fefea8e4e97db884048
Parents: 3b7e71f
Author: Bolke de Bruin 
Authored: Sat Jul 21 19:41:43 2018 +0200
Committer: Bolke de Bruin 
Committed: Sat Jul 28 13:02:54 2018 +0200

--
 MANIFEST.in | 1 +
 1 file changed, 1 insertion(+)
--


http://git-wip-us.apache.org/repos/asf/incubator-airflow/blob/90e89f0d/MANIFEST.in
--
diff --git a/MANIFEST.in b/MANIFEST.in
index c04b806..b069194 100644
--- a/MANIFEST.in
+++ b/MANIFEST.in
@@ -21,6 +21,7 @@ include LICENSE
 include DISCLAIMER
 include CHANGELOG.txt
 include README.md
+graft licenses/
 graft airflow/www/templates
 graft airflow/www/static
 graft airflow/www_rbac/static



[jira] [Commented] (AIRFLOW-2815) Notice cannot contain "onwards" and needs to be specific

2018-07-28 Thread ASF subversion and git services (JIRA)


[ 
https://issues.apache.org/jira/browse/AIRFLOW-2815?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16560720#comment-16560720
 ] 

ASF subversion and git services commented on AIRFLOW-2815:
--

Commit f0549211f5b026a5b2d5c3bc6b3b74d0bf0a6536 in incubator-airflow's branch 
refs/heads/v1-10-stable from [~bolke]
[ https://git-wip-us.apache.org/repos/asf?p=incubator-airflow.git;h=f054921 ]

[AIRFLOW-2815] Use correct copyright period

"onwards" is not specific enough

(cherry picked from commit 4f562ba8c7f12b8de74e40fe67cac7b4f38413b1)
Signed-off-by: Bolke de Bruin 


> Notice cannot contain "onwards" and needs to be specific
> 
>
> Key: AIRFLOW-2815
> URL: https://issues.apache.org/jira/browse/AIRFLOW-2815
> Project: Apache Airflow
>  Issue Type: Sub-task
>Reporter: Bolke de Bruin
>Priority: Major
>




--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Commented] (AIRFLOW-2815) Notice cannot contain "onwards" and needs to be specific

2018-07-28 Thread ASF subversion and git services (JIRA)


[ 
https://issues.apache.org/jira/browse/AIRFLOW-2815?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16560718#comment-16560718
 ] 

ASF subversion and git services commented on AIRFLOW-2815:
--

Commit 4f562ba8c7f12b8de74e40fe67cac7b4f38413b1 in incubator-airflow's branch 
refs/heads/master from [~bolke]
[ https://git-wip-us.apache.org/repos/asf?p=incubator-airflow.git;h=4f562ba ]

[AIRFLOW-2815] Use correct copyright period

"onwards" is not specific enough


> Notice cannot contain "onwards" and needs to be specific
> 
>
> Key: AIRFLOW-2815
> URL: https://issues.apache.org/jira/browse/AIRFLOW-2815
> Project: Apache Airflow
>  Issue Type: Sub-task
>Reporter: Bolke de Bruin
>Priority: Major
>




--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Commented] (AIRFLOW-2815) Notice cannot contain "onwards" and needs to be specific

2018-07-28 Thread ASF subversion and git services (JIRA)


[ 
https://issues.apache.org/jira/browse/AIRFLOW-2815?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16560719#comment-16560719
 ] 

ASF subversion and git services commented on AIRFLOW-2815:
--

Commit fdca7196d40d89a1a1a61454f4fb338eb24bebc5 in incubator-airflow's branch 
refs/heads/v1-10-test from [~bolke]
[ https://git-wip-us.apache.org/repos/asf?p=incubator-airflow.git;h=fdca719 ]

[AIRFLOW-2815] Use correct copyright period

"onwards" is not specific enough

(cherry picked from commit 4f562ba8c7f12b8de74e40fe67cac7b4f38413b1)
Signed-off-by: Bolke de Bruin 


> Notice cannot contain "onwards" and needs to be specific
> 
>
> Key: AIRFLOW-2815
> URL: https://issues.apache.org/jira/browse/AIRFLOW-2815
> Project: Apache Airflow
>  Issue Type: Sub-task
>Reporter: Bolke de Bruin
>Priority: Major
>




--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


incubator-airflow git commit: [AIRFLOW-2815] Use correct copyright period

2018-07-28 Thread bolke
Repository: incubator-airflow
Updated Branches:
  refs/heads/v1-10-stable 90e89f0d8 -> f0549211f


[AIRFLOW-2815] Use correct copyright period

"onwards" is not specific enough

(cherry picked from commit 4f562ba8c7f12b8de74e40fe67cac7b4f38413b1)
Signed-off-by: Bolke de Bruin 


Project: http://git-wip-us.apache.org/repos/asf/incubator-airflow/repo
Commit: http://git-wip-us.apache.org/repos/asf/incubator-airflow/commit/f0549211
Tree: http://git-wip-us.apache.org/repos/asf/incubator-airflow/tree/f0549211
Diff: http://git-wip-us.apache.org/repos/asf/incubator-airflow/diff/f0549211

Branch: refs/heads/v1-10-stable
Commit: f0549211f5b026a5b2d5c3bc6b3b74d0bf0a6536
Parents: 90e89f0
Author: Bolke de Bruin 
Authored: Sat Jul 28 13:10:43 2018 +0200
Committer: Bolke de Bruin 
Committed: Sat Jul 28 13:14:21 2018 +0200

--
 NOTICE | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
--


http://git-wip-us.apache.org/repos/asf/incubator-airflow/blob/f0549211/NOTICE
--
diff --git a/NOTICE b/NOTICE
index b1e78ad..a472730 100644
--- a/NOTICE
+++ b/NOTICE
@@ -1,5 +1,5 @@
 Apache Airflow
-Copyright 2016 and onwards The Apache Software Foundation
+Copyright 2016-2018 The Apache Software Foundation
 
 
 This product includes software developed at The Apache Software



incubator-airflow git commit: [AIRFLOW-2815] Use correct copyright period

2018-07-28 Thread bolke
Repository: incubator-airflow
Updated Branches:
  refs/heads/v1-10-test ca0eb6f87 -> fdca7196d


[AIRFLOW-2815] Use correct copyright period

"onwards" is not specific enough

(cherry picked from commit 4f562ba8c7f12b8de74e40fe67cac7b4f38413b1)
Signed-off-by: Bolke de Bruin 


Project: http://git-wip-us.apache.org/repos/asf/incubator-airflow/repo
Commit: http://git-wip-us.apache.org/repos/asf/incubator-airflow/commit/fdca7196
Tree: http://git-wip-us.apache.org/repos/asf/incubator-airflow/tree/fdca7196
Diff: http://git-wip-us.apache.org/repos/asf/incubator-airflow/diff/fdca7196

Branch: refs/heads/v1-10-test
Commit: fdca7196d40d89a1a1a61454f4fb338eb24bebc5
Parents: ca0eb6f
Author: Bolke de Bruin 
Authored: Sat Jul 28 13:10:43 2018 +0200
Committer: Bolke de Bruin 
Committed: Sat Jul 28 13:14:00 2018 +0200

--
 NOTICE | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
--


http://git-wip-us.apache.org/repos/asf/incubator-airflow/blob/fdca7196/NOTICE
--
diff --git a/NOTICE b/NOTICE
index b1e78ad..a472730 100644
--- a/NOTICE
+++ b/NOTICE
@@ -1,5 +1,5 @@
 Apache Airflow
-Copyright 2016 and onwards The Apache Software Foundation
+Copyright 2016-2018 The Apache Software Foundation
 
 
 This product includes software developed at The Apache Software



incubator-airflow git commit: [AIRFLOW-2815] Use correct copyright period

2018-07-28 Thread bolke
Repository: incubator-airflow
Updated Branches:
  refs/heads/master 3e7e42f02 -> 4f562ba8c


[AIRFLOW-2815] Use correct copyright period

"onwards" is not specific enough


Project: http://git-wip-us.apache.org/repos/asf/incubator-airflow/repo
Commit: http://git-wip-us.apache.org/repos/asf/incubator-airflow/commit/4f562ba8
Tree: http://git-wip-us.apache.org/repos/asf/incubator-airflow/tree/4f562ba8
Diff: http://git-wip-us.apache.org/repos/asf/incubator-airflow/diff/4f562ba8

Branch: refs/heads/master
Commit: 4f562ba8c7f12b8de74e40fe67cac7b4f38413b1
Parents: 3e7e42f
Author: Bolke de Bruin 
Authored: Sat Jul 28 13:10:43 2018 +0200
Committer: Bolke de Bruin 
Committed: Sat Jul 28 13:10:43 2018 +0200

--
 NOTICE | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
--


http://git-wip-us.apache.org/repos/asf/incubator-airflow/blob/4f562ba8/NOTICE
--
diff --git a/NOTICE b/NOTICE
index b1e78ad..a472730 100644
--- a/NOTICE
+++ b/NOTICE
@@ -1,5 +1,5 @@
 Apache Airflow
-Copyright 2016 and onwards The Apache Software Foundation
+Copyright 2016-2018 The Apache Software Foundation
 
 
 This product includes software developed at The Apache Software



[jira] [Created] (AIRFLOW-2815) Notice cannot contain "onwards" and needs to be specific

2018-07-28 Thread Bolke de Bruin (JIRA)
Bolke de Bruin created AIRFLOW-2815:
---

 Summary: Notice cannot contain "onwards" and needs to be specific
 Key: AIRFLOW-2815
 URL: https://issues.apache.org/jira/browse/AIRFLOW-2815
 Project: Apache Airflow
  Issue Type: Sub-task
Reporter: Bolke de Bruin






--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Created] (AIRFLOW-2816) Remove github copyright clause as it shouldnt be there

2018-07-28 Thread Bolke de Bruin (JIRA)
Bolke de Bruin created AIRFLOW-2816:
---

 Summary: Remove github copyright clause as it shouldnt be there
 Key: AIRFLOW-2816
 URL: https://issues.apache.org/jira/browse/AIRFLOW-2816
 Project: Apache Airflow
  Issue Type: Sub-task
Reporter: Bolke de Bruin


probably copy paste error



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


incubator-airflow git commit: [AIRFLOW-2810] Fix typo in Xcom model timestamp

2018-07-28 Thread bolke
Repository: incubator-airflow
Updated Branches:
  refs/heads/v1-10-stable 3b8c4ff7a -> 3b7e71f82


[AIRFLOW-2810] Fix typo in Xcom model timestamp

Fix typo in Xcom model timestamp field

No new testing - the field is already represented
in migrations

Closes #3652 from andywilco/fix_datetime_typo

(cherry picked from commit a338f3276835af45765d24a6e6d43ad4ba4d66ba)
Signed-off-by: Bolke de Bruin 


Project: http://git-wip-us.apache.org/repos/asf/incubator-airflow/repo
Commit: http://git-wip-us.apache.org/repos/asf/incubator-airflow/commit/3b7e71f8
Tree: http://git-wip-us.apache.org/repos/asf/incubator-airflow/tree/3b7e71f8
Diff: http://git-wip-us.apache.org/repos/asf/incubator-airflow/diff/3b7e71f8

Branch: refs/heads/v1-10-stable
Commit: 3b7e71f820bd312f2013d41eb7b2f9486a532036
Parents: 3b8c4ff
Author: Andy Wilcox 
Authored: Fri Jul 27 13:03:30 2018 -0400
Committer: Bolke de Bruin 
Committed: Sat Jul 28 13:00:57 2018 +0200

--
 airflow/models.py | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
--


http://git-wip-us.apache.org/repos/asf/incubator-airflow/blob/3b7e71f8/airflow/models.py
--
diff --git a/airflow/models.py b/airflow/models.py
index 6b244cd..b789fd1 100755
--- a/airflow/models.py
+++ b/airflow/models.py
@@ -4470,7 +4470,7 @@ class XCom(Base, LoggingMixin):
 key = Column(String(512))
 value = Column(LargeBinary)
 timestamp = Column(
-DateTime, default=timezone.utcnow, nullable=False)
+UtcDateTime, default=timezone.utcnow, nullable=False)
 execution_date = Column(UtcDateTime, nullable=False)
 
 # source information



incubator-airflow git commit: [AIRFLOW-2810] Fix typo in Xcom model timestamp

2018-07-28 Thread bolke
Repository: incubator-airflow
Updated Branches:
  refs/heads/v1-10-test b5ba06dd5 -> d8869a261


[AIRFLOW-2810] Fix typo in Xcom model timestamp

Fix typo in Xcom model timestamp field

No new testing - the field is already represented
in migrations

Closes #3652 from andywilco/fix_datetime_typo

(cherry picked from commit a338f3276835af45765d24a6e6d43ad4ba4d66ba)
Signed-off-by: Bolke de Bruin 


Project: http://git-wip-us.apache.org/repos/asf/incubator-airflow/repo
Commit: http://git-wip-us.apache.org/repos/asf/incubator-airflow/commit/d8869a26
Tree: http://git-wip-us.apache.org/repos/asf/incubator-airflow/tree/d8869a26
Diff: http://git-wip-us.apache.org/repos/asf/incubator-airflow/diff/d8869a26

Branch: refs/heads/v1-10-test
Commit: d8869a2610cc71d72d89e23a989c1050b4d8824e
Parents: b5ba06d
Author: Andy Wilcox 
Authored: Fri Jul 27 13:03:30 2018 -0400
Committer: Bolke de Bruin 
Committed: Sat Jul 28 13:00:35 2018 +0200

--
 airflow/models.py | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
--


http://git-wip-us.apache.org/repos/asf/incubator-airflow/blob/d8869a26/airflow/models.py
--
diff --git a/airflow/models.py b/airflow/models.py
index cb0165a..bf79f94 100755
--- a/airflow/models.py
+++ b/airflow/models.py
@@ -4469,7 +4469,7 @@ class XCom(Base, LoggingMixin):
 key = Column(String(512))
 value = Column(LargeBinary)
 timestamp = Column(
-DateTime, default=timezone.utcnow, nullable=False)
+UtcDateTime, default=timezone.utcnow, nullable=False)
 execution_date = Column(UtcDateTime, nullable=False)
 
 # source information



[jira] [Commented] (AIRFLOW-2810) Typo in Xcom model timestamp field

2018-07-28 Thread ASF subversion and git services (JIRA)


[ 
https://issues.apache.org/jira/browse/AIRFLOW-2810?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16560717#comment-16560717
 ] 

ASF subversion and git services commented on AIRFLOW-2810:
--

Commit 3b7e71f820bd312f2013d41eb7b2f9486a532036 in incubator-airflow's branch 
refs/heads/v1-10-stable from [~andywilco]
[ https://git-wip-us.apache.org/repos/asf?p=incubator-airflow.git;h=3b7e71f ]

[AIRFLOW-2810] Fix typo in Xcom model timestamp

Fix typo in Xcom model timestamp field

No new testing - the field is already represented
in migrations

Closes #3652 from andywilco/fix_datetime_typo

(cherry picked from commit a338f3276835af45765d24a6e6d43ad4ba4d66ba)
Signed-off-by: Bolke de Bruin 


> Typo in Xcom model timestamp field
> --
>
> Key: AIRFLOW-2810
> URL: https://issues.apache.org/jira/browse/AIRFLOW-2810
> Project: Apache Airflow
>  Issue Type: Bug
>  Components: models
>Affects Versions: 1.10
>Reporter: Andy Wilcox
>Assignee: Andy Wilcox
>Priority: Minor
>
>  
> Looks like a find/replace error.  Should be type UtcDateTime, is legacy 
> DateTime



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Commented] (AIRFLOW-2810) Typo in Xcom model timestamp field

2018-07-28 Thread ASF subversion and git services (JIRA)


[ 
https://issues.apache.org/jira/browse/AIRFLOW-2810?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16560716#comment-16560716
 ] 

ASF subversion and git services commented on AIRFLOW-2810:
--

Commit d8869a2610cc71d72d89e23a989c1050b4d8824e in incubator-airflow's branch 
refs/heads/v1-10-test from [~andywilco]
[ https://git-wip-us.apache.org/repos/asf?p=incubator-airflow.git;h=d8869a2 ]

[AIRFLOW-2810] Fix typo in Xcom model timestamp

Fix typo in Xcom model timestamp field

No new testing - the field is already represented
in migrations

Closes #3652 from andywilco/fix_datetime_typo

(cherry picked from commit a338f3276835af45765d24a6e6d43ad4ba4d66ba)
Signed-off-by: Bolke de Bruin 


> Typo in Xcom model timestamp field
> --
>
> Key: AIRFLOW-2810
> URL: https://issues.apache.org/jira/browse/AIRFLOW-2810
> Project: Apache Airflow
>  Issue Type: Bug
>  Components: models
>Affects Versions: 1.10
>Reporter: Andy Wilcox
>Assignee: Andy Wilcox
>Priority: Minor
>
>  
> Looks like a find/replace error.  Should be type UtcDateTime, is legacy 
> DateTime



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Created] (AIRFLOW-2818) Cant' identify DAG .py files with capital case extension ('.PY' rather than '.py')

2018-07-28 Thread Xiaodong DENG (JIRA)
Xiaodong DENG created AIRFLOW-2818:
--

 Summary: Cant' identify DAG .py files with capital case extension 
('.PY' rather than '.py')
 Key: AIRFLOW-2818
 URL: https://issues.apache.org/jira/browse/AIRFLOW-2818
 Project: Apache Airflow
  Issue Type: Bug
  Components: DAG
Reporter: Xiaodong DENG
Assignee: Xiaodong DENG


This bug is quite simply. 

In multiple scripts in which we try to identify and list python files, the 
capital case extension was not considered. For example, if I have a DAG file 
with file name "dag.py", it will be identified. However, if I change the name 
into "dag.PY", it will be missed by Airflow.

This is because simply using function *os.path.splitext()* can not handle case 
issue.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Updated] (AIRFLOW-2818) Cant' identify DAG .py files with capital case extension ('.PY', '.Py', '.pY')

2018-07-28 Thread Xiaodong DENG (JIRA)


 [ 
https://issues.apache.org/jira/browse/AIRFLOW-2818?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Xiaodong DENG updated AIRFLOW-2818:
---
Description: 
This bug is quite simple. 

In multiple scripts in which we try to identify and list python files, the 
capital case extension was not considered. For example, if I have a DAG file 
with file name "dag.py", it will be identified. However, if I change the name 
into "dag.PY", it will be missed by Airflow.

This is because simply using function *os.path.splitext()* can not handle case 
issue.

  was:
This bug is quite simply. 

In multiple scripts in which we try to identify and list python files, the 
capital case extension was not considered. For example, if I have a DAG file 
with file name "dag.py", it will be identified. However, if I change the name 
into "dag.PY", it will be missed by Airflow.

This is because simply using function *os.path.splitext()* can not handle case 
issue.


> Cant' identify DAG .py files with capital case extension ('.PY', '.Py', '.pY')
> --
>
> Key: AIRFLOW-2818
> URL: https://issues.apache.org/jira/browse/AIRFLOW-2818
> Project: Apache Airflow
>  Issue Type: Bug
>  Components: DAG
>Reporter: Xiaodong DENG
>Assignee: Xiaodong DENG
>Priority: Major
>
> This bug is quite simple. 
> In multiple scripts in which we try to identify and list python files, the 
> capital case extension was not considered. For example, if I have a DAG file 
> with file name "dag.py", it will be identified. However, if I change the name 
> into "dag.PY", it will be missed by Airflow.
> This is because simply using function *os.path.splitext()* can not handle 
> case issue.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Updated] (AIRFLOW-2780) Adds IMAP Hook to interact with a mail server

2018-07-28 Thread Felix Uellendall (JIRA)


 [ 
https://issues.apache.org/jira/browse/AIRFLOW-2780?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Felix Uellendall updated AIRFLOW-2780:
--
Summary: Adds IMAP Hook to interact with a mail server  (was: Add IMAP Hook 
to retrieve email attachments from a mail server)

> Adds IMAP Hook to interact with a mail server
> -
>
> Key: AIRFLOW-2780
> URL: https://issues.apache.org/jira/browse/AIRFLOW-2780
> Project: Apache Airflow
>  Issue Type: New Feature
>Reporter: Felix Uellendall
>Assignee: Felix Uellendall
>Priority: Major
>
> This Hook connects to a mail server via IMAP to be able to retrieve email 
> attachments by using [Python's IMAP 
> Library.|https://docs.python.org/3.6/library/imaplib.html]
> Features:
> - `has_mail_attachments`: Can be used in a `Sensor` to check if there is an 
> attachment on the mail server with the given name.
> - `retrieve_mail_attachments`: Can be used in an `Operator` to do sth. with 
> the attachments returned as list of tuple.
> - `download_mail_attachments`: Can be used in an `Operator` to download the 
> attachment.
> Tests:
> - `test_connect_and_disconnect`: this tests calls the ImapHook as 
> ContextManager
> - `test_has_mail_attachments_found`
> - `test_has_mail_attachments_not_found`
> - `test_has_mail_attachments_with_regex_found`
> - `test_has_mail_attachments_with_regex_not_found`
> - `test_retrieve_mail_attachments_found`
> - `test_retrieve_mail_attachments_not_found`
> - `test_retrieve_mail_attachments_with_regex_found`
> - `test_retrieve_mail_attachments_with_regex_not_found`
> - `test_download_mail_attachments_found`
> - `test_download_mail_attachments_not_found`
> - `test_download_mail_attachments_with_regex_found`
> - `test_download_mail_attachments_with_regex_not_found`



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Updated] (AIRFLOW-2818) Cant' identify DAG .py files with capital case extension ('.PY', '.Py', '.pY')

2018-07-28 Thread Xiaodong DENG (JIRA)


 [ 
https://issues.apache.org/jira/browse/AIRFLOW-2818?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Xiaodong DENG updated AIRFLOW-2818:
---
Summary: Cant' identify DAG .py files with capital case extension ('.PY', 
'.Py', '.pY')  (was: Cant' identify DAG .py files with capital case extension 
('.PY' rather than '.py'))

> Cant' identify DAG .py files with capital case extension ('.PY', '.Py', '.pY')
> --
>
> Key: AIRFLOW-2818
> URL: https://issues.apache.org/jira/browse/AIRFLOW-2818
> Project: Apache Airflow
>  Issue Type: Bug
>  Components: DAG
>Reporter: Xiaodong DENG
>Assignee: Xiaodong DENG
>Priority: Major
>
> This bug is quite simply. 
> In multiple scripts in which we try to identify and list python files, the 
> capital case extension was not considered. For example, if I have a DAG file 
> with file name "dag.py", it will be identified. However, if I change the name 
> into "dag.PY", it will be missed by Airflow.
> This is because simply using function *os.path.splitext()* can not handle 
> case issue.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Commented] (AIRFLOW-2807) Add support for External ID when using STS Assume Role

2018-07-28 Thread ASF subversion and git services (JIRA)


[ 
https://issues.apache.org/jira/browse/AIRFLOW-2807?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16560780#comment-16560780
 ] 

ASF subversion and git services commented on AIRFLOW-2807:
--

Commit 449a7fd1b72639f1eb2bbeb033e1642e8eaac96c in incubator-airflow's branch 
refs/heads/master from [~vvondra]
[ https://git-wip-us.apache.org/repos/asf?p=incubator-airflow.git;h=449a7fd ]

[AIRFLOW-2807] Support STS Assume Role External ID

Currently the role assumption method works only if
the granting account
does not specify an External ID. The external ID
is used to solved the
confused deputy problem. When using the AWS hook
to export data to
multiple customers, it's good security practice to
use the external ID.

There is no backwards compatibility break, the ID
will be `None` in
existing cases. Moto doesn't provide any
convenient way to verify the
value was passed in the credential response in
tests, so existing
test cases are kept.

Documentation: https://docs.aws.amazon.com/IAM/lat
est/UserGuide/id_roles_create_for-
user_externalid.html

Closes #3647 from vvondra/support_sts_external_id


> Add support for External ID when using STS Assume Role
> --
>
> Key: AIRFLOW-2807
> URL: https://issues.apache.org/jira/browse/AIRFLOW-2807
> Project: Apache Airflow
>  Issue Type: Improvement
>  Components: aws, boto3, hooks
>Affects Versions: 1.10.1
>Reporter: Vojtech Vondra
>Priority: Minor
> Fix For: 2.0.0
>
>
> Currently the role assumption method works only if the granting account does 
> not specify an External ID. The external ID is used to solved the confused 
> deputy problem. When using the AWS hook to export data to multiple customers, 
> it's good security practice to use the external ID.
>  Documentation: 
> https://docs.aws.amazon.com/IAM/latest/UserGuide/id_roles_create_for-user_externalid.html



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


incubator-airflow git commit: [AIRFLOW-2807] Support STS Assume Role External ID

2018-07-28 Thread bolke
Repository: incubator-airflow
Updated Branches:
  refs/heads/master a5fa1bf88 -> 449a7fd1b


[AIRFLOW-2807] Support STS Assume Role External ID

Currently the role assumption method works only if
the granting account
does not specify an External ID. The external ID
is used to solved the
confused deputy problem. When using the AWS hook
to export data to
multiple customers, it's good security practice to
use the external ID.

There is no backwards compatibility break, the ID
will be `None` in
existing cases. Moto doesn't provide any
convenient way to verify the
value was passed in the credential response in
tests, so existing
test cases are kept.

Documentation: https://docs.aws.amazon.com/IAM/lat
est/UserGuide/id_roles_create_for-
user_externalid.html

Closes #3647 from vvondra/support_sts_external_id


Project: http://git-wip-us.apache.org/repos/asf/incubator-airflow/repo
Commit: http://git-wip-us.apache.org/repos/asf/incubator-airflow/commit/449a7fd1
Tree: http://git-wip-us.apache.org/repos/asf/incubator-airflow/tree/449a7fd1
Diff: http://git-wip-us.apache.org/repos/asf/incubator-airflow/diff/449a7fd1

Branch: refs/heads/master
Commit: 449a7fd1b72639f1eb2bbeb033e1642e8eaac96c
Parents: a5fa1bf
Author: Vojtech Vondra 
Authored: Sat Jul 28 17:32:09 2018 +0200
Committer: Bolke de Bruin 
Committed: Sat Jul 28 17:32:09 2018 +0200

--
 airflow/contrib/hooks/aws_hook.py| 15 ---
 tests/contrib/hooks/test_aws_hook.py | 19 +++
 2 files changed, 31 insertions(+), 3 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/incubator-airflow/blob/449a7fd1/airflow/contrib/hooks/aws_hook.py
--
diff --git a/airflow/contrib/hooks/aws_hook.py 
b/airflow/contrib/hooks/aws_hook.py
index c712d2d..69a1b0b 100644
--- a/airflow/contrib/hooks/aws_hook.py
+++ b/airflow/contrib/hooks/aws_hook.py
@@ -116,6 +116,7 @@ class AwsHook(BaseHook):
 region_name = 
connection_object.extra_dejson.get('region_name')
 
 role_arn = connection_object.extra_dejson.get('role_arn')
+external_id = connection_object.extra_dejson.get('external_id')
 aws_account_id = 
connection_object.extra_dejson.get('aws_account_id')
 aws_iam_role = 
connection_object.extra_dejson.get('aws_iam_role')
 
@@ -130,9 +131,17 @@ class AwsHook(BaseHook):
 region_name=region_name)
 
 sts_client = sts_session.client('sts')
-sts_response = sts_client.assume_role(
-RoleArn=role_arn,
-RoleSessionName='Airflow_' + self.aws_conn_id)
+
+if external_id is None:
+sts_response = sts_client.assume_role(
+RoleArn=role_arn,
+RoleSessionName='Airflow_' + self.aws_conn_id)
+else:
+sts_response = sts_client.assume_role(
+RoleArn=role_arn,
+RoleSessionName='Airflow_' + self.aws_conn_id,
+ExternalId=external_id)
+
 aws_access_key_id = 
sts_response['Credentials']['AccessKeyId']
 aws_secret_access_key = 
sts_response['Credentials']['SecretAccessKey']
 aws_session_token = 
sts_response['Credentials']['SessionToken']

http://git-wip-us.apache.org/repos/asf/incubator-airflow/blob/449a7fd1/tests/contrib/hooks/test_aws_hook.py
--
diff --git a/tests/contrib/hooks/test_aws_hook.py 
b/tests/contrib/hooks/test_aws_hook.py
index 719d6d1..b27b37d 100644
--- a/tests/contrib/hooks/test_aws_hook.py
+++ b/tests/contrib/hooks/test_aws_hook.py
@@ -164,6 +164,25 @@ class TestAwsHook(unittest.TestCase):
  
'gRmpRV3zrkuWJOgQs8IZZaIv2BXIa2R4OlgkBN9bkUDNCJiBeb/AXlzBBko7b15'
  'fjrBs2+cTQtpZ3CYWFXG8C5zqx37wnOE49mRl/+OtkIKGO7fAE')
 
+@unittest.skipIf(mock_sts is None, 'mock_sts package not present')
+@mock.patch.object(AwsHook, 'get_connection')
+@mock_sts
+def test_get_credentials_from_role_arn_with_external_id(self, 
mock_get_connection):
+mock_connection = Connection(
+extra='{"role_arn":"arn:aws:iam::123456:role/role_arn",'
+  ' "external_id":"external_id"}')
+mock_get_connection.return_value = mock_connection
+hook = AwsHook()
+credentials_from_hook = hook.get_credentials()
+self.assertEqual(credentials_from_hook.access_key, 
'AKIAIOSFODNN7EXAMPLE')
+self.assertEqual(credentials_from_hook.secret_key,
+ 'aJalrXUtnFEMI/K7MDENG/bPxRfiCYzEXAMPLEKEY')
+

[jira] [Resolved] (AIRFLOW-2807) Add support for External ID when using STS Assume Role

2018-07-28 Thread Bolke de Bruin (JIRA)


 [ 
https://issues.apache.org/jira/browse/AIRFLOW-2807?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Bolke de Bruin resolved AIRFLOW-2807.
-
   Resolution: Fixed
Fix Version/s: 2.0.0

Issue resolved by pull request #3647
[https://github.com/apache/incubator-airflow/pull/3647]

> Add support for External ID when using STS Assume Role
> --
>
> Key: AIRFLOW-2807
> URL: https://issues.apache.org/jira/browse/AIRFLOW-2807
> Project: Apache Airflow
>  Issue Type: Improvement
>  Components: aws, boto3, hooks
>Affects Versions: 1.10.1
>Reporter: Vojtech Vondra
>Priority: Minor
> Fix For: 2.0.0
>
>
> Currently the role assumption method works only if the granting account does 
> not specify an External ID. The external ID is used to solved the confused 
> deputy problem. When using the AWS hook to export data to multiple customers, 
> it's good security practice to use the external ID.
>  Documentation: 
> https://docs.aws.amazon.com/IAM/latest/UserGuide/id_roles_create_for-user_externalid.html



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Resolved] (AIRFLOW-2814) Default Arg "file_process_interval" for class SchedulerJob is inconsistent with doc

2018-07-28 Thread Kaxil Naik (JIRA)


 [ 
https://issues.apache.org/jira/browse/AIRFLOW-2814?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Kaxil Naik resolved AIRFLOW-2814.
-
   Resolution: Fixed
Fix Version/s: 2.0.0

Issue resolved by pull request #3659
[https://github.com/apache/incubator-airflow/pull/3659]

> Default Arg "file_process_interval" for class SchedulerJob is inconsistent 
> with doc
> ---
>
> Key: AIRFLOW-2814
> URL: https://issues.apache.org/jira/browse/AIRFLOW-2814
> Project: Apache Airflow
>  Issue Type: Bug
>  Components: scheduler
>Reporter: Xiaodong DENG
>Assignee: Xiaodong DENG
>Priority: Critical
> Fix For: 2.0.0
>
>
> h2. Backgrond
> In 
> [https://github.com/XD-DENG/incubator-airflow/blob/master/airflow/jobs.py#L592]
>  , it was mentioned the default value of argument *file_process_interval* 
> should be 3 minutes (*file_process_interval:* Parse and schedule each file no 
> faster than this interval).
> The value is normally parsed from the default configuration. However, in the 
> default config_template, its value is 0 rather than 180 seconds 
> ([https://github.com/XD-DENG/incubator-airflow/blob/master/airflow/config_templates/default_airflow.cfg#L432]
>  ). 
> h2. Issue
> This means that actually that each file is parsed and scheduled without 
> letting Airflow "rest". This conflicts with the design purpose (by default 
> let it be 180 seconds) and may affect performance significantly.
> h2. My Proposal
> Change the value in the config template from 0 to 180.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Commented] (AIRFLOW-2814) Default Arg "file_process_interval" for class SchedulerJob is inconsistent with doc

2018-07-28 Thread ASF subversion and git services (JIRA)


[ 
https://issues.apache.org/jira/browse/AIRFLOW-2814?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16560865#comment-16560865
 ] 

ASF subversion and git services commented on AIRFLOW-2814:
--

Commit 9b7525f251f93e625214ff680710f92306df9433 in incubator-airflow's branch 
refs/heads/master from XD-DENG
[ https://git-wip-us.apache.org/repos/asf?p=incubator-airflow.git;h=9b7525f ]

[AIRFLOW-2814] Fix inconsistent default config

value of min_file_process_interval in config
template is 0
However it's supposed to be 180 according to
airflow/jobs.py line 592

Closes #3659 from XD-DENG/patch-3


> Default Arg "file_process_interval" for class SchedulerJob is inconsistent 
> with doc
> ---
>
> Key: AIRFLOW-2814
> URL: https://issues.apache.org/jira/browse/AIRFLOW-2814
> Project: Apache Airflow
>  Issue Type: Bug
>  Components: scheduler
>Reporter: Xiaodong DENG
>Assignee: Xiaodong DENG
>Priority: Critical
> Fix For: 2.0.0
>
>
> h2. Backgrond
> In 
> [https://github.com/XD-DENG/incubator-airflow/blob/master/airflow/jobs.py#L592]
>  , it was mentioned the default value of argument *file_process_interval* 
> should be 3 minutes (*file_process_interval:* Parse and schedule each file no 
> faster than this interval).
> The value is normally parsed from the default configuration. However, in the 
> default config_template, its value is 0 rather than 180 seconds 
> ([https://github.com/XD-DENG/incubator-airflow/blob/master/airflow/config_templates/default_airflow.cfg#L432]
>  ). 
> h2. Issue
> This means that actually that each file is parsed and scheduled without 
> letting Airflow "rest". This conflicts with the design purpose (by default 
> let it be 180 seconds) and may affect performance significantly.
> h2. My Proposal
> Change the value in the config template from 0 to 180.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


incubator-airflow git commit: [AIRFLOW-2814] Fix inconsistent default config

2018-07-28 Thread kaxilnaik
Repository: incubator-airflow
Updated Branches:
  refs/heads/master 47fa65760 -> 9b7525f25


[AIRFLOW-2814] Fix inconsistent default config

value of min_file_process_interval in config
template is 0
However it's supposed to be 180 according to
airflow/jobs.py line 592

Closes #3659 from XD-DENG/patch-3


Project: http://git-wip-us.apache.org/repos/asf/incubator-airflow/repo
Commit: http://git-wip-us.apache.org/repos/asf/incubator-airflow/commit/9b7525f2
Tree: http://git-wip-us.apache.org/repos/asf/incubator-airflow/tree/9b7525f2
Diff: http://git-wip-us.apache.org/repos/asf/incubator-airflow/diff/9b7525f2

Branch: refs/heads/master
Commit: 9b7525f251f93e625214ff680710f92306df9433
Parents: 47fa657
Author: XD-DENG 
Authored: Sat Jul 28 20:48:17 2018 +0100
Committer: Kaxil Naik 
Committed: Sat Jul 28 20:48:17 2018 +0100

--
 airflow/config_templates/default_airflow.cfg | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/incubator-airflow/blob/9b7525f2/airflow/config_templates/default_airflow.cfg
--
diff --git a/airflow/config_templates/default_airflow.cfg 
b/airflow/config_templates/default_airflow.cfg
index 29a6e6b..828db54 100644
--- a/airflow/config_templates/default_airflow.cfg
+++ b/airflow/config_templates/default_airflow.cfg
@@ -428,8 +428,8 @@ scheduler_heartbeat_sec = 5
 # -1 indicates to run continuously (see also num_runs)
 run_duration = -1
 
-# after how much time a new DAGs should be picked up from the filesystem
-min_file_process_interval = 0
+# after how much time (seconds) a new DAGs should be picked up from the 
filesystem
+min_file_process_interval = 180
 
 # How many seconds to wait between file-parsing loops to prevent the logs from 
being spammed.
 min_file_parsing_loop_time = 1



[1/2] incubator-airflow git commit: [AIRFLOW-2816] Fix license text in docs/license.rst

2018-07-28 Thread bolke
Repository: incubator-airflow
Updated Branches:
  refs/heads/v1-10-test fdca7196d -> fa7574700


[AIRFLOW-2816] Fix license text in docs/license.rst

(cherry picked from commit af15f1150d3c2e02df9a8d330ac3dcc18c9d3e97)
Signed-off-by: Bolke de Bruin 


Project: http://git-wip-us.apache.org/repos/asf/incubator-airflow/repo
Commit: http://git-wip-us.apache.org/repos/asf/incubator-airflow/commit/868f9855
Tree: http://git-wip-us.apache.org/repos/asf/incubator-airflow/tree/868f9855
Diff: http://git-wip-us.apache.org/repos/asf/incubator-airflow/diff/868f9855

Branch: refs/heads/v1-10-test
Commit: 868f9855739cd899509d40017361130f4609
Parents: fdca719
Author: Bolke de Bruin 
Authored: Sat Jul 28 13:20:26 2018 +0200
Committer: Bolke de Bruin 
Committed: Sat Jul 28 17:18:08 2018 +0200

--
 docs/license.rst | 40 +++-
 1 file changed, 3 insertions(+), 37 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/incubator-airflow/blob/868f9855/docs/license.rst
--
diff --git a/docs/license.rst b/docs/license.rst
index f920a01..3c53035 100644
--- a/docs/license.rst
+++ b/docs/license.rst
@@ -6,9 +6,9 @@ License
 
 ::
 
-Apache License
-Version 2.0, January 2004
-http://www.apache.org/licenses/
+  Apache License
+Version 2.0, January 2004
+ http://www.apache.org/licenses/
 
 TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION
 
@@ -180,37 +180,3 @@ License
defend, and hold each Contributor harmless for any liability
incurred by, or claims asserted against, such Contributor by reason
of your accepting any such warranty or additional liability.
-
-END OF TERMS AND CONDITIONS
-
-APPENDIX: How to apply the Apache License to your work.
-
-   To apply the Apache License to your work, attach the following
-   boilerplate notice, with the fields enclosed by brackets "[]"
-   replaced with your own identifying information. (Don't include
-   the brackets!)  The text should be enclosed in the appropriate
-   comment syntax for the file format. We also recommend that a
-   file or class name and description of purpose be included on the
-   same "printed page" as the copyright notice for easier
-   identification within third-party archives.
-
-Copyright 2015 Apache Software Foundation
-
-Licensed to the Apache Software Foundation (ASF) under one
-or more contributor license agreements.  See the NOTICE file
-distributed with this work for additional information
-regarding copyright ownership.  The ASF licenses this file
-to you under the Apache License, Version 2.0 (the
-"License"); you may not use this file except in compliance
-with the License.  You may obtain a copy of the License at
-
-  http://www.apache.org/licenses/LICENSE-2.0
-
-Unless required by applicable law or agreed to in writing,
-software distributed under the License is distributed on an
-"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
-KIND, either express or implied.  See the License for the
-specific language governing permissions and limitations
-under the License.
-Status API Training Shop Blog About
-© 2016 GitHub, Inc. Terms Privacy Security Contact Help



[2/2] incubator-airflow git commit: [AIRFLOW-2779] Update licenses and remove copyright headers

2018-07-28 Thread bolke
[AIRFLOW-2779] Update licenses and remove copyright headers

* ssh_hook is not a port anymore
* auth backends should never had a copyright clause
* minihivecluster isnt used

(cherry picked from commit a5fa1bf8886ee3ced1306f8c2ff58541a72db96c)
Signed-off-by: Bolke de Bruin 


Project: http://git-wip-us.apache.org/repos/asf/incubator-airflow/repo
Commit: http://git-wip-us.apache.org/repos/asf/incubator-airflow/commit/fa757470
Tree: http://git-wip-us.apache.org/repos/asf/incubator-airflow/tree/fa757470
Diff: http://git-wip-us.apache.org/repos/asf/incubator-airflow/diff/fa757470

Branch: refs/heads/v1-10-test
Commit: fa7574700627cc381b8135392a1c78a76349dc83
Parents: 868f985
Author: Bolke de Bruin 
Authored: Sat Jul 28 14:25:05 2018 +0200
Committer: Bolke de Bruin 
Committed: Sat Jul 28 17:18:22 2018 +0200

--
 LICENSE |  17 +-
 NOTICE  |  17 +-
 .../auth/backends/github_enterprise_auth.py |   2 +-
 airflow/contrib/auth/backends/google_auth.py|   2 +-
 airflow/contrib/hooks/ssh_hook.py   |   3 -
 airflow/minihivecluster.py  |  47 -
 licenses/LICENSE-bootstrap3-typeahead.txt   |  20 ++
 licenses/LICENSE-hue.txt| 202 +++
 licenses/LICENSE-jqclock.txt| 201 ++
 licenses/LICENSE-scikit-learn.txt   |  31 +++
 10 files changed, 487 insertions(+), 55 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/incubator-airflow/blob/fa757470/LICENSE
--
diff --git a/LICENSE b/LICENSE
index 405540c..2288b87 100644
--- a/LICENSE
+++ b/LICENSE
@@ -210,6 +210,18 @@ limitations under the License.
 
 
 
+Third party Apache 2.0 licenses
+
+
+The following components are provided under the Apache 2.0 License.
+See project link for details. The text of each license is also included
+at licenses/LICENSE-[project].txt.
+
+(ALv2 License) hue (https://github.com/cloudera/hue/)
+(ALv2 License) jqclock 
(https://github.com/JohnRDOrazio/jQuery-Clock-Plugin)
+(ALv2 License) bootstrap3-typeahead 
(https://github.com/bassjobsen/Bootstrap-3-Typeahead)
+
+
 MIT licenses
 
 
@@ -239,10 +251,11 @@ The text of each license is also included at 
licenses/LICENSE-[project].txt.
 
 BSD 3-Clause licenses
 
-The following components are provided under the BSD 2-Clause license.
-See file headers and project links for details.
+The following components are provided under the BSD 3-Clause license. See 
project links for details.
 The text of each license is also included at licenses/LICENSE-[project].txt.
 
 (BSD 3 License) Ace (https://github.com/ajaxorg/ace)
 (BSD 3 License) d3js (https://d3js.org)
 (BSD 3 License) parallel-coordinates 
(http://syntagmatic.github.com/parallel-coordinates/)
+(BSD 3 License) scikit-learn (https://github.com/scikit-learn/scikit-learn)
+

http://git-wip-us.apache.org/repos/asf/incubator-airflow/blob/fa757470/NOTICE
--
diff --git a/NOTICE b/NOTICE
index a472730..544c080 100644
--- a/NOTICE
+++ b/NOTICE
@@ -1,6 +1,21 @@
 Apache Airflow
 Copyright 2016-2018 The Apache Software Foundation
 
-
 This product includes software developed at The Apache Software
 Foundation (http://www.apache.org/).
+
+===
+
+Apache Airflow contains subcomponents with separate copyright notices and
+license terms. Your use of the source code for the these subcomponents
+is subject to the terms and conditions of their respective licenses.
+
+See the LICENSE file for a list of subcomponents and dependencies and
+their respective licenses.
+
+hue:
+-
+This product contains a modified portion of 'Hue' developed by Cloudera, Inc.
+(https://github.com/cloudera/hue/).
+
+* Copyright 2009-2017 Cloudera Inc.

http://git-wip-us.apache.org/repos/asf/incubator-airflow/blob/fa757470/airflow/contrib/auth/backends/github_enterprise_auth.py
--
diff --git a/airflow/contrib/auth/backends/github_enterprise_auth.py 
b/airflow/contrib/auth/backends/github_enterprise_auth.py
index 5196725..641b81e 100644
--- a/airflow/contrib/auth/backends/github_enterprise_auth.py
+++ 

[jira] [Commented] (AIRFLOW-2816) Remove github copyright clause as it shouldnt be there

2018-07-28 Thread ASF subversion and git services (JIRA)


[ 
https://issues.apache.org/jira/browse/AIRFLOW-2816?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16560768#comment-16560768
 ] 

ASF subversion and git services commented on AIRFLOW-2816:
--

Commit 868f9855739cd899509d40017361130f4609 in incubator-airflow's branch 
refs/heads/v1-10-test from [~bolke]
[ https://git-wip-us.apache.org/repos/asf?p=incubator-airflow.git;h=868f985 ]

[AIRFLOW-2816] Fix license text in docs/license.rst

(cherry picked from commit af15f1150d3c2e02df9a8d330ac3dcc18c9d3e97)
Signed-off-by: Bolke de Bruin 


> Remove github copyright clause as it shouldnt be there
> --
>
> Key: AIRFLOW-2816
> URL: https://issues.apache.org/jira/browse/AIRFLOW-2816
> Project: Apache Airflow
>  Issue Type: Sub-task
>Reporter: Bolke de Bruin
>Priority: Major
>
> probably copy paste error



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Commented] (AIRFLOW-2779) Verify and correct licenses

2018-07-28 Thread ASF subversion and git services (JIRA)


[ 
https://issues.apache.org/jira/browse/AIRFLOW-2779?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16560769#comment-16560769
 ] 

ASF subversion and git services commented on AIRFLOW-2779:
--

Commit fa7574700627cc381b8135392a1c78a76349dc83 in incubator-airflow's branch 
refs/heads/v1-10-test from [~bolke]
[ https://git-wip-us.apache.org/repos/asf?p=incubator-airflow.git;h=fa75747 ]

[AIRFLOW-2779] Update licenses and remove copyright headers

* ssh_hook is not a port anymore
* auth backends should never had a copyright clause
* minihivecluster isnt used

(cherry picked from commit a5fa1bf8886ee3ced1306f8c2ff58541a72db96c)
Signed-off-by: Bolke de Bruin 


> Verify and correct licenses
> ---
>
> Key: AIRFLOW-2779
> URL: https://issues.apache.org/jira/browse/AIRFLOW-2779
> Project: Apache Airflow
>  Issue Type: Improvement
>Reporter: Bolke de Bruin
>Priority: Major
>  Labels: licenses
> Fix For: 2.0.0, 1.10.1
>
>
> # {color:#00}/airflow/security/utils.py{color}
> {color:#00}2. ./airflow/security/kerberos.py{color}
> {color:#00}3. ./airflow/www_rbac/static/jqClock.min.js{color}
> {color:#00}4. ./airflow/www/static/bootstrap3-typeahead.min.js{color}
> {color:#00}5. 
> ./apache-airflow-1.10.0rc2+incubating/scripts/ci/flake8_diff.sh{color}
> {color:#00}6. {color}[https://www.apache.org/legal/resolved.html#optional]
> {color:#00}7. ./docs/license.rst{color}
> {color:#00}8. airflow/contrib/auth/backends/google_auth.py{color}
> {color:#00}9. 
> /airflow/contrib/auth/backends/github_enterprise_auth.py{color}
> {color:#00}10. /airflow/contrib/hooks/ssh_hook.py{color}
> {color:#00}11. /airflow/minihivecluster.py{color}
> {color:#00}This files [1][2] seem to be 3rd party ALv2 licensed files 
> that refers to a NOTICE file, that information in that NOTICE file (at the 
> very least the copyright into) should be in your NOTICE file. This should 
> also be noted in LICENSE.{color}
>  
> {color:#00}LICENSE is:
> - missing jQuery clock [3] and typeahead [4], as they are ALv2 it’s not 
> required to list them but it’s a good idea to do so.
> - missing the license for this [5]
> - this file [7] oddly has © 2016 GitHub, [Inc.at|http://inc.at/] the bottom 
> of it{color}
>  
>  * {color:#00}Year in NOTICE is not correct "2016 and onwards” isn’t 
> valid as copyright has an expiry date{color}
>  



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[2/2] incubator-airflow git commit: [AIRFLOW-2779] Update licenses and remove copyright headers

2018-07-28 Thread bolke
[AIRFLOW-2779] Update licenses and remove copyright headers

* ssh_hook is not a port anymore
* auth backends should never had a copyright clause
* minihivecluster isnt used


Project: http://git-wip-us.apache.org/repos/asf/incubator-airflow/repo
Commit: http://git-wip-us.apache.org/repos/asf/incubator-airflow/commit/a5fa1bf8
Tree: http://git-wip-us.apache.org/repos/asf/incubator-airflow/tree/a5fa1bf8
Diff: http://git-wip-us.apache.org/repos/asf/incubator-airflow/diff/a5fa1bf8

Branch: refs/heads/master
Commit: a5fa1bf8886ee3ced1306f8c2ff58541a72db96c
Parents: af15f11
Author: Bolke de Bruin 
Authored: Sat Jul 28 14:25:05 2018 +0200
Committer: Bolke de Bruin 
Committed: Sat Jul 28 14:25:05 2018 +0200

--
 LICENSE |  17 +-
 NOTICE  |  17 +-
 .../auth/backends/github_enterprise_auth.py |   2 +-
 airflow/contrib/auth/backends/google_auth.py|   2 +-
 airflow/contrib/hooks/ssh_hook.py   |   3 -
 airflow/minihivecluster.py  |  47 -
 licenses/LICENSE-bootstrap3-typeahead.txt   |  20 ++
 licenses/LICENSE-hue.txt| 202 +++
 licenses/LICENSE-jqclock.txt| 201 ++
 licenses/LICENSE-scikit-learn.txt   |  31 +++
 10 files changed, 487 insertions(+), 55 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/incubator-airflow/blob/a5fa1bf8/LICENSE
--
diff --git a/LICENSE b/LICENSE
index 405540c..2288b87 100644
--- a/LICENSE
+++ b/LICENSE
@@ -210,6 +210,18 @@ limitations under the License.
 
 
 
+Third party Apache 2.0 licenses
+
+
+The following components are provided under the Apache 2.0 License.
+See project link for details. The text of each license is also included
+at licenses/LICENSE-[project].txt.
+
+(ALv2 License) hue (https://github.com/cloudera/hue/)
+(ALv2 License) jqclock 
(https://github.com/JohnRDOrazio/jQuery-Clock-Plugin)
+(ALv2 License) bootstrap3-typeahead 
(https://github.com/bassjobsen/Bootstrap-3-Typeahead)
+
+
 MIT licenses
 
 
@@ -239,10 +251,11 @@ The text of each license is also included at 
licenses/LICENSE-[project].txt.
 
 BSD 3-Clause licenses
 
-The following components are provided under the BSD 2-Clause license.
-See file headers and project links for details.
+The following components are provided under the BSD 3-Clause license. See 
project links for details.
 The text of each license is also included at licenses/LICENSE-[project].txt.
 
 (BSD 3 License) Ace (https://github.com/ajaxorg/ace)
 (BSD 3 License) d3js (https://d3js.org)
 (BSD 3 License) parallel-coordinates 
(http://syntagmatic.github.com/parallel-coordinates/)
+(BSD 3 License) scikit-learn (https://github.com/scikit-learn/scikit-learn)
+

http://git-wip-us.apache.org/repos/asf/incubator-airflow/blob/a5fa1bf8/NOTICE
--
diff --git a/NOTICE b/NOTICE
index a472730..544c080 100644
--- a/NOTICE
+++ b/NOTICE
@@ -1,6 +1,21 @@
 Apache Airflow
 Copyright 2016-2018 The Apache Software Foundation
 
-
 This product includes software developed at The Apache Software
 Foundation (http://www.apache.org/).
+
+===
+
+Apache Airflow contains subcomponents with separate copyright notices and
+license terms. Your use of the source code for the these subcomponents
+is subject to the terms and conditions of their respective licenses.
+
+See the LICENSE file for a list of subcomponents and dependencies and
+their respective licenses.
+
+hue:
+-
+This product contains a modified portion of 'Hue' developed by Cloudera, Inc.
+(https://github.com/cloudera/hue/).
+
+* Copyright 2009-2017 Cloudera Inc.

http://git-wip-us.apache.org/repos/asf/incubator-airflow/blob/a5fa1bf8/airflow/contrib/auth/backends/github_enterprise_auth.py
--
diff --git a/airflow/contrib/auth/backends/github_enterprise_auth.py 
b/airflow/contrib/auth/backends/github_enterprise_auth.py
index 5196725..641b81e 100644
--- a/airflow/contrib/auth/backends/github_enterprise_auth.py
+++ b/airflow/contrib/auth/backends/github_enterprise_auth.py
@@ -1,4 +1,4 @@
-# Copyright 2015 Matthew Pelland (m...@pelland.io)
+# -*- coding: 

[jira] [Commented] (AIRFLOW-2816) Remove github copyright clause as it shouldnt be there

2018-07-28 Thread ASF subversion and git services (JIRA)


[ 
https://issues.apache.org/jira/browse/AIRFLOW-2816?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16560741#comment-16560741
 ] 

ASF subversion and git services commented on AIRFLOW-2816:
--

Commit af15f1150d3c2e02df9a8d330ac3dcc18c9d3e97 in incubator-airflow's branch 
refs/heads/master from [~bolke]
[ https://git-wip-us.apache.org/repos/asf?p=incubator-airflow.git;h=af15f11 ]

[AIRFLOW-2816] Fix license text in docs/license.rst


> Remove github copyright clause as it shouldnt be there
> --
>
> Key: AIRFLOW-2816
> URL: https://issues.apache.org/jira/browse/AIRFLOW-2816
> Project: Apache Airflow
>  Issue Type: Sub-task
>Reporter: Bolke de Bruin
>Priority: Major
>
> probably copy paste error



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Created] (AIRFLOW-2817) Force explicit choice on GPL dependency

2018-07-28 Thread Bolke de Bruin (JIRA)
Bolke de Bruin created AIRFLOW-2817:
---

 Summary: Force explicit choice on GPL dependency
 Key: AIRFLOW-2817
 URL: https://issues.apache.org/jira/browse/AIRFLOW-2817
 Project: Apache Airflow
  Issue Type: Improvement
Reporter: Bolke de Bruin


A more explicit choice on GPL dependency was required by the IPMC



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[1/2] incubator-airflow git commit: [AIRFLOW-2816] Fix license text in docs/license.rst

2018-07-28 Thread bolke
Repository: incubator-airflow
Updated Branches:
  refs/heads/master 4f562ba8c -> a5fa1bf88


[AIRFLOW-2816] Fix license text in docs/license.rst


Project: http://git-wip-us.apache.org/repos/asf/incubator-airflow/repo
Commit: http://git-wip-us.apache.org/repos/asf/incubator-airflow/commit/af15f115
Tree: http://git-wip-us.apache.org/repos/asf/incubator-airflow/tree/af15f115
Diff: http://git-wip-us.apache.org/repos/asf/incubator-airflow/diff/af15f115

Branch: refs/heads/master
Commit: af15f1150d3c2e02df9a8d330ac3dcc18c9d3e97
Parents: 4f562ba
Author: Bolke de Bruin 
Authored: Sat Jul 28 13:20:26 2018 +0200
Committer: Bolke de Bruin 
Committed: Sat Jul 28 13:20:26 2018 +0200

--
 docs/license.rst | 40 +++-
 1 file changed, 3 insertions(+), 37 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/incubator-airflow/blob/af15f115/docs/license.rst
--
diff --git a/docs/license.rst b/docs/license.rst
index f920a01..3c53035 100644
--- a/docs/license.rst
+++ b/docs/license.rst
@@ -6,9 +6,9 @@ License
 
 ::
 
-Apache License
-Version 2.0, January 2004
-http://www.apache.org/licenses/
+  Apache License
+Version 2.0, January 2004
+ http://www.apache.org/licenses/
 
 TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION
 
@@ -180,37 +180,3 @@ License
defend, and hold each Contributor harmless for any liability
incurred by, or claims asserted against, such Contributor by reason
of your accepting any such warranty or additional liability.
-
-END OF TERMS AND CONDITIONS
-
-APPENDIX: How to apply the Apache License to your work.
-
-   To apply the Apache License to your work, attach the following
-   boilerplate notice, with the fields enclosed by brackets "[]"
-   replaced with your own identifying information. (Don't include
-   the brackets!)  The text should be enclosed in the appropriate
-   comment syntax for the file format. We also recommend that a
-   file or class name and description of purpose be included on the
-   same "printed page" as the copyright notice for easier
-   identification within third-party archives.
-
-Copyright 2015 Apache Software Foundation
-
-Licensed to the Apache Software Foundation (ASF) under one
-or more contributor license agreements.  See the NOTICE file
-distributed with this work for additional information
-regarding copyright ownership.  The ASF licenses this file
-to you under the Apache License, Version 2.0 (the
-"License"); you may not use this file except in compliance
-with the License.  You may obtain a copy of the License at
-
-  http://www.apache.org/licenses/LICENSE-2.0
-
-Unless required by applicable law or agreed to in writing,
-software distributed under the License is distributed on an
-"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
-KIND, either express or implied.  See the License for the
-specific language governing permissions and limitations
-under the License.
-Status API Training Shop Blog About
-© 2016 GitHub, Inc. Terms Privacy Security Contact Help



[2/2] incubator-airflow git commit: [AIRFLOW-2816] Fix license text in docs/license.rst

2018-07-28 Thread bolke
[AIRFLOW-2816] Fix license text in docs/license.rst

(cherry picked from commit af15f1150d3c2e02df9a8d330ac3dcc18c9d3e97)
Signed-off-by: Bolke de Bruin 


Project: http://git-wip-us.apache.org/repos/asf/incubator-airflow/repo
Commit: http://git-wip-us.apache.org/repos/asf/incubator-airflow/commit/5f3a4cf3
Tree: http://git-wip-us.apache.org/repos/asf/incubator-airflow/tree/5f3a4cf3
Diff: http://git-wip-us.apache.org/repos/asf/incubator-airflow/diff/5f3a4cf3

Branch: refs/heads/v1-10-stable
Commit: 5f3a4cf3037a6caa2f464f3c9c43cb02513478aa
Parents: eb34cd5
Author: Bolke de Bruin 
Authored: Sat Jul 28 13:20:26 2018 +0200
Committer: Bolke de Bruin 
Committed: Sat Jul 28 17:18:44 2018 +0200

--
 docs/license.rst | 40 +++-
 1 file changed, 3 insertions(+), 37 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/incubator-airflow/blob/5f3a4cf3/docs/license.rst
--
diff --git a/docs/license.rst b/docs/license.rst
index f920a01..3c53035 100644
--- a/docs/license.rst
+++ b/docs/license.rst
@@ -6,9 +6,9 @@ License
 
 ::
 
-Apache License
-Version 2.0, January 2004
-http://www.apache.org/licenses/
+  Apache License
+Version 2.0, January 2004
+ http://www.apache.org/licenses/
 
 TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION
 
@@ -180,37 +180,3 @@ License
defend, and hold each Contributor harmless for any liability
incurred by, or claims asserted against, such Contributor by reason
of your accepting any such warranty or additional liability.
-
-END OF TERMS AND CONDITIONS
-
-APPENDIX: How to apply the Apache License to your work.
-
-   To apply the Apache License to your work, attach the following
-   boilerplate notice, with the fields enclosed by brackets "[]"
-   replaced with your own identifying information. (Don't include
-   the brackets!)  The text should be enclosed in the appropriate
-   comment syntax for the file format. We also recommend that a
-   file or class name and description of purpose be included on the
-   same "printed page" as the copyright notice for easier
-   identification within third-party archives.
-
-Copyright 2015 Apache Software Foundation
-
-Licensed to the Apache Software Foundation (ASF) under one
-or more contributor license agreements.  See the NOTICE file
-distributed with this work for additional information
-regarding copyright ownership.  The ASF licenses this file
-to you under the Apache License, Version 2.0 (the
-"License"); you may not use this file except in compliance
-with the License.  You may obtain a copy of the License at
-
-  http://www.apache.org/licenses/LICENSE-2.0
-
-Unless required by applicable law or agreed to in writing,
-software distributed under the License is distributed on an
-"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
-KIND, either express or implied.  See the License for the
-specific language governing permissions and limitations
-under the License.
-Status API Training Shop Blog About
-© 2016 GitHub, Inc. Terms Privacy Security Contact Help



[1/2] incubator-airflow git commit: [AIRFLOW-2779] Update licenses and remove copyright headers

2018-07-28 Thread bolke
Repository: incubator-airflow
Updated Branches:
  refs/heads/v1-10-stable f0549211f -> 5f3a4cf30


[AIRFLOW-2779] Update licenses and remove copyright headers

* ssh_hook is not a port anymore
* auth backends should never had a copyright clause
* minihivecluster isnt used

(cherry picked from commit a5fa1bf8886ee3ced1306f8c2ff58541a72db96c)
Signed-off-by: Bolke de Bruin 


Project: http://git-wip-us.apache.org/repos/asf/incubator-airflow/repo
Commit: http://git-wip-us.apache.org/repos/asf/incubator-airflow/commit/eb34cd59
Tree: http://git-wip-us.apache.org/repos/asf/incubator-airflow/tree/eb34cd59
Diff: http://git-wip-us.apache.org/repos/asf/incubator-airflow/diff/eb34cd59

Branch: refs/heads/v1-10-stable
Commit: eb34cd596b5995474aeaa6a13829df0d52673cde
Parents: f054921
Author: Bolke de Bruin 
Authored: Sat Jul 28 14:25:05 2018 +0200
Committer: Bolke de Bruin 
Committed: Sat Jul 28 17:18:41 2018 +0200

--
 LICENSE |  17 +-
 NOTICE  |  17 +-
 .../auth/backends/github_enterprise_auth.py |   2 +-
 airflow/contrib/auth/backends/google_auth.py|   2 +-
 airflow/contrib/hooks/ssh_hook.py   |   3 -
 airflow/minihivecluster.py  |  47 -
 licenses/LICENSE-bootstrap3-typeahead.txt   |  20 ++
 licenses/LICENSE-hue.txt| 202 +++
 licenses/LICENSE-jqclock.txt| 201 ++
 licenses/LICENSE-scikit-learn.txt   |  31 +++
 10 files changed, 487 insertions(+), 55 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/incubator-airflow/blob/eb34cd59/LICENSE
--
diff --git a/LICENSE b/LICENSE
index 405540c..2288b87 100644
--- a/LICENSE
+++ b/LICENSE
@@ -210,6 +210,18 @@ limitations under the License.
 
 
 
+Third party Apache 2.0 licenses
+
+
+The following components are provided under the Apache 2.0 License.
+See project link for details. The text of each license is also included
+at licenses/LICENSE-[project].txt.
+
+(ALv2 License) hue (https://github.com/cloudera/hue/)
+(ALv2 License) jqclock 
(https://github.com/JohnRDOrazio/jQuery-Clock-Plugin)
+(ALv2 License) bootstrap3-typeahead 
(https://github.com/bassjobsen/Bootstrap-3-Typeahead)
+
+
 MIT licenses
 
 
@@ -239,10 +251,11 @@ The text of each license is also included at 
licenses/LICENSE-[project].txt.
 
 BSD 3-Clause licenses
 
-The following components are provided under the BSD 2-Clause license.
-See file headers and project links for details.
+The following components are provided under the BSD 3-Clause license. See 
project links for details.
 The text of each license is also included at licenses/LICENSE-[project].txt.
 
 (BSD 3 License) Ace (https://github.com/ajaxorg/ace)
 (BSD 3 License) d3js (https://d3js.org)
 (BSD 3 License) parallel-coordinates 
(http://syntagmatic.github.com/parallel-coordinates/)
+(BSD 3 License) scikit-learn (https://github.com/scikit-learn/scikit-learn)
+

http://git-wip-us.apache.org/repos/asf/incubator-airflow/blob/eb34cd59/NOTICE
--
diff --git a/NOTICE b/NOTICE
index a472730..544c080 100644
--- a/NOTICE
+++ b/NOTICE
@@ -1,6 +1,21 @@
 Apache Airflow
 Copyright 2016-2018 The Apache Software Foundation
 
-
 This product includes software developed at The Apache Software
 Foundation (http://www.apache.org/).
+
+===
+
+Apache Airflow contains subcomponents with separate copyright notices and
+license terms. Your use of the source code for the these subcomponents
+is subject to the terms and conditions of their respective licenses.
+
+See the LICENSE file for a list of subcomponents and dependencies and
+their respective licenses.
+
+hue:
+-
+This product contains a modified portion of 'Hue' developed by Cloudera, Inc.
+(https://github.com/cloudera/hue/).
+
+* Copyright 2009-2017 Cloudera Inc.

http://git-wip-us.apache.org/repos/asf/incubator-airflow/blob/eb34cd59/airflow/contrib/auth/backends/github_enterprise_auth.py
--
diff --git a/airflow/contrib/auth/backends/github_enterprise_auth.py 
b/airflow/contrib/auth/backends/github_enterprise_auth.py
index 5196725..641b81e 100644

[jira] [Commented] (AIRFLOW-2816) Remove github copyright clause as it shouldnt be there

2018-07-28 Thread ASF subversion and git services (JIRA)


[ 
https://issues.apache.org/jira/browse/AIRFLOW-2816?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16560771#comment-16560771
 ] 

ASF subversion and git services commented on AIRFLOW-2816:
--

Commit 5f3a4cf3037a6caa2f464f3c9c43cb02513478aa in incubator-airflow's branch 
refs/heads/v1-10-stable from [~bolke]
[ https://git-wip-us.apache.org/repos/asf?p=incubator-airflow.git;h=5f3a4cf ]

[AIRFLOW-2816] Fix license text in docs/license.rst

(cherry picked from commit af15f1150d3c2e02df9a8d330ac3dcc18c9d3e97)
Signed-off-by: Bolke de Bruin 


> Remove github copyright clause as it shouldnt be there
> --
>
> Key: AIRFLOW-2816
> URL: https://issues.apache.org/jira/browse/AIRFLOW-2816
> Project: Apache Airflow
>  Issue Type: Sub-task
>Reporter: Bolke de Bruin
>Priority: Major
>
> probably copy paste error



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Commented] (AIRFLOW-2779) Verify and correct licenses

2018-07-28 Thread ASF subversion and git services (JIRA)


[ 
https://issues.apache.org/jira/browse/AIRFLOW-2779?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16560770#comment-16560770
 ] 

ASF subversion and git services commented on AIRFLOW-2779:
--

Commit eb34cd596b5995474aeaa6a13829df0d52673cde in incubator-airflow's branch 
refs/heads/v1-10-stable from [~bolke]
[ https://git-wip-us.apache.org/repos/asf?p=incubator-airflow.git;h=eb34cd5 ]

[AIRFLOW-2779] Update licenses and remove copyright headers

* ssh_hook is not a port anymore
* auth backends should never had a copyright clause
* minihivecluster isnt used

(cherry picked from commit a5fa1bf8886ee3ced1306f8c2ff58541a72db96c)
Signed-off-by: Bolke de Bruin 


> Verify and correct licenses
> ---
>
> Key: AIRFLOW-2779
> URL: https://issues.apache.org/jira/browse/AIRFLOW-2779
> Project: Apache Airflow
>  Issue Type: Improvement
>Reporter: Bolke de Bruin
>Priority: Major
>  Labels: licenses
> Fix For: 2.0.0, 1.10.1
>
>
> # {color:#00}/airflow/security/utils.py{color}
> {color:#00}2. ./airflow/security/kerberos.py{color}
> {color:#00}3. ./airflow/www_rbac/static/jqClock.min.js{color}
> {color:#00}4. ./airflow/www/static/bootstrap3-typeahead.min.js{color}
> {color:#00}5. 
> ./apache-airflow-1.10.0rc2+incubating/scripts/ci/flake8_diff.sh{color}
> {color:#00}6. {color}[https://www.apache.org/legal/resolved.html#optional]
> {color:#00}7. ./docs/license.rst{color}
> {color:#00}8. airflow/contrib/auth/backends/google_auth.py{color}
> {color:#00}9. 
> /airflow/contrib/auth/backends/github_enterprise_auth.py{color}
> {color:#00}10. /airflow/contrib/hooks/ssh_hook.py{color}
> {color:#00}11. /airflow/minihivecluster.py{color}
> {color:#00}This files [1][2] seem to be 3rd party ALv2 licensed files 
> that refers to a NOTICE file, that information in that NOTICE file (at the 
> very least the copyright into) should be in your NOTICE file. This should 
> also be noted in LICENSE.{color}
>  
> {color:#00}LICENSE is:
> - missing jQuery clock [3] and typeahead [4], as they are ALv2 it’s not 
> required to list them but it’s a good idea to do so.
> - missing the license for this [5]
> - this file [7] oddly has © 2016 GitHub, [Inc.at|http://inc.at/] the bottom 
> of it{color}
>  
>  * {color:#00}Year in NOTICE is not correct "2016 and onwards” isn’t 
> valid as copyright has an expiry date{color}
>  



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)