[jira] [Commented] (AIRFLOW-1739) Cleanup naming ambiguity with TestDbApiHook test class
[ https://issues.apache.org/jira/browse/AIRFLOW-1739?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16696436#comment-16696436 ] ASF GitHub Bot commented on AIRFLOW-1739: - msumit closed pull request #2709: [AIRFLOW-1739] Resolve TestDbApiHook naming ambiguity URL: https://github.com/apache/incubator-airflow/pull/2709 This is a PR merged from a forked repository. As GitHub hides the original diff on merge, it is displayed below for the sake of provenance: As this is a foreign pull request (from a fork), the diff is supplied below (as it won't show otherwise due to GitHub magic): diff --git a/tests/hooks/test_dbapi_hook.py b/tests/hooks/test_dbapi_hook.py index 3484ee99da..10cd09dea4 100644 --- a/tests/hooks/test_dbapi_hook.py +++ b/tests/hooks/test_dbapi_hook.py @@ -34,13 +34,13 @@ def setUp(self): self.conn.cursor.return_value = self.cur conn = self.conn -class TestDBApiHook(DbApiHook): +class UnitTestDbApiHook(DbApiHook): conn_name_attr = 'test_conn_id' def get_conn(self): return conn -self.db_hook = TestDBApiHook() +self.db_hook = UnitTestDbApiHook() def test_get_records(self): statement = "SQL" This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org > Cleanup naming ambiguity with TestDbApiHook test class > -- > > Key: AIRFLOW-1739 > URL: https://issues.apache.org/jira/browse/AIRFLOW-1739 > Project: Apache Airflow > Issue Type: Bug >Reporter: Andy Hadjigeorgiou >Assignee: Andy Hadjigeorgiou >Priority: Trivial > > The TestDbApiHook class creates a class whose name is TestDBApiHook - I'm > proposing a simple naming change to make sure the two are distinguishable. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[GitHub] msumit closed pull request #2709: [AIRFLOW-1739] Resolve TestDbApiHook naming ambiguity
msumit closed pull request #2709: [AIRFLOW-1739] Resolve TestDbApiHook naming ambiguity URL: https://github.com/apache/incubator-airflow/pull/2709 This is a PR merged from a forked repository. As GitHub hides the original diff on merge, it is displayed below for the sake of provenance: As this is a foreign pull request (from a fork), the diff is supplied below (as it won't show otherwise due to GitHub magic): diff --git a/tests/hooks/test_dbapi_hook.py b/tests/hooks/test_dbapi_hook.py index 3484ee99da..10cd09dea4 100644 --- a/tests/hooks/test_dbapi_hook.py +++ b/tests/hooks/test_dbapi_hook.py @@ -34,13 +34,13 @@ def setUp(self): self.conn.cursor.return_value = self.cur conn = self.conn -class TestDBApiHook(DbApiHook): +class UnitTestDbApiHook(DbApiHook): conn_name_attr = 'test_conn_id' def get_conn(self): return conn -self.db_hook = TestDBApiHook() +self.db_hook = UnitTestDbApiHook() def test_get_records(self): statement = "SQL" This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] chrissng removed a comment on issue #3937: [AIRFLOW-3103][AIRFLOW-3147] Update flask-appbuilder
chrissng removed a comment on issue #3937: [AIRFLOW-3103][AIRFLOW-3147] Update flask-appbuilder URL: https://github.com/apache/incubator-airflow/pull/3937#issuecomment-441149364 It seems that this change has caused the `Profile and Login` not appear in RBAC ui anymore, despite being logged-in. Am I the only one seeing this ? This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] chrissng commented on issue #3937: [AIRFLOW-3103][AIRFLOW-3147] Update flask-appbuilder
chrissng commented on issue #3937: [AIRFLOW-3103][AIRFLOW-3147] Update flask-appbuilder URL: https://github.com/apache/incubator-airflow/pull/3937#issuecomment-441149364 It seems that this change has caused the `Profile and Login` not appear in RBAC ui anymore, despite being logged-in. Am I the only one seeing this ? This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Commented] (AIRFLOW-3326) High Sierra Complaining 'in progress in another thread when fork() was called'
[
https://issues.apache.org/jira/browse/AIRFLOW-3326?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16696299#comment-16696299
]
Ryan Yuan commented on AIRFLOW-3326:
[~xnuinside] Yes.
And it is a known issue in OS High Sierra that also happened to other projects.
Reference:
[https://github.com/ansible/ansible/issues/32499]
[https://github.com/rtomayko/shotgun/issues/69]
There is no issues anymore after executing
{code:java}
export OBJC_DISABLE_INITIALIZE_FORK_SAFETY=YES{code}
> High Sierra Complaining 'in progress in another thread when fork() was called'
> --
>
> Key: AIRFLOW-3326
> URL: https://issues.apache.org/jira/browse/AIRFLOW-3326
> Project: Apache Airflow
> Issue Type: Bug
>Affects Versions: 1.10.0
> Environment: macOS High Sierra 10.13.6 (17G65)
>Reporter: Ryan Yuan
>Assignee: Ryan Yuan
>Priority: Blocker
>
> Inside the plugins folder, I have a hook that is a child class of
> BigQueryHook.
> {code:java}
> // code
> from airflow.contrib.hooks.bigquery_hook import BigQueryHook
> class BQHook(BigQueryHook):
> pass{code}
> When I run the airflow server, it keeps throwing messages complaining 'in
> progress in another thread when fork() was called', and I can't use the web
> server UI at all.
> {code:java}
> // messages from terminal
> objc[15098]: +[__NSPlaceholderDate initialize] may have been in progress in
> another thread when fork() was called.
> objc[15098]: +[__NSPlaceholderDate initialize] may have been in progress in
> another thread when fork() was called. We cannot safely call it or ignore it
> in the fork() child process. Crashing instead. Set a breakpoint on
> objc_initializeAfterForkError to debug.
> [2018-11-12 14:03:40 +1100] [15102] [INFO] Booting worker with pid: 15102
> [2018-11-12 14:03:40,792] {__init__.py:51} INFO - Using executor
> SequentialExecutor
> [2018-11-12 14:03:40,851] {base_hook.py:83} INFO - Using connection to:
> https://custom-data-z00100-dev.appspot.com/
> objc[15099]: +[__NSPlaceholderDate initialize] may have been in progress in
> another thread when fork() was called.
> objc[15099]: +[__NSPlaceholderDate initialize] may have been in progress in
> another thread when fork() was called. We cannot safely call it or ignore it
> in the fork() child process. Crashing instead. Set a breakpoint on
> objc_initializeAfterForkError to debug.
> [2018-11-12 14:03:40 +1100] [15103] [INFO] Booting worker with pid: 15103
> [2018-11-12 14:03:40,902] {base_hook.py:83} INFO - Using connection to:
> https://custom-data-z00100-dev.appspot.com/
> objc[15101]: +[__NSPlaceholderDate initialize] may have been in progress in
> another thread when fork() was called.
> objc[15101]: +[__NSPlaceholderDate initialize] may have been in progress in
> another thread when fork() was called. We cannot safely call it or ignore it
> in the fork() child process. Crashing instead. Set a breakpoint on
> objc_initializeAfterForkError to debug.
> [2018-11-12 14:03:40 +1100] [15104] [INFO] Booting worker with pid: 15104
> [2018-11-12 14:03:40,948] {base_hook.py:83} INFO - Using connection to:
> https://custom-data-z00100-dev.appspot.com/
> objc[15100]: +[__NSPlaceholderDate initialize] may have been in progress in
> another thread when fork() was called.
> objc[15100]: +[__NSPlaceholderDate initialize] may have been in progress in
> another thread when fork() was called. We cannot safely call it or ignore it
> in the fork() child process. Crashing instead. Set a breakpoint on
> objc_initializeAfterForkError to debug.
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (AIRFLOW-3348) Refresh run stats on dag refresh
[
https://issues.apache.org/jira/browse/AIRFLOW-3348?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16696280#comment-16696280
]
ASF GitHub Bot commented on AIRFLOW-3348:
-
feng-tao closed pull request #4197: [AIRFLOW-3348] update run statistics on
dag refresh
URL: https://github.com/apache/incubator-airflow/pull/4197
This is a PR merged from a forked repository.
As GitHub hides the original diff on merge, it is displayed below for
the sake of provenance:
As this is a foreign pull request (from a fork), the diff is supplied
below (as it won't show otherwise due to GitHub magic):
diff --git a/airflow/www/views.py b/airflow/www/views.py
index 67c31bb9b1..8792191e21 100644
--- a/airflow/www/views.py
+++ b/airflow/www/views.py
@@ -1877,6 +1877,8 @@ def refresh(self, session=None):
session.merge(orm_dag)
session.commit()
+models.DagStat.update([dag_id], session=session, dirty_only=False)
+
dagbag.get_dag(dag_id)
flash("DAG [{}] is now fresh as a daisy".format(dag_id))
return redirect(request.referrer)
diff --git a/airflow/www_rbac/views.py b/airflow/www_rbac/views.py
index 29e8da1b9c..1825e15000 100644
--- a/airflow/www_rbac/views.py
+++ b/airflow/www_rbac/views.py
@@ -1627,6 +1627,8 @@ def refresh(self, session=None):
# sync dag permission
appbuilder.sm.sync_perm_for_dag(dag_id)
+models.DagStat.update([dag_id], session=session, dirty_only=False)
+
dagbag.get_dag(dag_id)
flash("DAG [{}] is now fresh as a daisy".format(dag_id))
return redirect(request.referrer)
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
> Refresh run stats on dag refresh
>
>
> Key: AIRFLOW-3348
> URL: https://issues.apache.org/jira/browse/AIRFLOW-3348
> Project: Apache Airflow
> Issue Type: Improvement
>Reporter: Marcin Szymanski
>Assignee: Marcin Szymanski
>Priority: Minor
>
> In some cases dag run statistics may become outdated, for example, when dag
> run is deleted.
> As the view from which dag runs are deleted permits to delete runs from
> multiple dags in a single transaction, this seems to be the most reasonable
> place to update run statistics
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[GitHub] feng-tao closed pull request #4197: [AIRFLOW-3348] update run statistics on dag refresh
feng-tao closed pull request #4197: [AIRFLOW-3348] update run statistics on
dag refresh
URL: https://github.com/apache/incubator-airflow/pull/4197
This is a PR merged from a forked repository.
As GitHub hides the original diff on merge, it is displayed below for
the sake of provenance:
As this is a foreign pull request (from a fork), the diff is supplied
below (as it won't show otherwise due to GitHub magic):
diff --git a/airflow/www/views.py b/airflow/www/views.py
index 67c31bb9b1..8792191e21 100644
--- a/airflow/www/views.py
+++ b/airflow/www/views.py
@@ -1877,6 +1877,8 @@ def refresh(self, session=None):
session.merge(orm_dag)
session.commit()
+models.DagStat.update([dag_id], session=session, dirty_only=False)
+
dagbag.get_dag(dag_id)
flash("DAG [{}] is now fresh as a daisy".format(dag_id))
return redirect(request.referrer)
diff --git a/airflow/www_rbac/views.py b/airflow/www_rbac/views.py
index 29e8da1b9c..1825e15000 100644
--- a/airflow/www_rbac/views.py
+++ b/airflow/www_rbac/views.py
@@ -1627,6 +1627,8 @@ def refresh(self, session=None):
# sync dag permission
appbuilder.sm.sync_perm_for_dag(dag_id)
+models.DagStat.update([dag_id], session=session, dirty_only=False)
+
dagbag.get_dag(dag_id)
flash("DAG [{}] is now fresh as a daisy".format(dag_id))
return redirect(request.referrer)
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] ms32035 edited a comment on issue #4197: [AIRFLOW-3348] update run statistics on dag refresh
ms32035 edited a comment on issue #4197: [AIRFLOW-3348] update run statistics on dag refresh URL: https://github.com/apache/incubator-airflow/pull/4197#issuecomment-441128565 @feng-tao , modified as requested This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] ms32035 commented on issue #4197: [AIRFLOW-3348] update run statistics on dag refresh
ms32035 commented on issue #4197: [AIRFLOW-3348] update run statistics on dag refresh URL: https://github.com/apache/incubator-airflow/pull/4197#issuecomment-441128565 @ms32035 , modified as requested This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] codecov-io edited a comment on issue #4226: docs: Remove quotes from domain in Google Oauth
codecov-io edited a comment on issue #4226: docs: Remove quotes from domain in Google Oauth URL: https://github.com/apache/incubator-airflow/pull/4226#issuecomment-441109306 # [Codecov](https://codecov.io/gh/apache/incubator-airflow/pull/4226?src=pr=h1) Report > Merging [#4226](https://codecov.io/gh/apache/incubator-airflow/pull/4226?src=pr=desc) into [master](https://codecov.io/gh/apache/incubator-airflow/commit/0f545d74b8d78cd58186515adc4e430b25b4a741?src=pr=desc) will **not change** coverage. > The diff coverage is `n/a`. [](https://codecov.io/gh/apache/incubator-airflow/pull/4226?src=pr=tree) ```diff @@ Coverage Diff @@ ## master#4226 +/- ## === Coverage 77.81% 77.81% === Files 201 201 Lines 1634816348 === Hits1272112721 Misses 3627 3627 ``` -- [Continue to review full report at Codecov](https://codecov.io/gh/apache/incubator-airflow/pull/4226?src=pr=continue). > **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta) > `Δ = absolute (impact)`, `ø = not affected`, `? = missing data` > Powered by [Codecov](https://codecov.io/gh/apache/incubator-airflow/pull/4226?src=pr=footer). Last update [0f545d7...f0f79d3](https://codecov.io/gh/apache/incubator-airflow/pull/4226?src=pr=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments). This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] codecov-io edited a comment on issue #4226: docs: Remove quotes from domain in Google Oauth
codecov-io edited a comment on issue #4226: docs: Remove quotes from domain in Google Oauth URL: https://github.com/apache/incubator-airflow/pull/4226#issuecomment-441109306 # [Codecov](https://codecov.io/gh/apache/incubator-airflow/pull/4226?src=pr=h1) Report > Merging [#4226](https://codecov.io/gh/apache/incubator-airflow/pull/4226?src=pr=desc) into [master](https://codecov.io/gh/apache/incubator-airflow/commit/0f545d74b8d78cd58186515adc4e430b25b4a741?src=pr=desc) will **not change** coverage. > The diff coverage is `n/a`. [](https://codecov.io/gh/apache/incubator-airflow/pull/4226?src=pr=tree) ```diff @@ Coverage Diff @@ ## master#4226 +/- ## === Coverage 77.81% 77.81% === Files 201 201 Lines 1634816348 === Hits1272112721 Misses 3627 3627 ``` -- [Continue to review full report at Codecov](https://codecov.io/gh/apache/incubator-airflow/pull/4226?src=pr=continue). > **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta) > `Δ = absolute (impact)`, `ø = not affected`, `? = missing data` > Powered by [Codecov](https://codecov.io/gh/apache/incubator-airflow/pull/4226?src=pr=footer). Last update [0f545d7...f0f79d3](https://codecov.io/gh/apache/incubator-airflow/pull/4226?src=pr=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments). This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] codecov-io commented on issue #4226: docs: Remove quotes from domain in Google Oauth
codecov-io commented on issue #4226: docs: Remove quotes from domain in Google Oauth URL: https://github.com/apache/incubator-airflow/pull/4226#issuecomment-441109306 # [Codecov](https://codecov.io/gh/apache/incubator-airflow/pull/4226?src=pr=h1) Report > Merging [#4226](https://codecov.io/gh/apache/incubator-airflow/pull/4226?src=pr=desc) into [master](https://codecov.io/gh/apache/incubator-airflow/commit/0f545d74b8d78cd58186515adc4e430b25b4a741?src=pr=desc) will **not change** coverage. > The diff coverage is `n/a`. [](https://codecov.io/gh/apache/incubator-airflow/pull/4226?src=pr=tree) ```diff @@ Coverage Diff @@ ## master#4226 +/- ## === Coverage 77.81% 77.81% === Files 201 201 Lines 1634816348 === Hits1272112721 Misses 3627 3627 ``` -- [Continue to review full report at Codecov](https://codecov.io/gh/apache/incubator-airflow/pull/4226?src=pr=continue). > **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta) > `Δ = absolute (impact)`, `ø = not affected`, `? = missing data` > Powered by [Codecov](https://codecov.io/gh/apache/incubator-airflow/pull/4226?src=pr=footer). Last update [0f545d7...f0f79d3](https://codecov.io/gh/apache/incubator-airflow/pull/4226?src=pr=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments). This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] noqcks opened a new pull request #4226: Remove quotes from domains in Google Oauth
noqcks opened a new pull request #4226: Remove quotes from domains in Google Oauth URL: https://github.com/apache/incubator-airflow/pull/4226 The documentation example that uses quotes around the `domain` in Google Oauth authentication currently doesn't work. Using `domain` with quotes around domains results in "You don't seem to have access. Please contact your administrator" when logging in. Removing the quotes results in the ability to login with the `domain`. Also mentioned in this Stackoverflow: https://stackoverflow.com/a/52528091/10638329 This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] codecov-io commented on issue #4225: [AIRFLOW-3383] Rotate fernet keys.
codecov-io commented on issue #4225: [AIRFLOW-3383] Rotate fernet keys. URL: https://github.com/apache/incubator-airflow/pull/4225#issuecomment-441103479 # [Codecov](https://codecov.io/gh/apache/incubator-airflow/pull/4225?src=pr=h1) Report > Merging [#4225](https://codecov.io/gh/apache/incubator-airflow/pull/4225?src=pr=desc) into [master](https://codecov.io/gh/apache/incubator-airflow/commit/0f545d74b8d78cd58186515adc4e430b25b4a741?src=pr=desc) will **increase** coverage by `<.01%`. > The diff coverage is `62.5%`. [](https://codecov.io/gh/apache/incubator-airflow/pull/4225?src=pr=tree) ```diff @@Coverage Diff @@ ## master#4225 +/- ## == + Coverage 77.81% 77.81% +<.01% == Files 201 201 Lines 1634816362 +14 == + Hits1272112732 +11 - Misses 3627 3630 +3 ``` | [Impacted Files](https://codecov.io/gh/apache/incubator-airflow/pull/4225?src=pr=tree) | Coverage Δ | | |---|---|---| | [airflow/bin/cli.py](https://codecov.io/gh/apache/incubator-airflow/pull/4225/diff?src=pr=tree#diff-YWlyZmxvdy9iaW4vY2xpLnB5) | `64.25% <16.66%> (-0.35%)` | :arrow_down: | | [airflow/models.py](https://codecov.io/gh/apache/incubator-airflow/pull/4225/diff?src=pr=tree#diff-YWlyZmxvdy9tb2RlbHMucHk=) | `92.44% <90%> (+0.1%)` | :arrow_up: | -- [Continue to review full report at Codecov](https://codecov.io/gh/apache/incubator-airflow/pull/4225?src=pr=continue). > **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta) > `Δ = absolute (impact)`, `ø = not affected`, `? = missing data` > Powered by [Codecov](https://codecov.io/gh/apache/incubator-airflow/pull/4225?src=pr=footer). Last update [0f545d7...fd6cc58](https://codecov.io/gh/apache/incubator-airflow/pull/4225?src=pr=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments). This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Created] (AIRFLOW-3383) Simplify fernet key rotation
Josh Carp created AIRFLOW-3383: -- Summary: Simplify fernet key rotation Key: AIRFLOW-3383 URL: https://issues.apache.org/jira/browse/AIRFLOW-3383 Project: Apache Airflow Issue Type: Improvement Reporter: Josh Carp As far as I can tell, it's not straightforward to rotate the fernet key for encrypted passwords and extras. A user would have to generate a new key, restart airflow, and manually re-enter each value to be encrypted via the web interface. It should be possible to specify multiple fernet keys at once, and to easily re-encrypt values with a new key. The cryptography package provides a MultiFernet class with a rotate method that handles this use case, so I wrote up a patch that uses MultiFernet to support multiple keys and rotation via the command line. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (AIRFLOW-3383) Simplify fernet key rotation
[
https://issues.apache.org/jira/browse/AIRFLOW-3383?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16696178#comment-16696178
]
ASF GitHub Bot commented on AIRFLOW-3383:
-
jmcarp opened a new pull request #4225: [AIRFLOW-3383] Rotate fernet keys.
URL: https://github.com/apache/incubator-airflow/pull/4225
Make sure you have checked _all_ steps below.
### Jira
- [x] My PR addresses the following [Airflow
Jira](https://issues.apache.org/jira/browse/AIRFLOW/) issues and references
them in the PR title. For example, "\[AIRFLOW-XXX\] My Airflow PR"
- https://issues.apache.org/jira/browse/AIRFLOW-3383
- In case you are fixing a typo in the documentation you can prepend your
commit with \[AIRFLOW-XXX\], code changes always need a Jira issue.
### Description
- [x] Here are some details about my PR, including screenshots of any UI
changes:
As far as I can tell, it's not straightforward to rotate the fernet key for
encrypted passwords and extras. A user would have to generate a new key,
restart airflow, and manually re-enter each value to be encrypted via the web
interface. It should be possible to specify multiple fernet keys at once, and
to easily re-encrypt values with a new key. The cryptography package provides a
MultiFernet class with a rotate method that handles this use case, so I wrote
up a patch that uses MultiFernet to support multiple keys and rotation via the
command line.
With this approach, we can rotate keys by adding a new key at the start of
the FERNET_KEYS config variable, then running the rotate_credentials command
from the command line. If the approach makes sense, I'll write up some
documentation.
### Tests
- [x] My PR adds the following unit tests __OR__ does not need testing for
this extremely good reason:
### Commits
- [x] My commits all reference Jira issues in their subject lines, and I
have squashed multiple commits if they address the same issue. In addition, my
commits follow the guidelines from "[How to write a good git commit
message](http://chris.beams.io/posts/git-commit/)":
1. Subject is separated from body by a blank line
1. Subject is limited to 50 characters (not including Jira issue reference)
1. Subject does not end with a period
1. Subject uses the imperative mood ("add", not "adding")
1. Body wraps at 72 characters
1. Body explains "what" and "why", not "how"
### Documentation
- [ ] In case of new functionality, my PR adds documentation that describes
how to use it.
- When adding new operators/hooks/sensors, the autoclass documentation
generation needs to be added.
### Code Quality
- [x] Passes `flake8`
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
> Simplify fernet key rotation
>
>
> Key: AIRFLOW-3383
> URL: https://issues.apache.org/jira/browse/AIRFLOW-3383
> Project: Apache Airflow
> Issue Type: Improvement
>Reporter: Josh Carp
>Priority: Minor
>
> As far as I can tell, it's not straightforward to rotate the fernet key for
> encrypted passwords and extras. A user would have to generate a new key,
> restart airflow, and manually re-enter each value to be encrypted via the web
> interface. It should be possible to specify multiple fernet keys at once, and
> to easily re-encrypt values with a new key. The cryptography package provides
> a MultiFernet class with a rotate method that handles this use case, so I
> wrote up a patch that uses MultiFernet to support multiple keys and rotation
> via the command line.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[GitHub] jmcarp opened a new pull request #4225: [AIRFLOW-3383] Rotate fernet keys.
jmcarp opened a new pull request #4225: [AIRFLOW-3383] Rotate fernet keys.
URL: https://github.com/apache/incubator-airflow/pull/4225
Make sure you have checked _all_ steps below.
### Jira
- [x] My PR addresses the following [Airflow
Jira](https://issues.apache.org/jira/browse/AIRFLOW/) issues and references
them in the PR title. For example, "\[AIRFLOW-XXX\] My Airflow PR"
- https://issues.apache.org/jira/browse/AIRFLOW-3383
- In case you are fixing a typo in the documentation you can prepend your
commit with \[AIRFLOW-XXX\], code changes always need a Jira issue.
### Description
- [x] Here are some details about my PR, including screenshots of any UI
changes:
As far as I can tell, it's not straightforward to rotate the fernet key for
encrypted passwords and extras. A user would have to generate a new key,
restart airflow, and manually re-enter each value to be encrypted via the web
interface. It should be possible to specify multiple fernet keys at once, and
to easily re-encrypt values with a new key. The cryptography package provides a
MultiFernet class with a rotate method that handles this use case, so I wrote
up a patch that uses MultiFernet to support multiple keys and rotation via the
command line.
With this approach, we can rotate keys by adding a new key at the start of
the FERNET_KEYS config variable, then running the rotate_credentials command
from the command line. If the approach makes sense, I'll write up some
documentation.
### Tests
- [x] My PR adds the following unit tests __OR__ does not need testing for
this extremely good reason:
### Commits
- [x] My commits all reference Jira issues in their subject lines, and I
have squashed multiple commits if they address the same issue. In addition, my
commits follow the guidelines from "[How to write a good git commit
message](http://chris.beams.io/posts/git-commit/)":
1. Subject is separated from body by a blank line
1. Subject is limited to 50 characters (not including Jira issue reference)
1. Subject does not end with a period
1. Subject uses the imperative mood ("add", not "adding")
1. Body wraps at 72 characters
1. Body explains "what" and "why", not "how"
### Documentation
- [ ] In case of new functionality, my PR adds documentation that describes
how to use it.
- When adding new operators/hooks/sensors, the autoclass documentation
generation needs to be added.
### Code Quality
- [x] Passes `flake8`
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] codecov-io edited a comment on issue #4224: [AIRFLOW-XXX] Make sql param as required in BigQueryOperator
codecov-io edited a comment on issue #4224: [AIRFLOW-XXX] Make sql param as required in BigQueryOperator URL: https://github.com/apache/incubator-airflow/pull/4224#issuecomment-441080295 # [Codecov](https://codecov.io/gh/apache/incubator-airflow/pull/4224?src=pr=h1) Report > Merging [#4224](https://codecov.io/gh/apache/incubator-airflow/pull/4224?src=pr=desc) into [master](https://codecov.io/gh/apache/incubator-airflow/commit/0f545d74b8d78cd58186515adc4e430b25b4a741?src=pr=desc) will **not change** coverage. > The diff coverage is `n/a`. [](https://codecov.io/gh/apache/incubator-airflow/pull/4224?src=pr=tree) ```diff @@ Coverage Diff @@ ## master#4224 +/- ## === Coverage 77.81% 77.81% === Files 201 201 Lines 1634816348 === Hits1272112721 Misses 3627 3627 ``` -- [Continue to review full report at Codecov](https://codecov.io/gh/apache/incubator-airflow/pull/4224?src=pr=continue). > **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta) > `Δ = absolute (impact)`, `ø = not affected`, `? = missing data` > Powered by [Codecov](https://codecov.io/gh/apache/incubator-airflow/pull/4224?src=pr=footer). Last update [0f545d7...a9017bd](https://codecov.io/gh/apache/incubator-airflow/pull/4224?src=pr=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments). This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] codecov-io commented on issue #4224: [AIRFLOW-XXX] Make sql param as required in BigQueryOperator
codecov-io commented on issue #4224: [AIRFLOW-XXX] Make sql param as required in BigQueryOperator URL: https://github.com/apache/incubator-airflow/pull/4224#issuecomment-441080295 # [Codecov](https://codecov.io/gh/apache/incubator-airflow/pull/4224?src=pr=h1) Report > Merging [#4224](https://codecov.io/gh/apache/incubator-airflow/pull/4224?src=pr=desc) into [master](https://codecov.io/gh/apache/incubator-airflow/commit/0f545d74b8d78cd58186515adc4e430b25b4a741?src=pr=desc) will **not change** coverage. > The diff coverage is `n/a`. [](https://codecov.io/gh/apache/incubator-airflow/pull/4224?src=pr=tree) ```diff @@ Coverage Diff @@ ## master#4224 +/- ## === Coverage 77.81% 77.81% === Files 201 201 Lines 1634816348 === Hits1272112721 Misses 3627 3627 ``` -- [Continue to review full report at Codecov](https://codecov.io/gh/apache/incubator-airflow/pull/4224?src=pr=continue). > **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta) > `Δ = absolute (impact)`, `ø = not affected`, `? = missing data` > Powered by [Codecov](https://codecov.io/gh/apache/incubator-airflow/pull/4224?src=pr=footer). Last update [0f545d7...a9017bd](https://codecov.io/gh/apache/incubator-airflow/pull/4224?src=pr=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments). This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] XD-DENG commented on issue #4223: [AIRFLOW-XXX] Remove unnecessary usage of "# noqa" in airflow/bin/cli.py
XD-DENG commented on issue #4223: [AIRFLOW-XXX] Remove unnecessary usage of "# noqa" in airflow/bin/cli.py URL: https://github.com/apache/incubator-airflow/pull/4223#issuecomment-441079622 One test (out of 9) failed due to transient error (https://travis-ci.org/apache/incubator-airflow/builds/458472441). This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] kaxil commented on issue #4224: [AIRFLOW-XXX] Make sql param as required in BigQueryOperator
kaxil commented on issue #4224: [AIRFLOW-XXX] Make sql param as required in BigQueryOperator URL: https://github.com/apache/incubator-airflow/pull/4224#issuecomment-441073808 Instead of allowing None and then adding condition, we should remove it. This was made earlier when we had `bql` and `sql` and wanted to deprecate `bql` This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] kaxil opened a new pull request #4224: [AIRFLOW-XXX] Make sql param as required in BigQueryOperator
kaxil opened a new pull request #4224: [AIRFLOW-XXX] Make sql param as required
in BigQueryOperator
URL: https://github.com/apache/incubator-airflow/pull/4224
Make sure you have checked _all_ steps below.
### Jira
- [x] My PR addresses the following [Airflow
Jira](https://issues.apache.org/jira/browse/AIRFLOW/) issues and references
them in the PR title. For example, "\[AIRFLOW-XXX\] My Airflow PR"
- https://issues.apache.org/jira/browse/AIRFLOW-XXX
- In case you are fixing a typo in the documentation you can prepend your
commit with \[AIRFLOW-XXX\], code changes always need a Jira issue.
### Description
- [x] Here are some details about my PR, including screenshots of any UI
changes:
### Tests
- [x] My PR adds the following unit tests __OR__ does not need testing for
this extremely good reason:
### Commits
- [x] My commits all reference Jira issues in their subject lines, and I
have squashed multiple commits if they address the same issue. In addition, my
commits follow the guidelines from "[How to write a good git commit
message](http://chris.beams.io/posts/git-commit/)":
1. Subject is separated from body by a blank line
1. Subject is limited to 50 characters (not including Jira issue reference)
1. Subject does not end with a period
1. Subject uses the imperative mood ("add", not "adding")
1. Body wraps at 72 characters
1. Body explains "what" and "why", not "how"
### Documentation
- [x] In case of new functionality, my PR adds documentation that describes
how to use it.
- When adding new operators/hooks/sensors, the autoclass documentation
generation needs to be added.
### Code Quality
- [x] Passes `flake8`
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] XD-DENG opened a new pull request #4223: [AIRFLOW-XXX] Remove unnecessary usage of "# noqa" in airflow/bin/cli.py
XD-DENG opened a new pull request #4223: [AIRFLOW-XXX] Remove unnecessary usage of "# noqa" in airflow/bin/cli.py URL: https://github.com/apache/incubator-airflow/pull/4223 ### Jira Simple change without JIRA needed. ### Description - [x] Here are some details about my PR, including screenshots of any UI changes: These `# noqa` were added in https://github.com/apache/incubator-airflow/commit/fbac2bff2beb804e82cc9aea8fc3f42db5a3595e#diff-1c2404a3a60f829127232842250ff406 two years ago. Possibly they were added for some good reasons, or some restriction at that time point. But now I don't think we still need them. ### Tests Have passed tests in my own fork. ### Code Quality - [x] Passes `flake8` This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Resolved] (AIRFLOW-3263) CLI 'run' method sometimes exits with error when there is a race on killing airflow job
[
https://issues.apache.org/jira/browse/AIRFLOW-3263?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Kaxil Naik resolved AIRFLOW-3263.
-
Resolution: Fixed
Resolved by https://github.com/apache/incubator-airflow/pull/4108
> CLI 'run' method sometimes exits with error when there is a race on killing
> airflow job
> ---
>
> Key: AIRFLOW-3263
> URL: https://issues.apache.org/jira/browse/AIRFLOW-3263
> Project: Apache Airflow
> Issue Type: Bug
> Components: cli
>Reporter: Jarek Potiuk
>Assignee: Jarek Potiuk
>Priority: Minor
>
> Sometimes when you run tasks from command line you get exit code = 1 due to
> race condition (job runner tries to get process group from the process that
> has already been terminated in the meantime)
> This results in such exception:
> Traceback (most recent call last):
> File "/Users/potiuk/.virtualenvs/incubator-airflow/bin/airflow", line 7, in
>
> exec(compile(f.read(), __file__, 'exec'))
> File
> "/Users/potiuk/code/google-airflow-breeze/polidea/incubator-airflow/airflow/bin/airflow",
> line 32, in
> args.func(args)
> File
> "/Users/potiuk/code/google-airflow-breeze/polidea/incubator-airflow/airflow/utils/cli.py",
> line 74, in wrapper
> return f(*args, **kwargs)
> File
> "/Users/potiuk/code/google-airflow-breeze/polidea/incubator-airflow/airflow/bin/cli.py",
> line 536, in run
> _run(args, dag, ti)
> File
> "/Users/potiuk/code/google-airflow-breeze/polidea/incubator-airflow/airflow/bin/cli.py",
> line 447, in _run
> run_job.run()
> File
> "/Users/potiuk/code/google-airflow-breeze/polidea/incubator-airflow/airflow/jobs.py",
> line 203, in run
> self._execute()
> File
> "/Users/potiuk/code/google-airflow-breeze/polidea/incubator-airflow/airflow/jobs.py",
> line 2666, in _execute
> self.on_kill()
> File
> "/Users/potiuk/code/google-airflow-breeze/polidea/incubator-airflow/airflow/jobs.py",
> line 2669, in on_kill
> self.task_runner.terminate()
> File
> "/Users/potiuk/code/google-airflow-breeze/polidea/incubator-airflow/airflow/task/task_runner/standard_task_runner.py",
> line 41, in terminate
> reap_process_group(self.process.pid, self.log)
> File
> "/Users/potiuk/code/google-airflow-breeze/polidea/incubator-airflow/airflow/utils/helpers.py",
> line 237, in reap_process_group
> log.info("Sending %s to GPID %s", sig, os.getpgid(pid))
> OSError: [Errno 3] No such process
>
> I am going to provide a fix shortly.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (AIRFLOW-3263) CLI 'run' method sometimes exits with error when there is a race on killing airflow job
[
https://issues.apache.org/jira/browse/AIRFLOW-3263?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Kaxil Naik updated AIRFLOW-3263:
Fix Version/s: 2.0.0
> CLI 'run' method sometimes exits with error when there is a race on killing
> airflow job
> ---
>
> Key: AIRFLOW-3263
> URL: https://issues.apache.org/jira/browse/AIRFLOW-3263
> Project: Apache Airflow
> Issue Type: Bug
> Components: cli
>Reporter: Jarek Potiuk
>Assignee: Jarek Potiuk
>Priority: Minor
> Fix For: 2.0.0
>
>
> Sometimes when you run tasks from command line you get exit code = 1 due to
> race condition (job runner tries to get process group from the process that
> has already been terminated in the meantime)
> This results in such exception:
> Traceback (most recent call last):
> File "/Users/potiuk/.virtualenvs/incubator-airflow/bin/airflow", line 7, in
>
> exec(compile(f.read(), __file__, 'exec'))
> File
> "/Users/potiuk/code/google-airflow-breeze/polidea/incubator-airflow/airflow/bin/airflow",
> line 32, in
> args.func(args)
> File
> "/Users/potiuk/code/google-airflow-breeze/polidea/incubator-airflow/airflow/utils/cli.py",
> line 74, in wrapper
> return f(*args, **kwargs)
> File
> "/Users/potiuk/code/google-airflow-breeze/polidea/incubator-airflow/airflow/bin/cli.py",
> line 536, in run
> _run(args, dag, ti)
> File
> "/Users/potiuk/code/google-airflow-breeze/polidea/incubator-airflow/airflow/bin/cli.py",
> line 447, in _run
> run_job.run()
> File
> "/Users/potiuk/code/google-airflow-breeze/polidea/incubator-airflow/airflow/jobs.py",
> line 203, in run
> self._execute()
> File
> "/Users/potiuk/code/google-airflow-breeze/polidea/incubator-airflow/airflow/jobs.py",
> line 2666, in _execute
> self.on_kill()
> File
> "/Users/potiuk/code/google-airflow-breeze/polidea/incubator-airflow/airflow/jobs.py",
> line 2669, in on_kill
> self.task_runner.terminate()
> File
> "/Users/potiuk/code/google-airflow-breeze/polidea/incubator-airflow/airflow/task/task_runner/standard_task_runner.py",
> line 41, in terminate
> reap_process_group(self.process.pid, self.log)
> File
> "/Users/potiuk/code/google-airflow-breeze/polidea/incubator-airflow/airflow/utils/helpers.py",
> line 237, in reap_process_group
> log.info("Sending %s to GPID %s", sig, os.getpgid(pid))
> OSError: [Errno 3] No such process
>
> I am going to provide a fix shortly.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (AIRFLOW-3263) CLI 'run' method sometimes exits with error when there is a race on killing airflow job
[
https://issues.apache.org/jira/browse/AIRFLOW-3263?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16695841#comment-16695841
]
ASF GitHub Bot commented on AIRFLOW-3263:
-
kaxil closed pull request #4108: [AIRFLOW-3263] Ignore exception when 'run'
tries to kill already killed job
URL: https://github.com/apache/incubator-airflow/pull/4108
This is a PR merged from a forked repository.
As GitHub hides the original diff on merge, it is displayed below for
the sake of provenance:
As this is a foreign pull request (from a fork), the diff is supplied
below (as it won't show otherwise due to GitHub magic):
diff --git a/airflow/utils/helpers.py b/airflow/utils/helpers.py
index 3ebd5f75c6..328147c1cf 100644
--- a/airflow/utils/helpers.py
+++ b/airflow/utils/helpers.py
@@ -22,6 +22,8 @@
from __future__ import print_function
from __future__ import unicode_literals
+import errno
+
import psutil
from builtins import input
@@ -234,7 +236,15 @@ def on_terminate(p):
children = parent.children(recursive=True)
children.append(parent)
-log.info("Sending %s to GPID %s", sig, os.getpgid(pid))
+try:
+pg = os.getpgid(pid)
+except OSError as err:
+# Skip if not such process - we experience a race and it just
terminated
+if err.errno == errno.ESRCH:
+return
+raise
+
+log.info("Sending %s to GPID %s", sig, pg)
os.killpg(os.getpgid(pid), sig)
gone, alive = psutil.wait_procs(children, timeout=timeout,
callback=on_terminate)
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
> CLI 'run' method sometimes exits with error when there is a race on killing
> airflow job
> ---
>
> Key: AIRFLOW-3263
> URL: https://issues.apache.org/jira/browse/AIRFLOW-3263
> Project: Apache Airflow
> Issue Type: Bug
> Components: cli
>Reporter: Jarek Potiuk
>Assignee: Jarek Potiuk
>Priority: Minor
>
> Sometimes when you run tasks from command line you get exit code = 1 due to
> race condition (job runner tries to get process group from the process that
> has already been terminated in the meantime)
> This results in such exception:
> Traceback (most recent call last):
> File "/Users/potiuk/.virtualenvs/incubator-airflow/bin/airflow", line 7, in
>
> exec(compile(f.read(), __file__, 'exec'))
> File
> "/Users/potiuk/code/google-airflow-breeze/polidea/incubator-airflow/airflow/bin/airflow",
> line 32, in
> args.func(args)
> File
> "/Users/potiuk/code/google-airflow-breeze/polidea/incubator-airflow/airflow/utils/cli.py",
> line 74, in wrapper
> return f(*args, **kwargs)
> File
> "/Users/potiuk/code/google-airflow-breeze/polidea/incubator-airflow/airflow/bin/cli.py",
> line 536, in run
> _run(args, dag, ti)
> File
> "/Users/potiuk/code/google-airflow-breeze/polidea/incubator-airflow/airflow/bin/cli.py",
> line 447, in _run
> run_job.run()
> File
> "/Users/potiuk/code/google-airflow-breeze/polidea/incubator-airflow/airflow/jobs.py",
> line 203, in run
> self._execute()
> File
> "/Users/potiuk/code/google-airflow-breeze/polidea/incubator-airflow/airflow/jobs.py",
> line 2666, in _execute
> self.on_kill()
> File
> "/Users/potiuk/code/google-airflow-breeze/polidea/incubator-airflow/airflow/jobs.py",
> line 2669, in on_kill
> self.task_runner.terminate()
> File
> "/Users/potiuk/code/google-airflow-breeze/polidea/incubator-airflow/airflow/task/task_runner/standard_task_runner.py",
> line 41, in terminate
> reap_process_group(self.process.pid, self.log)
> File
> "/Users/potiuk/code/google-airflow-breeze/polidea/incubator-airflow/airflow/utils/helpers.py",
> line 237, in reap_process_group
> log.info("Sending %s to GPID %s", sig, os.getpgid(pid))
> OSError: [Errno 3] No such process
>
> I am going to provide a fix shortly.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[GitHub] kaxil closed pull request #4108: [AIRFLOW-3263] Ignore exception when 'run' tries to kill already killed job
kaxil closed pull request #4108: [AIRFLOW-3263] Ignore exception when 'run'
tries to kill already killed job
URL: https://github.com/apache/incubator-airflow/pull/4108
This is a PR merged from a forked repository.
As GitHub hides the original diff on merge, it is displayed below for
the sake of provenance:
As this is a foreign pull request (from a fork), the diff is supplied
below (as it won't show otherwise due to GitHub magic):
diff --git a/airflow/utils/helpers.py b/airflow/utils/helpers.py
index 3ebd5f75c6..328147c1cf 100644
--- a/airflow/utils/helpers.py
+++ b/airflow/utils/helpers.py
@@ -22,6 +22,8 @@
from __future__ import print_function
from __future__ import unicode_literals
+import errno
+
import psutil
from builtins import input
@@ -234,7 +236,15 @@ def on_terminate(p):
children = parent.children(recursive=True)
children.append(parent)
-log.info("Sending %s to GPID %s", sig, os.getpgid(pid))
+try:
+pg = os.getpgid(pid)
+except OSError as err:
+# Skip if not such process - we experience a race and it just
terminated
+if err.errno == errno.ESRCH:
+return
+raise
+
+log.info("Sending %s to GPID %s", sig, pg)
os.killpg(os.getpgid(pid), sig)
gone, alive = psutil.wait_procs(children, timeout=timeout,
callback=on_terminate)
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] kaxil commented on issue #2635: [AIRFLOW-1561] Fix scheduler to pick up example DAGs without other DAGs
kaxil commented on issue #2635: [AIRFLOW-1561] Fix scheduler to pick up example DAGs without other DAGs URL: https://github.com/apache/incubator-airflow/pull/2635#issuecomment-441010451 Looks logical, can you rebase the master please. Once that is done I will test this and if everything works file, will merge This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Resolved] (AIRFLOW-3382) Fix incorrect docstring in DatastoreHook
[ https://issues.apache.org/jira/browse/AIRFLOW-3382?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kaxil Naik resolved AIRFLOW-3382. - Resolution: Fixed Fix Version/s: 2.0.0 Resolved by https://github.com/apache/incubator-airflow/pull/4222 > Fix incorrect docstring in DatastoreHook > > > Key: AIRFLOW-3382 > URL: https://issues.apache.org/jira/browse/AIRFLOW-3382 > Project: Apache Airflow > Issue Type: Bug >Reporter: Ryan Yuan >Assignee: Ryan Yuan >Priority: Minor > Fix For: 2.0.0 > > > One of the docstrings in DatastoreHook incorrectly states 'Google Cloud > Storage' instead of "Google Cloud Datastore" -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (AIRFLOW-3382) Fix incorrect docstring in DatastoreHook
[ https://issues.apache.org/jira/browse/AIRFLOW-3382?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16695821#comment-16695821 ] ASF GitHub Bot commented on AIRFLOW-3382: - kaxil closed pull request #4222: [AIRFLOW-3382] Fix incorrect docstring in DatastoreHook URL: https://github.com/apache/incubator-airflow/pull/4222 This is a PR merged from a forked repository. As GitHub hides the original diff on merge, it is displayed below for the sake of provenance: As this is a foreign pull request (from a fork), the diff is supplied below (as it won't show otherwise due to GitHub magic): diff --git a/airflow/contrib/hooks/datastore_hook.py b/airflow/contrib/hooks/datastore_hook.py index b8c3ca00a0..c055e347d6 100644 --- a/airflow/contrib/hooks/datastore_hook.py +++ b/airflow/contrib/hooks/datastore_hook.py @@ -41,7 +41,7 @@ def __init__(self, def get_conn(self, version='v1'): """ -Returns a Google Cloud Storage service object. +Returns a Google Cloud Datastore service object. """ http_authorized = self._authorize() return build( This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org > Fix incorrect docstring in DatastoreHook > > > Key: AIRFLOW-3382 > URL: https://issues.apache.org/jira/browse/AIRFLOW-3382 > Project: Apache Airflow > Issue Type: Bug >Reporter: Ryan Yuan >Assignee: Ryan Yuan >Priority: Minor > > One of the docstrings in DatastoreHook incorrectly states 'Google Cloud > Storage' instead of "Google Cloud Datastore" -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[GitHub] kaxil closed pull request #4222: [AIRFLOW-3382] Fix incorrect docstring in DatastoreHook
kaxil closed pull request #4222: [AIRFLOW-3382] Fix incorrect docstring in DatastoreHook URL: https://github.com/apache/incubator-airflow/pull/4222 This is a PR merged from a forked repository. As GitHub hides the original diff on merge, it is displayed below for the sake of provenance: As this is a foreign pull request (from a fork), the diff is supplied below (as it won't show otherwise due to GitHub magic): diff --git a/airflow/contrib/hooks/datastore_hook.py b/airflow/contrib/hooks/datastore_hook.py index b8c3ca00a0..c055e347d6 100644 --- a/airflow/contrib/hooks/datastore_hook.py +++ b/airflow/contrib/hooks/datastore_hook.py @@ -41,7 +41,7 @@ def __init__(self, def get_conn(self, version='v1'): """ -Returns a Google Cloud Storage service object. +Returns a Google Cloud Datastore service object. """ http_authorized = self._authorize() return build( This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] codecov-io edited a comment on issue #4197: [AIRFLOW-3348] update run statistics on dag refresh
codecov-io edited a comment on issue #4197: [AIRFLOW-3348] update run statistics on dag refresh URL: https://github.com/apache/incubator-airflow/pull/4197#issuecomment-439221145 # [Codecov](https://codecov.io/gh/apache/incubator-airflow/pull/4197?src=pr=h1) Report > Merging [#4197](https://codecov.io/gh/apache/incubator-airflow/pull/4197?src=pr=desc) into [master](https://codecov.io/gh/apache/incubator-airflow/commit/8668ef869d3d844dac746ec88609d3710a1264ab?src=pr=desc) will **increase** coverage by `0.12%`. > The diff coverage is `100%`. [](https://codecov.io/gh/apache/incubator-airflow/pull/4197?src=pr=tree) ```diff @@Coverage Diff @@ ## master#4197 +/- ## == + Coverage 77.69% 77.82% +0.12% == Files 199 201 +2 Lines 1630916343 +34 == + Hits1267212719 +47 + Misses 3637 3624 -13 ``` | [Impacted Files](https://codecov.io/gh/apache/incubator-airflow/pull/4197?src=pr=tree) | Coverage Δ | | |---|---|---| | [airflow/www/views.py](https://codecov.io/gh/apache/incubator-airflow/pull/4197/diff?src=pr=tree#diff-YWlyZmxvdy93d3cvdmlld3MucHk=) | `69.37% <100%> (+0.01%)` | :arrow_up: | | [airflow/www\_rbac/views.py](https://codecov.io/gh/apache/incubator-airflow/pull/4197/diff?src=pr=tree#diff-YWlyZmxvdy93d3dfcmJhYy92aWV3cy5weQ==) | `72.4% <100%> (+0.08%)` | :arrow_up: | | [airflow/models.py](https://codecov.io/gh/apache/incubator-airflow/pull/4197/diff?src=pr=tree#diff-YWlyZmxvdy9tb2RlbHMucHk=) | `92.29% <0%> (-0.04%)` | :arrow_down: | | [airflow/api/common/experimental/trigger\_dag.py](https://codecov.io/gh/apache/incubator-airflow/pull/4197/diff?src=pr=tree#diff-YWlyZmxvdy9hcGkvY29tbW9uL2V4cGVyaW1lbnRhbC90cmlnZ2VyX2RhZy5weQ==) | `100% <0%> (ø)` | :arrow_up: | | [airflow/sensors/\_\_init\_\_.py](https://codecov.io/gh/apache/incubator-airflow/pull/4197/diff?src=pr=tree#diff-YWlyZmxvdy9zZW5zb3JzL19faW5pdF9fLnB5) | `100% <0%> (ø)` | | | [airflow/operators/\_\_init\_\_.py](https://codecov.io/gh/apache/incubator-airflow/pull/4197/diff?src=pr=tree#diff-YWlyZmxvdy9vcGVyYXRvcnMvX19pbml0X18ucHk=) | `100% <0%> (ø)` | | | [airflow/operators/python\_operator.py](https://codecov.io/gh/apache/incubator-airflow/pull/4197/diff?src=pr=tree#diff-YWlyZmxvdy9vcGVyYXRvcnMvcHl0aG9uX29wZXJhdG9yLnB5) | `95.09% <0%> (+0.06%)` | :arrow_up: | | [airflow/plugins\_manager.py](https://codecov.io/gh/apache/incubator-airflow/pull/4197/diff?src=pr=tree#diff-YWlyZmxvdy9wbHVnaW5zX21hbmFnZXIucHk=) | `93.1% <0%> (+1.14%)` | :arrow_up: | | [airflow/\_\_init\_\_.py](https://codecov.io/gh/apache/incubator-airflow/pull/4197/diff?src=pr=tree#diff-YWlyZmxvdy9fX2luaXRfXy5weQ==) | `80% <0%> (+5.71%)` | :arrow_up: | | ... and [3 more](https://codecov.io/gh/apache/incubator-airflow/pull/4197/diff?src=pr=tree-more) | | -- [Continue to review full report at Codecov](https://codecov.io/gh/apache/incubator-airflow/pull/4197?src=pr=continue). > **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta) > `Δ = absolute (impact)`, `ø = not affected`, `? = missing data` > Powered by [Codecov](https://codecov.io/gh/apache/incubator-airflow/pull/4197?src=pr=footer). Last update [8668ef8...75319e2](https://codecov.io/gh/apache/incubator-airflow/pull/4197?src=pr=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments). This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] codecov-io edited a comment on issue #4197: [AIRFLOW-3348] update run statistics on dag refresh
codecov-io edited a comment on issue #4197: [AIRFLOW-3348] update run statistics on dag refresh URL: https://github.com/apache/incubator-airflow/pull/4197#issuecomment-439221145 # [Codecov](https://codecov.io/gh/apache/incubator-airflow/pull/4197?src=pr=h1) Report > Merging [#4197](https://codecov.io/gh/apache/incubator-airflow/pull/4197?src=pr=desc) into [master](https://codecov.io/gh/apache/incubator-airflow/commit/8668ef869d3d844dac746ec88609d3710a1264ab?src=pr=desc) will **increase** coverage by `0.12%`. > The diff coverage is `100%`. [](https://codecov.io/gh/apache/incubator-airflow/pull/4197?src=pr=tree) ```diff @@Coverage Diff @@ ## master#4197 +/- ## == + Coverage 77.69% 77.82% +0.12% == Files 199 201 +2 Lines 1630916343 +34 == + Hits1267212719 +47 + Misses 3637 3624 -13 ``` | [Impacted Files](https://codecov.io/gh/apache/incubator-airflow/pull/4197?src=pr=tree) | Coverage Δ | | |---|---|---| | [airflow/www/views.py](https://codecov.io/gh/apache/incubator-airflow/pull/4197/diff?src=pr=tree#diff-YWlyZmxvdy93d3cvdmlld3MucHk=) | `69.37% <100%> (+0.01%)` | :arrow_up: | | [airflow/www\_rbac/views.py](https://codecov.io/gh/apache/incubator-airflow/pull/4197/diff?src=pr=tree#diff-YWlyZmxvdy93d3dfcmJhYy92aWV3cy5weQ==) | `72.4% <100%> (+0.08%)` | :arrow_up: | | [airflow/models.py](https://codecov.io/gh/apache/incubator-airflow/pull/4197/diff?src=pr=tree#diff-YWlyZmxvdy9tb2RlbHMucHk=) | `92.29% <0%> (-0.04%)` | :arrow_down: | | [airflow/api/common/experimental/trigger\_dag.py](https://codecov.io/gh/apache/incubator-airflow/pull/4197/diff?src=pr=tree#diff-YWlyZmxvdy9hcGkvY29tbW9uL2V4cGVyaW1lbnRhbC90cmlnZ2VyX2RhZy5weQ==) | `100% <0%> (ø)` | :arrow_up: | | [airflow/sensors/\_\_init\_\_.py](https://codecov.io/gh/apache/incubator-airflow/pull/4197/diff?src=pr=tree#diff-YWlyZmxvdy9zZW5zb3JzL19faW5pdF9fLnB5) | `100% <0%> (ø)` | | | [airflow/operators/\_\_init\_\_.py](https://codecov.io/gh/apache/incubator-airflow/pull/4197/diff?src=pr=tree#diff-YWlyZmxvdy9vcGVyYXRvcnMvX19pbml0X18ucHk=) | `100% <0%> (ø)` | | | [airflow/operators/python\_operator.py](https://codecov.io/gh/apache/incubator-airflow/pull/4197/diff?src=pr=tree#diff-YWlyZmxvdy9vcGVyYXRvcnMvcHl0aG9uX29wZXJhdG9yLnB5) | `95.09% <0%> (+0.06%)` | :arrow_up: | | [airflow/plugins\_manager.py](https://codecov.io/gh/apache/incubator-airflow/pull/4197/diff?src=pr=tree#diff-YWlyZmxvdy9wbHVnaW5zX21hbmFnZXIucHk=) | `93.1% <0%> (+1.14%)` | :arrow_up: | | [airflow/\_\_init\_\_.py](https://codecov.io/gh/apache/incubator-airflow/pull/4197/diff?src=pr=tree#diff-YWlyZmxvdy9fX2luaXRfXy5weQ==) | `80% <0%> (+5.71%)` | :arrow_up: | | ... and [3 more](https://codecov.io/gh/apache/incubator-airflow/pull/4197/diff?src=pr=tree-more) | | -- [Continue to review full report at Codecov](https://codecov.io/gh/apache/incubator-airflow/pull/4197?src=pr=continue). > **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta) > `Δ = absolute (impact)`, `ø = not affected`, `? = missing data` > Powered by [Codecov](https://codecov.io/gh/apache/incubator-airflow/pull/4197?src=pr=footer). Last update [8668ef8...75319e2](https://codecov.io/gh/apache/incubator-airflow/pull/4197?src=pr=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments). This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] mrkm4ntr edited a comment on issue #2635: [AIRFLOW-1561] Fix scheduler to pick up example DAGs without other DAGs
mrkm4ntr edited a comment on issue #2635: [AIRFLOW-1561] Fix scheduler to pick up example DAGs without other DAGs URL: https://github.com/apache/incubator-airflow/pull/2635#issuecomment-440975678 @xnuinside Sure. However I would like to start after the reviewers are assigned. This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] mrkm4ntr commented on issue #2635: [AIRFLOW-1561] Fix scheduler to pick up example DAGs without other DAGs
mrkm4ntr commented on issue #2635: [AIRFLOW-1561] Fix scheduler to pick up example DAGs without other DAGs URL: https://github.com/apache/incubator-airflow/pull/2635#issuecomment-440975678 @xnuinside Sure. However I would like to start after the reviewers react. This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
