[GitHub] codecov-io edited a comment on issue #4364: [AIRFLOW-3550] Standardize GKE hook.
codecov-io edited a comment on issue #4364: [AIRFLOW-3550] Standardize GKE hook. URL: https://github.com/apache/incubator-airflow/pull/4364#issuecomment-449657055 # [Codecov](https://codecov.io/gh/apache/incubator-airflow/pull/4364?src=pr=h1) Report > Merging [#4364](https://codecov.io/gh/apache/incubator-airflow/pull/4364?src=pr=desc) into [master](https://codecov.io/gh/apache/incubator-airflow/commit/7a6acbf5b343e4a6895d1cc8af75ecc02b4fd0e8?src=pr=desc) will **not change** coverage. > The diff coverage is `n/a`. [](https://codecov.io/gh/apache/incubator-airflow/pull/4364?src=pr=tree) ```diff @@ Coverage Diff @@ ## master#4364 +/- ## === Coverage 78.12% 78.12% === Files 202 202 Lines 1648316483 === Hits1287812878 Misses 3605 3605 ``` -- [Continue to review full report at Codecov](https://codecov.io/gh/apache/incubator-airflow/pull/4364?src=pr=continue). > **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta) > `Δ = absolute (impact)`, `ø = not affected`, `? = missing data` > Powered by [Codecov](https://codecov.io/gh/apache/incubator-airflow/pull/4364?src=pr=footer). Last update [7a6acbf...4b3ef97](https://codecov.io/gh/apache/incubator-airflow/pull/4364?src=pr=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments). This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] codecov-io commented on issue #4364: [AIRFLOW-3550] Standardize GKE hook.
codecov-io commented on issue #4364: [AIRFLOW-3550] Standardize GKE hook. URL: https://github.com/apache/incubator-airflow/pull/4364#issuecomment-449657055 # [Codecov](https://codecov.io/gh/apache/incubator-airflow/pull/4364?src=pr=h1) Report > Merging [#4364](https://codecov.io/gh/apache/incubator-airflow/pull/4364?src=pr=desc) into [master](https://codecov.io/gh/apache/incubator-airflow/commit/7a6acbf5b343e4a6895d1cc8af75ecc02b4fd0e8?src=pr=desc) will **not change** coverage. > The diff coverage is `n/a`. [](https://codecov.io/gh/apache/incubator-airflow/pull/4364?src=pr=tree) ```diff @@ Coverage Diff @@ ## master#4364 +/- ## === Coverage 78.12% 78.12% === Files 202 202 Lines 1648316483 === Hits1287812878 Misses 3605 3605 ``` -- [Continue to review full report at Codecov](https://codecov.io/gh/apache/incubator-airflow/pull/4364?src=pr=continue). > **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta) > `Δ = absolute (impact)`, `ø = not affected`, `? = missing data` > Powered by [Codecov](https://codecov.io/gh/apache/incubator-airflow/pull/4364?src=pr=footer). Last update [7a6acbf...4b3ef97](https://codecov.io/gh/apache/incubator-airflow/pull/4364?src=pr=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments). This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] codecov-io edited a comment on issue #4368: AIRFLOW-3561 - improve queries
codecov-io edited a comment on issue #4368: AIRFLOW-3561 - improve queries URL: https://github.com/apache/incubator-airflow/pull/4368#issuecomment-449655224 # [Codecov](https://codecov.io/gh/apache/incubator-airflow/pull/4368?src=pr=h1) Report > Merging [#4368](https://codecov.io/gh/apache/incubator-airflow/pull/4368?src=pr=desc) into [master](https://codecov.io/gh/apache/incubator-airflow/commit/7a6acbf5b343e4a6895d1cc8af75ecc02b4fd0e8?src=pr=desc) will **decrease** coverage by `<.01%`. > The diff coverage is `100%`. [](https://codecov.io/gh/apache/incubator-airflow/pull/4368?src=pr=tree) ```diff @@Coverage Diff @@ ## master#4368 +/- ## == - Coverage 78.12% 78.12% -0.01% == Files 202 202 Lines 1648316490 +7 == + Hits1287812883 +5 - Misses 3605 3607 +2 ``` | [Impacted Files](https://codecov.io/gh/apache/incubator-airflow/pull/4368?src=pr=tree) | Coverage Δ | | |---|---|---| | [airflow/www/views.py](https://codecov.io/gh/apache/incubator-airflow/pull/4368/diff?src=pr=tree#diff-YWlyZmxvdy93d3cvdmlld3MucHk=) | `69.46% <100%> (+0.06%)` | :arrow_up: | | [airflow/models/\_\_init\_\_.py](https://codecov.io/gh/apache/incubator-airflow/pull/4368/diff?src=pr=tree#diff-YWlyZmxvdy9tb2RlbHMvX19pbml0X18ucHk=) | `92.63% <0%> (-0.05%)` | :arrow_down: | -- [Continue to review full report at Codecov](https://codecov.io/gh/apache/incubator-airflow/pull/4368?src=pr=continue). > **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta) > `Δ = absolute (impact)`, `ø = not affected`, `? = missing data` > Powered by [Codecov](https://codecov.io/gh/apache/incubator-airflow/pull/4368?src=pr=footer). Last update [7a6acbf...bed94ae](https://codecov.io/gh/apache/incubator-airflow/pull/4368?src=pr=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments). This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] codecov-io commented on issue #4368: AIRFLOW-3561 - improve queries
codecov-io commented on issue #4368: AIRFLOW-3561 - improve queries URL: https://github.com/apache/incubator-airflow/pull/4368#issuecomment-449655224 # [Codecov](https://codecov.io/gh/apache/incubator-airflow/pull/4368?src=pr=h1) Report > Merging [#4368](https://codecov.io/gh/apache/incubator-airflow/pull/4368?src=pr=desc) into [master](https://codecov.io/gh/apache/incubator-airflow/commit/7a6acbf5b343e4a6895d1cc8af75ecc02b4fd0e8?src=pr=desc) will **decrease** coverage by `<.01%`. > The diff coverage is `100%`. [](https://codecov.io/gh/apache/incubator-airflow/pull/4368?src=pr=tree) ```diff @@Coverage Diff @@ ## master#4368 +/- ## == - Coverage 78.12% 78.12% -0.01% == Files 202 202 Lines 1648316490 +7 == + Hits1287812883 +5 - Misses 3605 3607 +2 ``` | [Impacted Files](https://codecov.io/gh/apache/incubator-airflow/pull/4368?src=pr=tree) | Coverage Δ | | |---|---|---| | [airflow/www/views.py](https://codecov.io/gh/apache/incubator-airflow/pull/4368/diff?src=pr=tree#diff-YWlyZmxvdy93d3cvdmlld3MucHk=) | `69.46% <100%> (+0.06%)` | :arrow_up: | | [airflow/models/\_\_init\_\_.py](https://codecov.io/gh/apache/incubator-airflow/pull/4368/diff?src=pr=tree#diff-YWlyZmxvdy9tb2RlbHMvX19pbml0X18ucHk=) | `92.63% <0%> (-0.05%)` | :arrow_down: | -- [Continue to review full report at Codecov](https://codecov.io/gh/apache/incubator-airflow/pull/4368?src=pr=continue). > **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta) > `Δ = absolute (impact)`, `ø = not affected`, `? = missing data` > Powered by [Codecov](https://codecov.io/gh/apache/incubator-airflow/pull/4368?src=pr=footer). Last update [7a6acbf...bed94ae](https://codecov.io/gh/apache/incubator-airflow/pull/4368?src=pr=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments). This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] ffinfo opened a new pull request #4368: AIRFLOW-3561 - improve queries
ffinfo opened a new pull request #4368: AIRFLOW-3561 - improve queries
URL: https://github.com/apache/incubator-airflow/pull/4368
Make sure you have checked _all_ steps below.
### Jira
- [ ] My PR addresses the following [Airflow
Jira](https://issues.apache.org/jira/browse/AIRFLOW/) issues and references
them in the PR title. For example, "\[AIRFLOW-3561\] My Airflow PR"
- https://issues.apache.org/jira/browse/AIRFLOW-3561
### Description
- [ ] Here are some details about my PR, including screenshots of any UI
changes:
### Tests
- [ ] My PR adds the following unit tests __OR__ does not need testing for
this extremely good reason:
### Commits
- [ ] My commits all reference Jira issues in their subject lines, and I
have squashed multiple commits if they address the same issue. In addition, my
commits follow the guidelines from "[How to write a good git commit
message](http://chris.beams.io/posts/git-commit/)":
1. Subject is separated from body by a blank line
1. Subject is limited to 50 characters (not including Jira issue reference)
1. Subject does not end with a period
1. Subject uses the imperative mood ("add", not "adding")
1. Body wraps at 72 characters
1. Body explains "what" and "why", not "how"
### Documentation
- [ ] In case of new functionality, my PR adds documentation that describes
how to use it.
- When adding new operators/hooks/sensors, the autoclass documentation
generation needs to be added.
- All the public functions and the classes in the PR contain docstrings
that explain what it does
### Code Quality
- [ ] Passes `flake8`
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] codecov-io edited a comment on issue #4367: [AIRFLOW-3551] Improve BashOperator Test Coverage
codecov-io edited a comment on issue #4367: [AIRFLOW-3551] Improve BashOperator Test Coverage URL: https://github.com/apache/incubator-airflow/pull/4367#issuecomment-449651441 # [Codecov](https://codecov.io/gh/apache/incubator-airflow/pull/4367?src=pr=h1) Report > Merging [#4367](https://codecov.io/gh/apache/incubator-airflow/pull/4367?src=pr=desc) into [master](https://codecov.io/gh/apache/incubator-airflow/commit/7a6acbf5b343e4a6895d1cc8af75ecc02b4fd0e8?src=pr=desc) will **increase** coverage by `<.01%`. > The diff coverage is `100%`. [](https://codecov.io/gh/apache/incubator-airflow/pull/4367?src=pr=tree) ```diff @@Coverage Diff @@ ## master#4367 +/- ## == + Coverage 78.12% 78.13% +<.01% == Files 202 202 Lines 1648316482 -1 == Hits1287812878 + Misses 3605 3604 -1 ``` | [Impacted Files](https://codecov.io/gh/apache/incubator-airflow/pull/4367?src=pr=tree) | Coverage Δ | | |---|---|---| | [airflow/operators/bash\_operator.py](https://codecov.io/gh/apache/incubator-airflow/pull/4367/diff?src=pr=tree#diff-YWlyZmxvdy9vcGVyYXRvcnMvYmFzaF9vcGVyYXRvci5weQ==) | `92.98% <100%> (+1.6%)` | :arrow_up: | -- [Continue to review full report at Codecov](https://codecov.io/gh/apache/incubator-airflow/pull/4367?src=pr=continue). > **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta) > `Δ = absolute (impact)`, `ø = not affected`, `? = missing data` > Powered by [Codecov](https://codecov.io/gh/apache/incubator-airflow/pull/4367?src=pr=footer). Last update [7a6acbf...ba6109f](https://codecov.io/gh/apache/incubator-airflow/pull/4367?src=pr=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments). This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] codecov-io edited a comment on issue #4367: [AIRFLOW-3551] Improve BashOperator Test Coverage
codecov-io edited a comment on issue #4367: [AIRFLOW-3551] Improve BashOperator Test Coverage URL: https://github.com/apache/incubator-airflow/pull/4367#issuecomment-449651441 # [Codecov](https://codecov.io/gh/apache/incubator-airflow/pull/4367?src=pr=h1) Report > Merging [#4367](https://codecov.io/gh/apache/incubator-airflow/pull/4367?src=pr=desc) into [master](https://codecov.io/gh/apache/incubator-airflow/commit/7a6acbf5b343e4a6895d1cc8af75ecc02b4fd0e8?src=pr=desc) will **increase** coverage by `<.01%`. > The diff coverage is `100%`. [](https://codecov.io/gh/apache/incubator-airflow/pull/4367?src=pr=tree) ```diff @@Coverage Diff @@ ## master#4367 +/- ## == + Coverage 78.12% 78.13% +<.01% == Files 202 202 Lines 1648316482 -1 == Hits1287812878 + Misses 3605 3604 -1 ``` | [Impacted Files](https://codecov.io/gh/apache/incubator-airflow/pull/4367?src=pr=tree) | Coverage Δ | | |---|---|---| | [airflow/operators/bash\_operator.py](https://codecov.io/gh/apache/incubator-airflow/pull/4367/diff?src=pr=tree#diff-YWlyZmxvdy9vcGVyYXRvcnMvYmFzaF9vcGVyYXRvci5weQ==) | `92.98% <100%> (+1.6%)` | :arrow_up: | -- [Continue to review full report at Codecov](https://codecov.io/gh/apache/incubator-airflow/pull/4367?src=pr=continue). > **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta) > `Δ = absolute (impact)`, `ø = not affected`, `? = missing data` > Powered by [Codecov](https://codecov.io/gh/apache/incubator-airflow/pull/4367?src=pr=footer). Last update [7a6acbf...ba6109f](https://codecov.io/gh/apache/incubator-airflow/pull/4367?src=pr=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments). This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] codecov-io commented on issue #4367: [AIRFLOW-3551] Improve BashOperator Test Coverage
codecov-io commented on issue #4367: [AIRFLOW-3551] Improve BashOperator Test Coverage URL: https://github.com/apache/incubator-airflow/pull/4367#issuecomment-449651441 # [Codecov](https://codecov.io/gh/apache/incubator-airflow/pull/4367?src=pr=h1) Report > Merging [#4367](https://codecov.io/gh/apache/incubator-airflow/pull/4367?src=pr=desc) into [master](https://codecov.io/gh/apache/incubator-airflow/commit/7a6acbf5b343e4a6895d1cc8af75ecc02b4fd0e8?src=pr=desc) will **increase** coverage by `<.01%`. > The diff coverage is `100%`. [](https://codecov.io/gh/apache/incubator-airflow/pull/4367?src=pr=tree) ```diff @@Coverage Diff @@ ## master#4367 +/- ## == + Coverage 78.12% 78.13% +<.01% == Files 202 202 Lines 1648316482 -1 == Hits1287812878 + Misses 3605 3604 -1 ``` | [Impacted Files](https://codecov.io/gh/apache/incubator-airflow/pull/4367?src=pr=tree) | Coverage Δ | | |---|---|---| | [airflow/operators/bash\_operator.py](https://codecov.io/gh/apache/incubator-airflow/pull/4367/diff?src=pr=tree#diff-YWlyZmxvdy9vcGVyYXRvcnMvYmFzaF9vcGVyYXRvci5weQ==) | `92.98% <100%> (+1.6%)` | :arrow_up: | -- [Continue to review full report at Codecov](https://codecov.io/gh/apache/incubator-airflow/pull/4367?src=pr=continue). > **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta) > `Δ = absolute (impact)`, `ø = not affected`, `? = missing data` > Powered by [Codecov](https://codecov.io/gh/apache/incubator-airflow/pull/4367?src=pr=footer). Last update [7a6acbf...ba6109f](https://codecov.io/gh/apache/incubator-airflow/pull/4367?src=pr=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments). This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] codecov-io edited a comment on issue #4366: [AIRFLOW-3551] Improve BashOperator Test Coverage
codecov-io edited a comment on issue #4366: [AIRFLOW-3551] Improve BashOperator Test Coverage URL: https://github.com/apache/incubator-airflow/pull/4366#issuecomment-449649495 # [Codecov](https://codecov.io/gh/apache/incubator-airflow/pull/4366?src=pr=h1) Report > Merging [#4366](https://codecov.io/gh/apache/incubator-airflow/pull/4366?src=pr=desc) into [master](https://codecov.io/gh/apache/incubator-airflow/commit/7a6acbf5b343e4a6895d1cc8af75ecc02b4fd0e8?src=pr=desc) will **increase** coverage by `<.01%`. > The diff coverage is `100%`. [](https://codecov.io/gh/apache/incubator-airflow/pull/4366?src=pr=tree) ```diff @@Coverage Diff @@ ## master#4366 +/- ## == + Coverage 78.12% 78.13% +<.01% == Files 202 202 Lines 1648316482 -1 == Hits1287812878 + Misses 3605 3604 -1 ``` | [Impacted Files](https://codecov.io/gh/apache/incubator-airflow/pull/4366?src=pr=tree) | Coverage Δ | | |---|---|---| | [airflow/operators/bash\_operator.py](https://codecov.io/gh/apache/incubator-airflow/pull/4366/diff?src=pr=tree#diff-YWlyZmxvdy9vcGVyYXRvcnMvYmFzaF9vcGVyYXRvci5weQ==) | `92.98% <100%> (+1.6%)` | :arrow_up: | -- [Continue to review full report at Codecov](https://codecov.io/gh/apache/incubator-airflow/pull/4366?src=pr=continue). > **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta) > `Δ = absolute (impact)`, `ø = not affected`, `? = missing data` > Powered by [Codecov](https://codecov.io/gh/apache/incubator-airflow/pull/4366?src=pr=footer). Last update [7a6acbf...ba6109f](https://codecov.io/gh/apache/incubator-airflow/pull/4366?src=pr=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments). This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] codecov-io edited a comment on issue #4366: [AIRFLOW-3551] Improve BashOperator Test Coverage
codecov-io edited a comment on issue #4366: [AIRFLOW-3551] Improve BashOperator Test Coverage URL: https://github.com/apache/incubator-airflow/pull/4366#issuecomment-449649495 # [Codecov](https://codecov.io/gh/apache/incubator-airflow/pull/4366?src=pr=h1) Report > Merging [#4366](https://codecov.io/gh/apache/incubator-airflow/pull/4366?src=pr=desc) into [master](https://codecov.io/gh/apache/incubator-airflow/commit/7a6acbf5b343e4a6895d1cc8af75ecc02b4fd0e8?src=pr=desc) will **increase** coverage by `<.01%`. > The diff coverage is `100%`. [](https://codecov.io/gh/apache/incubator-airflow/pull/4366?src=pr=tree) ```diff @@Coverage Diff @@ ## master#4366 +/- ## == + Coverage 78.12% 78.13% +<.01% == Files 202 202 Lines 1648316482 -1 == Hits1287812878 + Misses 3605 3604 -1 ``` | [Impacted Files](https://codecov.io/gh/apache/incubator-airflow/pull/4366?src=pr=tree) | Coverage Δ | | |---|---|---| | [airflow/operators/bash\_operator.py](https://codecov.io/gh/apache/incubator-airflow/pull/4366/diff?src=pr=tree#diff-YWlyZmxvdy9vcGVyYXRvcnMvYmFzaF9vcGVyYXRvci5weQ==) | `92.98% <100%> (+1.6%)` | :arrow_up: | -- [Continue to review full report at Codecov](https://codecov.io/gh/apache/incubator-airflow/pull/4366?src=pr=continue). > **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta) > `Δ = absolute (impact)`, `ø = not affected`, `? = missing data` > Powered by [Codecov](https://codecov.io/gh/apache/incubator-airflow/pull/4366?src=pr=footer). Last update [7a6acbf...ba6109f](https://codecov.io/gh/apache/incubator-airflow/pull/4366?src=pr=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments). This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] codecov-io edited a comment on issue #4366: [AIRFLOW-3551] Improve BashOperator Test Coverage
codecov-io edited a comment on issue #4366: [AIRFLOW-3551] Improve BashOperator Test Coverage URL: https://github.com/apache/incubator-airflow/pull/4366#issuecomment-449649495 # [Codecov](https://codecov.io/gh/apache/incubator-airflow/pull/4366?src=pr=h1) Report > Merging [#4366](https://codecov.io/gh/apache/incubator-airflow/pull/4366?src=pr=desc) into [master](https://codecov.io/gh/apache/incubator-airflow/commit/7a6acbf5b343e4a6895d1cc8af75ecc02b4fd0e8?src=pr=desc) will **increase** coverage by `<.01%`. > The diff coverage is `100%`. [](https://codecov.io/gh/apache/incubator-airflow/pull/4366?src=pr=tree) ```diff @@Coverage Diff @@ ## master#4366 +/- ## == + Coverage 78.12% 78.13% +<.01% == Files 202 202 Lines 1648316482 -1 == Hits1287812878 + Misses 3605 3604 -1 ``` | [Impacted Files](https://codecov.io/gh/apache/incubator-airflow/pull/4366?src=pr=tree) | Coverage Δ | | |---|---|---| | [airflow/operators/bash\_operator.py](https://codecov.io/gh/apache/incubator-airflow/pull/4366/diff?src=pr=tree#diff-YWlyZmxvdy9vcGVyYXRvcnMvYmFzaF9vcGVyYXRvci5weQ==) | `92.98% <100%> (+1.6%)` | :arrow_up: | -- [Continue to review full report at Codecov](https://codecov.io/gh/apache/incubator-airflow/pull/4366?src=pr=continue). > **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta) > `Δ = absolute (impact)`, `ø = not affected`, `? = missing data` > Powered by [Codecov](https://codecov.io/gh/apache/incubator-airflow/pull/4366?src=pr=footer). Last update [7a6acbf...ba6109f](https://codecov.io/gh/apache/incubator-airflow/pull/4366?src=pr=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments). This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] codecov-io commented on issue #4366: [AIRFLOW-3551] Improve BashOperator Test Coverage
codecov-io commented on issue #4366: [AIRFLOW-3551] Improve BashOperator Test Coverage URL: https://github.com/apache/incubator-airflow/pull/4366#issuecomment-449649495 # [Codecov](https://codecov.io/gh/apache/incubator-airflow/pull/4366?src=pr=h1) Report > Merging [#4366](https://codecov.io/gh/apache/incubator-airflow/pull/4366?src=pr=desc) into [master](https://codecov.io/gh/apache/incubator-airflow/commit/7a6acbf5b343e4a6895d1cc8af75ecc02b4fd0e8?src=pr=desc) will **increase** coverage by `<.01%`. > The diff coverage is `100%`. [](https://codecov.io/gh/apache/incubator-airflow/pull/4366?src=pr=tree) ```diff @@Coverage Diff @@ ## master#4366 +/- ## == + Coverage 78.12% 78.13% +<.01% == Files 202 202 Lines 1648316482 -1 == Hits1287812878 + Misses 3605 3604 -1 ``` | [Impacted Files](https://codecov.io/gh/apache/incubator-airflow/pull/4366?src=pr=tree) | Coverage Δ | | |---|---|---| | [airflow/operators/bash\_operator.py](https://codecov.io/gh/apache/incubator-airflow/pull/4366/diff?src=pr=tree#diff-YWlyZmxvdy9vcGVyYXRvcnMvYmFzaF9vcGVyYXRvci5weQ==) | `92.98% <100%> (+1.6%)` | :arrow_up: | -- [Continue to review full report at Codecov](https://codecov.io/gh/apache/incubator-airflow/pull/4366?src=pr=continue). > **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta) > `Δ = absolute (impact)`, `ø = not affected`, `? = missing data` > Powered by [Codecov](https://codecov.io/gh/apache/incubator-airflow/pull/4366?src=pr=footer). Last update [7a6acbf...ba6109f](https://codecov.io/gh/apache/incubator-airflow/pull/4366?src=pr=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments). This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] codecov-io edited a comment on issue #4366: [AIRFLOW-3551] Improve BashOperator Test Coverage
codecov-io edited a comment on issue #4366: [AIRFLOW-3551] Improve BashOperator Test Coverage URL: https://github.com/apache/incubator-airflow/pull/4366#issuecomment-449649495 # [Codecov](https://codecov.io/gh/apache/incubator-airflow/pull/4366?src=pr=h1) Report > Merging [#4366](https://codecov.io/gh/apache/incubator-airflow/pull/4366?src=pr=desc) into [master](https://codecov.io/gh/apache/incubator-airflow/commit/7a6acbf5b343e4a6895d1cc8af75ecc02b4fd0e8?src=pr=desc) will **increase** coverage by `<.01%`. > The diff coverage is `100%`. [](https://codecov.io/gh/apache/incubator-airflow/pull/4366?src=pr=tree) ```diff @@Coverage Diff @@ ## master#4366 +/- ## == + Coverage 78.12% 78.13% +<.01% == Files 202 202 Lines 1648316482 -1 == Hits1287812878 + Misses 3605 3604 -1 ``` | [Impacted Files](https://codecov.io/gh/apache/incubator-airflow/pull/4366?src=pr=tree) | Coverage Δ | | |---|---|---| | [airflow/operators/bash\_operator.py](https://codecov.io/gh/apache/incubator-airflow/pull/4366/diff?src=pr=tree#diff-YWlyZmxvdy9vcGVyYXRvcnMvYmFzaF9vcGVyYXRvci5weQ==) | `92.98% <100%> (+1.6%)` | :arrow_up: | -- [Continue to review full report at Codecov](https://codecov.io/gh/apache/incubator-airflow/pull/4366?src=pr=continue). > **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta) > `Δ = absolute (impact)`, `ø = not affected`, `? = missing data` > Powered by [Codecov](https://codecov.io/gh/apache/incubator-airflow/pull/4366?src=pr=footer). Last update [7a6acbf...ba6109f](https://codecov.io/gh/apache/incubator-airflow/pull/4366?src=pr=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments). This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] feluelle opened a new pull request #4367: [AIRFLOW-3551] Improve BashOperator Test Coverage
feluelle opened a new pull request #4367: [AIRFLOW-3551] Improve BashOperator
Test Coverage
URL: https://github.com/apache/incubator-airflow/pull/4367
Make sure you have checked _all_ steps below.
### Jira
- [x] My PR addresses the following [Airflow
Jira](https://issues.apache.org/jira/browse/AIRFLOW/) issues and references
them in the PR title. For example, "\[AIRFLOW-XXX\] My Airflow PR"
- https://issues.apache.org/jira/browse/AIRFLOW-3551
- In case you are fixing a typo in the documentation you can prepend your
commit with \[AIRFLOW-XXX\], code changes always need a Jira issue.
### Description
- [x] Here are some details about my PR, including screenshots of any UI
changes:
- adds test case for xcom_push=True
- refactoring
### Tests
- [x] My PR adds the following unit tests __OR__ does not need testing for
this extremely good reason:
### Commits
- [x] My commits all reference Jira issues in their subject lines, and I
have squashed multiple commits if they address the same issue. In addition, my
commits follow the guidelines from "[How to write a good git commit
message](http://chris.beams.io/posts/git-commit/)":
1. Subject is separated from body by a blank line
1. Subject is limited to 50 characters (not including Jira issue reference)
1. Subject does not end with a period
1. Subject uses the imperative mood ("add", not "adding")
1. Body wraps at 72 characters
1. Body explains "what" and "why", not "how"
### Documentation
- [x] In case of new functionality, my PR adds documentation that describes
how to use it.
- When adding new operators/hooks/sensors, the autoclass documentation
generation needs to be added.
- All the public functions and the classes in the PR contain docstrings
that explain what it does
### Code Quality
- [x] Passes `flake8`
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] feluelle closed pull request #4366: [AIRFLOW-3551] Improve BashOperator Test Coverage
feluelle closed pull request #4366: [AIRFLOW-3551] Improve BashOperator Test
Coverage
URL: https://github.com/apache/incubator-airflow/pull/4366
This is a PR merged from a forked repository.
As GitHub hides the original diff on merge, it is displayed below for
the sake of provenance:
As this is a foreign pull request (from a fork), the diff is supplied
below (as it won't show otherwise due to GitHub magic):
diff --git a/airflow/operators/bash_operator.py
b/airflow/operators/bash_operator.py
index 13aa44fc85..a2217adf40 100644
--- a/airflow/operators/bash_operator.py
+++ b/airflow/operators/bash_operator.py
@@ -20,11 +20,10 @@
import os
import signal
+from builtins import bytes
from subprocess import Popen, STDOUT, PIPE
from tempfile import gettempdir, NamedTemporaryFile
-from builtins import bytes
-
from airflow.exceptions import AirflowException
from airflow.models import BaseOperator
from airflow.utils.decorators import apply_defaults
@@ -66,13 +65,12 @@ class BashOperator(BaseOperator):
ui_color = '#f0ede4'
@apply_defaults
-def __init__(
-self,
-bash_command,
-xcom_push=False,
-env=None,
-output_encoding='utf-8',
-*args, **kwargs):
+def __init__(self,
+ bash_command,
+ xcom_push=False,
+ env=None,
+ output_encoding='utf-8',
+ *args, **kwargs):
super(BashOperator, self).__init__(*args, **kwargs)
self.bash_command = bash_command
@@ -85,14 +83,14 @@ def execute(self, context):
Execute the bash command in a temporary directory
which will be cleaned afterwards
"""
-self.log.info("Tmp dir root location: \n %s", gettempdir())
+self.log.info('Tmp dir root location: \n %s', gettempdir())
# Prepare env for child process.
if self.env is None:
self.env = os.environ.copy()
-airflow_context_vars = context_to_airflow_vars(context,
- in_env_var_format=True)
-self.log.info("Exporting the following env vars:\n" +
+
+airflow_context_vars = context_to_airflow_vars(context,
in_env_var_format=True)
+self.log.info('Exporting the following env vars:\n' +
'\n'.join(["{}={}".format(k, v)
for k, v in
airflow_context_vars.items()]))
@@ -101,16 +99,11 @@ def execute(self, context):
self.lineage_data = self.bash_command
with TemporaryDirectory(prefix='airflowtmp') as tmp_dir:
-with NamedTemporaryFile(dir=tmp_dir, prefix=self.task_id) as f:
-
-f.write(bytes(self.bash_command, 'utf_8'))
-f.flush()
-fname = f.name
-script_location = os.path.abspath(fname)
-self.log.info(
-"Temporary script location: %s",
-script_location
-)
+with NamedTemporaryFile(dir=tmp_dir, prefix=self.task_id) as
tmp_file:
+tmp_file.write(bytes(self.bash_command, 'utf_8'))
+tmp_file.flush()
+script_location = os.path.abspath(tmp_file.name)
+self.log.info('Temporary script location: %s', script_location)
def pre_exec():
# Restore default signal disposition and invoke setsid
@@ -119,32 +112,33 @@ def pre_exec():
signal.signal(getattr(signal, sig), signal.SIG_DFL)
os.setsid()
-self.log.info("Running command: %s", self.bash_command)
-sp = Popen(
-['bash', fname],
-stdout=PIPE, stderr=STDOUT,
-cwd=tmp_dir, env=self.env,
+self.log.info('Running command: %s', self.bash_command)
+sub_process = Popen(
+['bash', tmp_file.name],
+stdout=PIPE,
+stderr=STDOUT,
+cwd=tmp_dir,
+env=self.env,
preexec_fn=pre_exec)
-self.sp = sp
+self.sub_process = sub_process
-self.log.info("Output:")
+self.log.info('Output:')
line = ''
-for line in iter(sp.stdout.readline, b''):
-line = line.decode(self.output_encoding).rstrip()
+for raw_line in iter(sub_process.stdout.readline, b''):
+line = raw_line.decode(self.output_encoding).rstrip()
self.log.info(line)
-sp.wait()
-self.log.info(
-"Command exited with return code %s",
-sp.returncode
-)
-if sp.returncode:
-raise
[GitHub] feluelle opened a new pull request #4366: [AIRFLOW-3552] Improve BashOperator Test Coverage
feluelle opened a new pull request #4366: [AIRFLOW-3552] Improve BashOperator
Test Coverage
URL: https://github.com/apache/incubator-airflow/pull/4366
Make sure you have checked _all_ steps below.
### Jira
- [x] My PR addresses the following [Airflow
Jira](https://issues.apache.org/jira/browse/AIRFLOW/) issues and references
them in the PR title. For example, "\[AIRFLOW-XXX\] My Airflow PR"
- https://issues.apache.org/jira/browse/AIRFLOW-3551
- In case you are fixing a typo in the documentation you can prepend your
commit with \[AIRFLOW-XXX\], code changes always need a Jira issue.
### Description
- [x] Here are some details about my PR, including screenshots of any UI
changes:
- adds test case for xcom_push=True
- refactoring
### Tests
- [x] My PR adds the following unit tests __OR__ does not need testing for
this extremely good reason:
### Commits
- [x] My commits all reference Jira issues in their subject lines, and I
have squashed multiple commits if they address the same issue. In addition, my
commits follow the guidelines from "[How to write a good git commit
message](http://chris.beams.io/posts/git-commit/)":
1. Subject is separated from body by a blank line
1. Subject is limited to 50 characters (not including Jira issue reference)
1. Subject does not end with a period
1. Subject uses the imperative mood ("add", not "adding")
1. Body wraps at 72 characters
1. Body explains "what" and "why", not "how"
### Documentation
- [x] In case of new functionality, my PR adds documentation that describes
how to use it.
- When adding new operators/hooks/sensors, the autoclass documentation
generation needs to be added.
- All the public functions and the classes in the PR contain docstrings
that explain what it does
### Code Quality
- [x] Passes `flake8`
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] codecov-io edited a comment on issue #4056: [AIRFLOW-3207] option to stop task pushing result to xcom
codecov-io edited a comment on issue #4056: [AIRFLOW-3207] option to stop task pushing result to xcom URL: https://github.com/apache/incubator-airflow/pull/4056#issuecomment-429951253 # [Codecov](https://codecov.io/gh/apache/incubator-airflow/pull/4056?src=pr=h1) Report > Merging [#4056](https://codecov.io/gh/apache/incubator-airflow/pull/4056?src=pr=desc) into [master](https://codecov.io/gh/apache/incubator-airflow/commit/7a6acbf5b343e4a6895d1cc8af75ecc02b4fd0e8?src=pr=desc) will **increase** coverage by `<.01%`. > The diff coverage is `100%`. [](https://codecov.io/gh/apache/incubator-airflow/pull/4056?src=pr=tree) ```diff @@Coverage Diff @@ ## master#4056 +/- ## == + Coverage 78.12% 78.13% +<.01% == Files 202 202 Lines 1648316484 +1 == + Hits1287812879 +1 Misses 3605 3605 ``` | [Impacted Files](https://codecov.io/gh/apache/incubator-airflow/pull/4056?src=pr=tree) | Coverage Δ | | |---|---|---| | [airflow/models/\_\_init\_\_.py](https://codecov.io/gh/apache/incubator-airflow/pull/4056/diff?src=pr=tree#diff-YWlyZmxvdy9tb2RlbHMvX19pbml0X18ucHk=) | `92.68% <100%> (ø)` | :arrow_up: | -- [Continue to review full report at Codecov](https://codecov.io/gh/apache/incubator-airflow/pull/4056?src=pr=continue). > **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta) > `Δ = absolute (impact)`, `ø = not affected`, `? = missing data` > Powered by [Codecov](https://codecov.io/gh/apache/incubator-airflow/pull/4056?src=pr=footer). Last update [7a6acbf...b3a2f5b](https://codecov.io/gh/apache/incubator-airflow/pull/4056?src=pr=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments). This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] codecov-io edited a comment on issue #4056: [AIRFLOW-3207] option to stop task pushing result to xcom
codecov-io edited a comment on issue #4056: [AIRFLOW-3207] option to stop task pushing result to xcom URL: https://github.com/apache/incubator-airflow/pull/4056#issuecomment-429951253 # [Codecov](https://codecov.io/gh/apache/incubator-airflow/pull/4056?src=pr=h1) Report > Merging [#4056](https://codecov.io/gh/apache/incubator-airflow/pull/4056?src=pr=desc) into [master](https://codecov.io/gh/apache/incubator-airflow/commit/7a6acbf5b343e4a6895d1cc8af75ecc02b4fd0e8?src=pr=desc) will **increase** coverage by `<.01%`. > The diff coverage is `100%`. [](https://codecov.io/gh/apache/incubator-airflow/pull/4056?src=pr=tree) ```diff @@Coverage Diff @@ ## master#4056 +/- ## == + Coverage 78.12% 78.13% +<.01% == Files 202 202 Lines 1648316484 +1 == + Hits1287812879 +1 Misses 3605 3605 ``` | [Impacted Files](https://codecov.io/gh/apache/incubator-airflow/pull/4056?src=pr=tree) | Coverage Δ | | |---|---|---| | [airflow/models/\_\_init\_\_.py](https://codecov.io/gh/apache/incubator-airflow/pull/4056/diff?src=pr=tree#diff-YWlyZmxvdy9tb2RlbHMvX19pbml0X18ucHk=) | `92.68% <100%> (ø)` | :arrow_up: | -- [Continue to review full report at Codecov](https://codecov.io/gh/apache/incubator-airflow/pull/4056?src=pr=continue). > **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta) > `Δ = absolute (impact)`, `ø = not affected`, `? = missing data` > Powered by [Codecov](https://codecov.io/gh/apache/incubator-airflow/pull/4056?src=pr=footer). Last update [7a6acbf...b3a2f5b](https://codecov.io/gh/apache/incubator-airflow/pull/4056?src=pr=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments). This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] OmerJog commented on issue #4302: [AIRFLOW-2641] Fix MySqlToHiveTransfer to handle MySQL DECIMAL correctly
OmerJog commented on issue #4302: [AIRFLOW-2641] Fix MySqlToHiveTransfer to handle MySQL DECIMAL correctly URL: https://github.com/apache/incubator-airflow/pull/4302#issuecomment-449624481 Still waiting for review :) This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] bolkedebruin closed pull request #4361: [AIRFLOW-3558] Improve default tox flake8 excludes
bolkedebruin closed pull request #4361: [AIRFLOW-3558] Improve default tox flake8 excludes URL: https://github.com/apache/incubator-airflow/pull/4361 This is a PR merged from a forked repository. As GitHub hides the original diff on merge, it is displayed below for the sake of provenance: As this is a foreign pull request (from a fork), the diff is supplied below (as it won't show otherwise due to GitHub magic): diff --git a/.flake8 b/.flake8 index 2723df1f10..9c4bbcd661 100644 --- a/.flake8 +++ b/.flake8 @@ -1,3 +1,4 @@ [flake8] max-line-length = 110 ignore = E731 +exclude = .git, env, venv, .venv, VENV, .tox, .eggs, .* This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] kaxil commented on issue #4363: [AIRFLOW-3560] Add WeekEnd & DayOfWeek Sensors
kaxil commented on issue #4363: [AIRFLOW-3560] Add WeekEnd & DayOfWeek Sensors URL: https://github.com/apache/incubator-airflow/pull/4363#issuecomment-449617519 @feng-tao Not Really, the TimeDeltaSensor adds X sec/min/days but doesn't necessarily check what day it is. DayOfWeekSensor here would give the user explicit option to specify that. WeekEnd Just checks if it is Sat or Sun This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] kaxil commented on a change in pull request #4359: [AIRFLOW-3150] Make execution_date templated in TriggerDagRunOperator
kaxil commented on a change in pull request #4359: [AIRFLOW-3150] Make execution_date templated in TriggerDagRunOperator URL: https://github.com/apache/incubator-airflow/pull/4359#discussion_r243753098 ## File path: airflow/operators/dagrun_operator.py ## @@ -64,10 +66,26 @@ def __init__( super(TriggerDagRunOperator, self).__init__(*args, **kwargs) self.python_callable = python_callable self.trigger_dag_id = trigger_dag_id -self.execution_date = execution_date + +if isinstance(execution_date, datetime.datetime): +self.execution_date = execution_date.isoformat() +elif isinstance(execution_date, six.string_types): +self.execution_date = execution_date +elif execution_date is None: +self.execution_date = execution_date Review comment: It uses the current time when execution date is None This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] kaxil commented on a change in pull request #4359: [AIRFLOW-3150] Make execution_date templated in TriggerDagRunOperator
kaxil commented on a change in pull request #4359: [AIRFLOW-3150] Make execution_date templated in TriggerDagRunOperator URL: https://github.com/apache/incubator-airflow/pull/4359#discussion_r243753086 ## File path: airflow/operators/dagrun_operator.py ## @@ -64,10 +66,26 @@ def __init__( super(TriggerDagRunOperator, self).__init__(*args, **kwargs) self.python_callable = python_callable self.trigger_dag_id = trigger_dag_id -self.execution_date = execution_date + +if isinstance(execution_date, datetime.datetime): +self.execution_date = execution_date.isoformat() +elif isinstance(execution_date, six.string_types): +self.execution_date = execution_date +elif execution_date is None: +self.execution_date = execution_date Review comment: No, it is the default value as well and will also break backwards compatiblility. This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] eladkal commented on issue #4365: [AIRFLOW-1684] - Branching based on XCom variable (Docs)
eladkal commented on issue #4365: [AIRFLOW-1684] - Branching based on XCom variable (Docs) URL: https://github.com/apache/incubator-airflow/pull/4365#issuecomment-449617129 @feluelle follow our chat on Slack. Finally had the time to submit PR :) This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] eladkal opened a new pull request #4365: [AIRFLOW-1684] - Branching based on XCom variable (Docs)
eladkal opened a new pull request #4365: [AIRFLOW-1684] - Branching based on XCom variable (Docs) URL: https://github.com/apache/incubator-airflow/pull/4365 Elaborate how to use branching with XComs ### Jira - [ ] My PR addresses the following [Airflow Jira](https://issues.apache.org/jira/browse/AIRFLOW-1684) issues and references them in the PR title. ### Description - Elaborate how to use branching with XComs This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] feng-tao commented on issue #4363: [AIRFLOW-3560] Add WeekEnd & DayOfWeek Sensors
feng-tao commented on issue #4363: [AIRFLOW-3560] Add WeekEnd & DayOfWeek Sensors URL: https://github.com/apache/incubator-airflow/pull/4363#issuecomment-449615720 is it possible to use TimeDeltaSensor for this purpose? This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] feng-tao commented on a change in pull request #4359: [AIRFLOW-3150] Make execution_date templated in TriggerDagRunOperator
feng-tao commented on a change in pull request #4359: [AIRFLOW-3150] Make execution_date templated in TriggerDagRunOperator URL: https://github.com/apache/incubator-airflow/pull/4359#discussion_r243752391 ## File path: airflow/operators/dagrun_operator.py ## @@ -64,10 +66,26 @@ def __init__( super(TriggerDagRunOperator, self).__init__(*args, **kwargs) self.python_callable = python_callable self.trigger_dag_id = trigger_dag_id -self.execution_date = execution_date + +if isinstance(execution_date, datetime.datetime): +self.execution_date = execution_date.isoformat() +elif isinstance(execution_date, six.string_types): +self.execution_date = execution_date +elif execution_date is None: +self.execution_date = execution_date Review comment: if execution_date is None, should we raise error ? This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] jmcarp opened a new pull request #4364: [AIRFLOW-3550] Standardize GKE hook.
jmcarp opened a new pull request #4364: [AIRFLOW-3550] Standardize GKE hook.
URL: https://github.com/apache/incubator-airflow/pull/4364
Refactor `GKEClusterHook` to subclass `GoogleCloudBaseHook` and
authenticate with connection credentials.
Make sure you have checked _all_ steps below.
### Jira
- [x] My PR addresses the following [Airflow
Jira](https://issues.apache.org/jira/browse/AIRFLOW/) issues and references
them in the PR title. For example, "\[AIRFLOW-XXX\] My Airflow PR"
- https://issues.apache.org/jira/browse/AIRFLOW-3550
- In case you are fixing a typo in the documentation you can prepend your
commit with \[AIRFLOW-XXX\], code changes always need a Jira issue.
### Description
- [x] Here are some details about my PR, including screenshots of any UI
changes:
Refactor `GKEClusterHook` to subclass `GoogleCloudBaseHook` and authenticate
with connection credentials.
### Tests
- [x] My PR adds the following unit tests __OR__ does not need testing for
this extremely good reason:
### Commits
- [x] My commits all reference Jira issues in their subject lines, and I
have squashed multiple commits if they address the same issue. In addition, my
commits follow the guidelines from "[How to write a good git commit
message](http://chris.beams.io/posts/git-commit/)":
1. Subject is separated from body by a blank line
1. Subject is limited to 50 characters (not including Jira issue reference)
1. Subject does not end with a period
1. Subject uses the imperative mood ("add", not "adding")
1. Body wraps at 72 characters
1. Body explains "what" and "why", not "how"
### Documentation
- [x] In case of new functionality, my PR adds documentation that describes
how to use it.
- When adding new operators/hooks/sensors, the autoclass documentation
generation needs to be added.
- All the public functions and the classes in the PR contain docstrings
that explain what it does
### Code Quality
- [x] Passes `flake8`
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] stale[bot] commented on issue #4133: [AIRFLOW-3270] Allow passwordless-binding for LDAP auth backend
stale[bot] commented on issue #4133: [AIRFLOW-3270] Allow passwordless-binding for LDAP auth backend URL: https://github.com/apache/incubator-airflow/pull/4133#issuecomment-449611637 This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions. This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] kaxil opened a new pull request #4363: [AIRFLOW-3560] Add WeekEnd & DayOfWeek Sensors
kaxil opened a new pull request #4363: [AIRFLOW-3560] Add WeekEnd & DayOfWeek
Sensors
URL: https://github.com/apache/incubator-airflow/pull/4363
Make sure you have checked _all_ steps below.
### Jira
- [x] My PR addresses the following [Airflow
Jira](https://issues.apache.org/jira/browse/AIRFLOW/) issues and references
them in the PR title. For example, "\[AIRFLOW-XXX\] My Airflow PR"
- https://issues.apache.org/jira/browse/AIRFLOW-3560
### Description
- [x] Here are some details about my PR, including screenshots of any UI
changes:
One of the use-cases we had is we wanted to run certain tasks only on
Weekends or certain days of the weeks. Along the way, I have seen more people
requiring the same.
### Tests
- [x] My PR adds the following unit tests __OR__ does not need testing for
this extremely good reason:
* `DayOfWeekSensorTests`
* `WeekEndSensorTests`
### Commits
- [x] My commits all reference Jira issues in their subject lines, and I
have squashed multiple commits if they address the same issue. In addition, my
commits follow the guidelines from "[How to write a good git commit
message](http://chris.beams.io/posts/git-commit/)":
1. Subject is separated from body by a blank line
1. Subject is limited to 50 characters (not including Jira issue reference)
1. Subject does not end with a period
1. Subject uses the imperative mood ("add", not "adding")
1. Body wraps at 72 characters
1. Body explains "what" and "why", not "how"
### Documentation
- [x] In case of new functionality, my PR adds documentation that describes
how to use it.
- When adding new operators/hooks/sensors, the autoclass documentation
generation needs to be added.
- All the public functions and the classes in the PR contain docstrings
that explain what it does
### Code Quality
- [x] Passes `flake8`
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] codecov-io commented on issue #4359: [AIRFLOW-3150] Make execution_date templated in TriggerDagRunOperator
codecov-io commented on issue #4359: [AIRFLOW-3150] Make execution_date templated in TriggerDagRunOperator URL: https://github.com/apache/incubator-airflow/pull/4359#issuecomment-449597919 # [Codecov](https://codecov.io/gh/apache/incubator-airflow/pull/4359?src=pr=h1) Report > Merging [#4359](https://codecov.io/gh/apache/incubator-airflow/pull/4359?src=pr=desc) into [master](https://codecov.io/gh/apache/incubator-airflow/commit/0e365665957d09c1aa612014b40994e3634ef70e?src=pr=desc) will **increase** coverage by `<.01%`. > The diff coverage is `93.33%`. [](https://codecov.io/gh/apache/incubator-airflow/pull/4359?src=pr=tree) ```diff @@Coverage Diff @@ ## master#4359 +/- ## == + Coverage 78.12% 78.13% +<.01% == Files 202 202 Lines 1648316495 +12 == + Hits1287812889 +11 - Misses 3605 3606 +1 ``` | [Impacted Files](https://codecov.io/gh/apache/incubator-airflow/pull/4359?src=pr=tree) | Coverage Δ | | |---|---|---| | [airflow/operators/dagrun\_operator.py](https://codecov.io/gh/apache/incubator-airflow/pull/4359/diff?src=pr=tree#diff-YWlyZmxvdy9vcGVyYXRvcnMvZGFncnVuX29wZXJhdG9yLnB5) | `94.73% <93.33%> (-1.42%)` | :arrow_down: | -- [Continue to review full report at Codecov](https://codecov.io/gh/apache/incubator-airflow/pull/4359?src=pr=continue). > **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta) > `Δ = absolute (impact)`, `ø = not affected`, `? = missing data` > Powered by [Codecov](https://codecov.io/gh/apache/incubator-airflow/pull/4359?src=pr=footer). Last update [0e36566...670c29a](https://codecov.io/gh/apache/incubator-airflow/pull/4359?src=pr=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments). This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] codecov-io edited a comment on issue #4359: [AIRFLOW-3150] Make execution_date templated in TriggerDagRunOperator
codecov-io edited a comment on issue #4359: [AIRFLOW-3150] Make execution_date templated in TriggerDagRunOperator URL: https://github.com/apache/incubator-airflow/pull/4359#issuecomment-449597919 # [Codecov](https://codecov.io/gh/apache/incubator-airflow/pull/4359?src=pr=h1) Report > Merging [#4359](https://codecov.io/gh/apache/incubator-airflow/pull/4359?src=pr=desc) into [master](https://codecov.io/gh/apache/incubator-airflow/commit/0e365665957d09c1aa612014b40994e3634ef70e?src=pr=desc) will **increase** coverage by `<.01%`. > The diff coverage is `93.33%`. [](https://codecov.io/gh/apache/incubator-airflow/pull/4359?src=pr=tree) ```diff @@Coverage Diff @@ ## master#4359 +/- ## == + Coverage 78.12% 78.13% +<.01% == Files 202 202 Lines 1648316495 +12 == + Hits1287812889 +11 - Misses 3605 3606 +1 ``` | [Impacted Files](https://codecov.io/gh/apache/incubator-airflow/pull/4359?src=pr=tree) | Coverage Δ | | |---|---|---| | [airflow/operators/dagrun\_operator.py](https://codecov.io/gh/apache/incubator-airflow/pull/4359/diff?src=pr=tree#diff-YWlyZmxvdy9vcGVyYXRvcnMvZGFncnVuX29wZXJhdG9yLnB5) | `94.73% <93.33%> (-1.42%)` | :arrow_down: | -- [Continue to review full report at Codecov](https://codecov.io/gh/apache/incubator-airflow/pull/4359?src=pr=continue). > **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta) > `Δ = absolute (impact)`, `ø = not affected`, `? = missing data` > Powered by [Codecov](https://codecov.io/gh/apache/incubator-airflow/pull/4359?src=pr=footer). Last update [0e36566...670c29a](https://codecov.io/gh/apache/incubator-airflow/pull/4359?src=pr=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments). This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] codecov-io edited a comment on issue #4362: [AIRFLOW-3559] Add missing options to DatadogHook.
codecov-io edited a comment on issue #4362: [AIRFLOW-3559] Add missing options to DatadogHook. URL: https://github.com/apache/incubator-airflow/pull/4362#issuecomment-449595748 # [Codecov](https://codecov.io/gh/apache/incubator-airflow/pull/4362?src=pr=h1) Report > Merging [#4362](https://codecov.io/gh/apache/incubator-airflow/pull/4362?src=pr=desc) into [master](https://codecov.io/gh/apache/incubator-airflow/commit/0e365665957d09c1aa612014b40994e3634ef70e?src=pr=desc) will **not change** coverage. > The diff coverage is `n/a`. [](https://codecov.io/gh/apache/incubator-airflow/pull/4362?src=pr=tree) ```diff @@ Coverage Diff @@ ## master#4362 +/- ## === Coverage 78.12% 78.12% === Files 202 202 Lines 1648316483 === Hits1287812878 Misses 3605 3605 ``` -- [Continue to review full report at Codecov](https://codecov.io/gh/apache/incubator-airflow/pull/4362?src=pr=continue). > **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta) > `Δ = absolute (impact)`, `ø = not affected`, `? = missing data` > Powered by [Codecov](https://codecov.io/gh/apache/incubator-airflow/pull/4362?src=pr=footer). Last update [0e36566...359f778](https://codecov.io/gh/apache/incubator-airflow/pull/4362?src=pr=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments). This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] codecov-io commented on issue #4362: [AIRFLOW-3559] Add missing options to DatadogHook.
codecov-io commented on issue #4362: [AIRFLOW-3559] Add missing options to DatadogHook. URL: https://github.com/apache/incubator-airflow/pull/4362#issuecomment-449595748 # [Codecov](https://codecov.io/gh/apache/incubator-airflow/pull/4362?src=pr=h1) Report > Merging [#4362](https://codecov.io/gh/apache/incubator-airflow/pull/4362?src=pr=desc) into [master](https://codecov.io/gh/apache/incubator-airflow/commit/0e365665957d09c1aa612014b40994e3634ef70e?src=pr=desc) will **not change** coverage. > The diff coverage is `n/a`. [](https://codecov.io/gh/apache/incubator-airflow/pull/4362?src=pr=tree) ```diff @@ Coverage Diff @@ ## master#4362 +/- ## === Coverage 78.12% 78.12% === Files 202 202 Lines 1648316483 === Hits1287812878 Misses 3605 3605 ``` -- [Continue to review full report at Codecov](https://codecov.io/gh/apache/incubator-airflow/pull/4362?src=pr=continue). > **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta) > `Δ = absolute (impact)`, `ø = not affected`, `? = missing data` > Powered by [Codecov](https://codecov.io/gh/apache/incubator-airflow/pull/4362?src=pr=footer). Last update [0e36566...359f778](https://codecov.io/gh/apache/incubator-airflow/pull/4362?src=pr=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments). This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] stale[bot] closed pull request #2367: [AIRFLOW-1077] Warn about subdag deadlock case
stale[bot] closed pull request #2367: [AIRFLOW-1077] Warn about subdag deadlock case URL: https://github.com/apache/incubator-airflow/pull/2367 This is a PR merged from a forked repository. As GitHub hides the original diff on merge, it is displayed below for the sake of provenance: As this is a foreign pull request (from a fork), the diff is supplied below (as it won't show otherwise due to GitHub magic): diff --git a/docs/concepts.rst b/docs/concepts.rst index 33a6ea44c7..56fbd2a531 100644 --- a/docs/concepts.rst +++ b/docs/concepts.rst @@ -457,10 +457,10 @@ Not like this, where the join task is skipped .. image:: img/branch_bad.png -SubDAGs +SubDags === -SubDAGs are perfect for repeating patterns. Defining a function that returns a +SubDags are perfect for repeating patterns. Defining a function that returns a DAG object is a nice design pattern when using Airflow. Airbnb uses the *stage-check-exchange* pattern when loading data. Data is staged @@ -472,13 +472,13 @@ As another example, consider the following DAG: .. image:: img/subdag_before.png -We can combine all of the parallel ``task-*`` operators into a single SubDAG, +We can combine all of the parallel ``task-*`` operators into a single SubDag, so that the resulting DAG resembles the following: .. image:: img/subdag_after.png -Note that SubDAG operators should contain a factory method that returns a DAG -object. This will prevent the SubDAG from being treated like a separate DAG in +Note that SubDag operators should contain a factory method that returns a DAG +object. This will prevent the SubDag from being treated like a separate DAG in the main UI. For example: .. code:: python @@ -503,7 +503,7 @@ the main UI. For example: return dag -This SubDAG can then be referenced in your main DAG file: +This SubDag can then be referenced in your main DAG file: .. code:: python @@ -531,29 +531,36 @@ This SubDAG can then be referenced in your main DAG file: ) You can zoom into a SubDagOperator from the graph view of the main DAG to show -the tasks contained within the SubDAG: +the tasks contained within the SubDag: .. image:: img/subdag_zoom.png -Some other tips when using SubDAGs: +Some other tips when using SubDags: -- by convention, a SubDAG's ``dag_id`` should be prefixed by its parent and +- by convention, a SubDag's ``dag_id`` should be prefixed by its parent and a dot. As in ``parent.child`` -- share arguments between the main DAG and the SubDAG by passing arguments to - the SubDAG operator (as demonstrated above) -- SubDAGs must have a schedule and be enabled. If the SubDAG's schedule is - set to ``None`` or ``@once``, the SubDAG will succeed without having done +- share arguments between the main DAG and the SubDag by passing arguments to + the SubDag operator (as demonstrated above) +- SubDags must have a schedule and be enabled. If the SubDag's schedule is + set to ``None`` or ``@once``, the SubDag will succeed without having done anything - clearing a SubDagOperator also clears the state of the tasks within - marking success on a SubDagOperator does not affect the state of the tasks within -- refrain from using ``depends_on_past=True`` in tasks within the SubDAG as +- refrain from using ``depends_on_past=True`` in tasks within the SubDag as this can be confusing -- it is possible to specify an executor for the SubDAG. It is common to use - the SequentialExecutor if you want to run the SubDAG in-process and +- it is possible to specify an executor for the SubDag. It is common to use + the SequentialExecutor if you want to run the SubDag in-process and effectively limit its parallelism to one. Using LocalExecutor can be problematic as it may over-subscribe your worker, running multiple tasks in a single slot +- do not create more SubDags then your concurrency limit or the scheduler + will deadlock. Each SubDags counts towards your concurrency limit. For + example, if you have a concurrency limit of 16 and you have 25 SubDags, + the 16 SubDags will be scheduled, effectively blocking any of the tasks + within the given SubDags. You can work around this by setting the SubDag's + executor to SequentialExecutor. This allows multiple SubDag to run + concurrently without locking the tasks within the SubDag See ``airflow/example_dags`` for a demonstration. This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] codecov-io commented on issue #4361: [AIRFLOW-3558] Improve default tox flake8 excludes
codecov-io commented on issue #4361: [AIRFLOW-3558] Improve default tox flake8 excludes URL: https://github.com/apache/incubator-airflow/pull/4361#issuecomment-449592064 # [Codecov](https://codecov.io/gh/apache/incubator-airflow/pull/4361?src=pr=h1) Report > Merging [#4361](https://codecov.io/gh/apache/incubator-airflow/pull/4361?src=pr=desc) into [master](https://codecov.io/gh/apache/incubator-airflow/commit/0e365665957d09c1aa612014b40994e3634ef70e?src=pr=desc) will **not change** coverage. > The diff coverage is `n/a`. [](https://codecov.io/gh/apache/incubator-airflow/pull/4361?src=pr=tree) ```diff @@ Coverage Diff @@ ## master#4361 +/- ## === Coverage 78.12% 78.12% === Files 202 202 Lines 1648316483 === Hits1287812878 Misses 3605 3605 ``` -- [Continue to review full report at Codecov](https://codecov.io/gh/apache/incubator-airflow/pull/4361?src=pr=continue). > **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta) > `Δ = absolute (impact)`, `ø = not affected`, `? = missing data` > Powered by [Codecov](https://codecov.io/gh/apache/incubator-airflow/pull/4361?src=pr=footer). Last update [0e36566...a18c8af](https://codecov.io/gh/apache/incubator-airflow/pull/4361?src=pr=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments). This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] codecov-io edited a comment on issue #4361: [AIRFLOW-3558] Improve default tox flake8 excludes
codecov-io edited a comment on issue #4361: [AIRFLOW-3558] Improve default tox flake8 excludes URL: https://github.com/apache/incubator-airflow/pull/4361#issuecomment-449592064 # [Codecov](https://codecov.io/gh/apache/incubator-airflow/pull/4361?src=pr=h1) Report > Merging [#4361](https://codecov.io/gh/apache/incubator-airflow/pull/4361?src=pr=desc) into [master](https://codecov.io/gh/apache/incubator-airflow/commit/0e365665957d09c1aa612014b40994e3634ef70e?src=pr=desc) will **not change** coverage. > The diff coverage is `n/a`. [](https://codecov.io/gh/apache/incubator-airflow/pull/4361?src=pr=tree) ```diff @@ Coverage Diff @@ ## master#4361 +/- ## === Coverage 78.12% 78.12% === Files 202 202 Lines 1648316483 === Hits1287812878 Misses 3605 3605 ``` -- [Continue to review full report at Codecov](https://codecov.io/gh/apache/incubator-airflow/pull/4361?src=pr=continue). > **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta) > `Δ = absolute (impact)`, `ø = not affected`, `? = missing data` > Powered by [Codecov](https://codecov.io/gh/apache/incubator-airflow/pull/4361?src=pr=footer). Last update [0e36566...a18c8af](https://codecov.io/gh/apache/incubator-airflow/pull/4361?src=pr=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments). This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] jmcarp opened a new pull request #4362: [AIRFLOW-3559] Add missing options to DatadogHook.
jmcarp opened a new pull request #4362: [AIRFLOW-3559] Add missing options to
DatadogHook.
URL: https://github.com/apache/incubator-airflow/pull/4362
Make sure you have checked _all_ steps below.
### Jira
- [x] My PR addresses the following [Airflow
Jira](https://issues.apache.org/jira/browse/AIRFLOW/) issues and references
them in the PR title. For example, "\[AIRFLOW-XXX\] My Airflow PR"
- https://issues.apache.org/jira/browse/AIRFLOW-3559
- In case you are fixing a typo in the documentation you can prepend your
commit with \[AIRFLOW-XXX\], code changes always need a Jira issue.
### Description
- [x] Here are some details about my PR, including screenshots of any UI
changes:
Adds missing arguments to `DatadogHook`.
### Tests
- [x] My PR adds the following unit tests __OR__ does not need testing for
this extremely good reason:
Backfills missing tests for `DatadogHook`
### Commits
- [x] My commits all reference Jira issues in their subject lines, and I
have squashed multiple commits if they address the same issue. In addition, my
commits follow the guidelines from "[How to write a good git commit
message](http://chris.beams.io/posts/git-commit/)":
1. Subject is separated from body by a blank line
1. Subject is limited to 50 characters (not including Jira issue reference)
1. Subject does not end with a period
1. Subject uses the imperative mood ("add", not "adding")
1. Body wraps at 72 characters
1. Body explains "what" and "why", not "how"
### Documentation
- [x] In case of new functionality, my PR adds documentation that describes
how to use it.
- When adding new operators/hooks/sensors, the autoclass documentation
generation needs to be added.
- All the public functions and the classes in the PR contain docstrings
that explain what it does
### Code Quality
- [x] Passes `flake8`
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] holdenk opened a new pull request #4361: [AIRFLOW-3558] Improve default tox flake8 excludes
holdenk opened a new pull request #4361: [AIRFLOW-3558] Improve default tox
flake8 excludes
URL: https://github.com/apache/incubator-airflow/pull/4361
Right now our gitignore skips a bunch of temporary Python directories
but our flake8 config will still test against them, leading to
unnecessary error messages. This changes the excludes
to skip the common directories that can cause false flake8 failures.
Make sure you have checked _all_ steps below.
### Jira
- [ X ] My PR addresses the following [Airflow
Jira](https://issues.apache.org/jira/browse/AIRFLOW/) issues and references
them in the PR title. For example, "\[AIRFLOW-XXX\] My Airflow PR"
- https://issues.apache.org/jira/browse/AIRFLOW-XXX
- In case you are fixing a typo in the documentation you can prepend your
commit with \[AIRFLOW-XXX\], code changes always need a Jira issue.
### Description
- [ X ] Here are some details about my PR, including screenshots of any UI
changes:
Right now our gitignore skips a bunch of temporary Python directories
but our flake8 config will still test against them, leading to
unnecessary error messages. This changes the excludes
to skip the common directories that can cause false flake8 failures.
This should not impact end users.
### Tests
- [ X ] My PR adds the following unit tests __OR__ does not need testing for
this extremely good reason:
The existing flake8 env still runs from tox
### Commits
- [ X ] My commits all reference Jira issues in their subject lines, and I
have squashed multiple commits if they address the same issue. In addition, my
commits follow the guidelines from "[How to write a good git commit
message](http://chris.beams.io/posts/git-commit/)":
1. Subject is separated from body by a blank line
1. Subject is limited to 50 characters (not including Jira issue reference)
1. Subject does not end with a period
1. Subject uses the imperative mood ("add", not "adding")
1. Body wraps at 72 characters
1. Body explains "what" and "why", not "how"
### Documentation
- [ X ] In case of new functionality, my PR adds documentation that
describes how to use it.
- When adding new operators/hooks/sensors, the autoclass documentation
generation needs to be added.
- All the public functions and the classes in the PR contain docstrings
that explain what it does
No new functionality.
### Code Quality
- [ X ] Passes `flake8`
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] holdenk opened a new pull request #4360: [AIRFLOW-1191] Simplify override of spark submit command
holdenk opened a new pull request #4360: [AIRFLOW-1191] Simplify override of
spark submit command
URL: https://github.com/apache/incubator-airflow/pull/4360
This will better support distros which ship spark 1 & 2 ( and eventually 3)
Make sure you have checked _all_ steps below.
### Jira
- [ X ] My PR addresses the following [Airflow
Jira](https://issues.apache.org/jira/browse/AIRFLOW/) issues and references
them in the PR title. For example, "\[AIRFLOW-XXX\] My Airflow PR"
- https://issues.apache.org/jira/browse/AIRFLOW-XXX
- In case you are fixing a typo in the documentation you can prepend your
commit with \[AIRFLOW-XXX\], code changes always need a Jira issue.
### Description
- [ X ] Here are some details about my PR, including screenshots of any UI
changes:
Adds a spark_binary param to the spark submit operator to allow folks to
more easily configure the operator to use a different binary, as is needed for
some distros of the Hadoop ecosystem which ship multiple version of Spark.
### Tests
- [ X ] My PR adds the following unit tests __OR__ does not need testing for
this extremely good reason:
Updates the existing test_spark_submit_operator to check for override
### Commits
- [ X ] My commits all reference Jira issues in their subject lines, and I
have squashed multiple commits if they address the same issue. In addition, my
commits follow the guidelines from "[How to write a good git commit
message](http://chris.beams.io/posts/git-commit/)":
1. Subject is separated from body by a blank line
1. Subject is limited to 50 characters (not including Jira issue reference)
1. Subject does not end with a period
1. Subject uses the imperative mood ("add", not "adding")
1. Body wraps at 72 characters
1. Body explains "what" and "why", not "how"
### Documentation
- [ X ] In case of new functionality, my PR adds documentation that
describes how to use it.
- When adding new operators/hooks/sensors, the autoclass documentation
generation needs to be added.
- All the public functions and the classes in the PR contain docstrings
that explain what it does
docstring update
### Code Quality
- [ ] Passes `flake8`
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] kaxil closed pull request #4349: [AIRFLOW-850] Add a PythonSensor
kaxil closed pull request #4349: [AIRFLOW-850] Add a PythonSensor
URL: https://github.com/apache/incubator-airflow/pull/4349
This is a PR merged from a forked repository.
As GitHub hides the original diff on merge, it is displayed below for
the sake of provenance:
As this is a foreign pull request (from a fork), the diff is supplied
below (as it won't show otherwise due to GitHub magic):
diff --git a/airflow/contrib/sensors/python_sensor.py
b/airflow/contrib/sensors/python_sensor.py
new file mode 100644
index 00..68bc7497ea
--- /dev/null
+++ b/airflow/contrib/sensors/python_sensor.py
@@ -0,0 +1,81 @@
+# -*- coding: utf-8 -*-
+#
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+
+from airflow.sensors.base_sensor_operator import BaseSensorOperator
+from airflow.utils.decorators import apply_defaults
+
+
+class PythonSensor(BaseSensorOperator):
+"""
+Waits for a Python callable to return True.
+
+User could put input argument in templates_dict
+e.g templates_dict = {'start_ds': 1970}
+and access the argument by calling `kwargs['templates_dict']['start_ds']`
+in the the callable
+
+:param python_callable: A reference to an object that is callable
+:type python_callable: python callable
+:param op_kwargs: a dictionary of keyword arguments that will get unpacked
+in your function
+:type op_kwargs: dict
+:param op_args: a list of positional arguments that will get unpacked when
+calling your callable
+:type op_args: list
+:param provide_context: if set to true, Airflow will pass a set of
+keyword arguments that can be used in your function. This set of
+kwargs correspond exactly to what you can use in your jinja
+templates. For this to work, you need to define `**kwargs` in your
+function header.
+:type provide_context: bool
+:param templates_dict: a dictionary where the values are templates that
+will get templated by the Airflow engine sometime between
+``__init__`` and ``execute`` takes place and are made available
+in your callable's context after the template has been applied.
+:type templates_dict: dict of str
+"""
+
+template_fields = ('templates_dict',)
+template_ext = tuple()
+
+@apply_defaults
+def __init__(
+self,
+python_callable,
+op_args=None,
+op_kwargs=None,
+provide_context=False,
+templates_dict=None,
+*args, **kwargs):
+super(PythonSensor, self).__init__(*args, **kwargs)
+self.python_callable = python_callable
+self.op_args = op_args or []
+self.op_kwargs = op_kwargs or {}
+self.provide_context = provide_context
+self.templates_dict = templates_dict
+
+def poke(self, context):
+if self.provide_context:
+context.update(self.op_kwargs)
+context['templates_dict'] = self.templates_dict
+self.op_kwargs = context
+
+self.log.info("Poking callable: " + str(self.python_callable))
+return_value = self.python_callable(*self.op_args, **self.op_kwargs)
+return bool(return_value)
diff --git a/docs/code.rst b/docs/code.rst
index 61414ecbd6..e890adffec 100644
--- a/docs/code.rst
+++ b/docs/code.rst
@@ -256,6 +256,7 @@ Sensors
.. autoclass::
airflow.contrib.sensors.imap_attachment_sensor.ImapAttachmentSensor
.. autoclass:: airflow.contrib.sensors.jira_sensor.JiraSensor
.. autoclass:: airflow.contrib.sensors.pubsub_sensor.PubSubPullSensor
+.. autoclass:: airflow.contrib.sensors.python_sensor.PythonSensor
.. autoclass:: airflow.contrib.sensors.qubole_sensor.QuboleSensor
.. autoclass:: airflow.contrib.sensors.redis_key_sensor.RedisKeySensor
.. autoclass::
airflow.contrib.sensors.sagemaker_base_sensor.SageMakerBaseSensor
diff --git a/tests/contrib/sensors/test_python_sensor.py
b/tests/contrib/sensors/test_python_sensor.py
new file mode 100644
index 00..6c852e748a
--- /dev/null
+++ b/tests/contrib/sensors/test_python_sensor.py
@@ -0,0 +1,72 @@
+# -*- coding: utf-8 -*-
+#
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the
[GitHub] kaxil closed pull request #4357: [AIRFLOW-3557] Fix various typos
kaxil closed pull request #4357: [AIRFLOW-3557] Fix various typos
URL: https://github.com/apache/incubator-airflow/pull/4357
This is a PR merged from a forked repository.
As GitHub hides the original diff on merge, it is displayed below for
the sake of provenance:
As this is a foreign pull request (from a fork), the diff is supplied
below (as it won't show otherwise due to GitHub magic):
diff --git a/CHANGELOG.txt b/CHANGELOG.txt
index abb0563d71..98a1103792 100644
--- a/CHANGELOG.txt
+++ b/CHANGELOG.txt
@@ -24,7 +24,7 @@ Improvements:
[AIRFLOW-2622] Add "confirm=False" option to SFTPOperator
[AIRFLOW-2662] support affinity & nodeSelector policies for kubernetes
executor/operator
[AIRFLOW-2709] Improve error handling in Databricks hook
-[AIRFLOW-2723] Update lxml dependancy to >= 4.0.
+[AIRFLOW-2723] Update lxml dependency to >= 4.0.
[AIRFLOW-2763] No precheck mechanism in place during worker initialisation for
the connection to metadata database
[AIRFLOW-2789] Add ability to create single node cluster to
DataprocClusterCreateOperator
[AIRFLOW-2797] Add ability to create Google Dataproc cluster with custom image
@@ -269,7 +269,7 @@ AIRFLOW 1.10.0, 2018-08-03
[AIRFLOW-2429] Make Airflow flake8 compliant
[AIRFLOW-2491] Resolve flask version conflict
[AIRFLOW-2484] Remove duplicate key in MySQL to GCS Op
-[ARIFLOW-2458] Add cassandra-to-gcs operator
+[AIRFLOW-2458] Add cassandra-to-gcs operator
[AIRFLOW-2477] Improve time units for task duration and landing times charts
for RBAC UI
[AIRFLOW-2474] Only import snakebite if using py2
[AIRFLOW-48] Parse connection uri querystring
@@ -1504,7 +1504,7 @@ AIRFLOW 1.8.0, 2017-03-12
[AIRFLOW-784] Pin funcsigs to 1.0.0
[AIRFLOW-624] Fix setup.py to not import airflow.version as version
[AIRFLOW-779] Task should fail with specific message when deleted
-[AIRFLOW-778] Fix completey broken MetastorePartitionSensor
+[AIRFLOW-778] Fix completely broken MetastorePartitionSensor
[AIRFLOW-739] Set pickle_info log to debug
[AIRFLOW-771] Make S3 logs append instead of clobber
[AIRFLOW-773] Fix flaky datetime addition in api test
diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md
index 556a5d847b..2a60f1dc3c 100644
--- a/CONTRIBUTING.md
+++ b/CONTRIBUTING.md
@@ -166,10 +166,10 @@ There are three ways to setup an Apache Airflow
development environment.
tox -e py35-backend_mysql
```
- If you wish to run individual tests inside of docker enviroment you can do
as follows:
+ If you wish to run individual tests inside of Docker environment you can do
as follows:
```bash
-# From the container (with your desired enviroment) with druid hook
+# From the container (with your desired environment) with druid hook
tox -e py35-backend_mysql -- tests/hooks/test_druid_hook.py
```
diff --git a/airflow/contrib/hooks/bigquery_hook.py
b/airflow/contrib/hooks/bigquery_hook.py
index 5cab013b28..30a16305db 100644
--- a/airflow/contrib/hooks/bigquery_hook.py
+++ b/airflow/contrib/hooks/bigquery_hook.py
@@ -1594,7 +1594,7 @@ def insert_all(self, project_id, dataset_id, table_id,
self.log.info('All row(s) inserted successfully:
{}:{}.{}'.format(
dataset_project_id, dataset_id, table_id))
else:
-error_msg = '{} insert error(s) occured: {}:{}.{}. Details:
{}'.format(
+error_msg = '{} insert error(s) occurred: {}:{}.{}. Details:
{}'.format(
len(resp['insertErrors']),
dataset_project_id, dataset_id, table_id,
resp['insertErrors'])
if fail_on_error:
diff --git a/airflow/contrib/hooks/emr_hook.py
b/airflow/contrib/hooks/emr_hook.py
index f9fd3f04de..fcdf4ac848 100644
--- a/airflow/contrib/hooks/emr_hook.py
+++ b/airflow/contrib/hooks/emr_hook.py
@@ -23,7 +23,7 @@
class EmrHook(AwsHook):
"""
-Interact with AWS EMR. emr_conn_id is only neccessary for using the
+Interact with AWS EMR. emr_conn_id is only necessary for using the
create_job_flow method.
"""
diff --git a/airflow/executors/celery_executor.py
b/airflow/executors/celery_executor.py
index 98ce6efba7..10694ea4b7 100644
--- a/airflow/executors/celery_executor.py
+++ b/airflow/executors/celery_executor.py
@@ -74,7 +74,7 @@ def execute_command(command_to_exec):
class ExceptionWithTraceback(object):
"""
-Wrapper class used to propogate exceptions to parent processes from
subprocesses.
+Wrapper class used to propagate exceptions to parent processes from
subprocesses.
:param exception: The exception to wrap
:type exception: Exception
:param traceback: The stacktrace to wrap
diff --git a/airflow/sensors/base_sensor_operator.py
b/airflow/sensors/base_sensor_operator.py
index 1dc59dd230..eaa6306770 100644
--- a/airflow/sensors/base_sensor_operator.py
+++ b/airflow/sensors/base_sensor_operator.py
@@ -48,7 +48,7 @@ class BaseSensorOperator(BaseOperator, SkipMixin):
When set to ``poke`` the
[GitHub] kaxil opened a new pull request #4359: [AIRFLOW-3150] Make execution_date templated in TriggerDagRunOperator
kaxil opened a new pull request #4359: [AIRFLOW-3150] Make execution_date
templated in TriggerDagRunOperator
URL: https://github.com/apache/incubator-airflow/pull/4359
Make sure you have checked _all_ steps below.
### Jira
- [x] My PR addresses the following [Airflow
Jira](https://issues.apache.org/jira/browse/AIRFLOW/) issues and references
them in the PR title. For example, "\[AIRFLOW-XXX\] My Airflow PR"
- https://issues.apache.org/jira/browse/AIRFLOW-3150
### Description
- [x] Here are some details about my PR, including screenshots of any UI
changes:
### Tests
- [x] My PR adds the following unit tests __OR__ does not need testing for
this extremely good reason:
* `test_trigger_dagrun_with_str_execution_date`
### Commits
- [x] My commits all reference Jira issues in their subject lines, and I
have squashed multiple commits if they address the same issue. In addition, my
commits follow the guidelines from "[How to write a good git commit
message](http://chris.beams.io/posts/git-commit/)":
1. Subject is separated from body by a blank line
1. Subject is limited to 50 characters (not including Jira issue reference)
1. Subject does not end with a period
1. Subject uses the imperative mood ("add", not "adding")
1. Body wraps at 72 characters
1. Body explains "what" and "why", not "how"
### Documentation
- [x] In case of new functionality, my PR adds documentation that describes
how to use it.
- When adding new operators/hooks/sensors, the autoclass documentation
generation needs to be added.
- All the public functions and the classes in the PR contain docstrings
that explain what it does
### Code Quality
- [x] Passes `flake8`
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] stale[bot] commented on issue #4127: Bug Fix: Secrets object and key separated by ":"
stale[bot] commented on issue #4127: Bug Fix: Secrets object and key separated by ":" URL: https://github.com/apache/incubator-airflow/pull/4127#issuecomment-449581166 This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions. This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] XD-DENG commented on issue #4355: [AIRFLOW-XXX] Update log level and return value
XD-DENG commented on issue #4355: [AIRFLOW-XXX] Update log level and return value URL: https://github.com/apache/incubator-airflow/pull/4355#issuecomment-449577116 A JIRA ticket would be needed, given this is a code change. You can get the details in the original PR template. This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] codecov-io edited a comment on issue #4357: [AIRFLOW-3557] Fix various typos
codecov-io edited a comment on issue #4357: [AIRFLOW-3557] Fix various typos URL: https://github.com/apache/incubator-airflow/pull/4357#issuecomment-449563258 # [Codecov](https://codecov.io/gh/apache/incubator-airflow/pull/4357?src=pr=h1) Report > Merging [#4357](https://codecov.io/gh/apache/incubator-airflow/pull/4357?src=pr=desc) into [master](https://codecov.io/gh/apache/incubator-airflow/commit/701b877a2eb4aac151a643f6eb735abf303a4e5e?src=pr=desc) will **not change** coverage. > The diff coverage is `n/a`. [](https://codecov.io/gh/apache/incubator-airflow/pull/4357?src=pr=tree) ```diff @@ Coverage Diff @@ ## master#4357 +/- ## === Coverage 78.12% 78.12% === Files 202 202 Lines 1648316483 === Hits1287812878 Misses 3605 3605 ``` | [Impacted Files](https://codecov.io/gh/apache/incubator-airflow/pull/4357?src=pr=tree) | Coverage Δ | | |---|---|---| | [airflow/sensors/base\_sensor\_operator.py](https://codecov.io/gh/apache/incubator-airflow/pull/4357/diff?src=pr=tree#diff-YWlyZmxvdy9zZW5zb3JzL2Jhc2Vfc2Vuc29yX29wZXJhdG9yLnB5) | `97.87% <ø> (ø)` | :arrow_up: | | [airflow/executors/celery\_executor.py](https://codecov.io/gh/apache/incubator-airflow/pull/4357/diff?src=pr=tree#diff-YWlyZmxvdy9leGVjdXRvcnMvY2VsZXJ5X2V4ZWN1dG9yLnB5) | `76.92% <ø> (ø)` | :arrow_up: | -- [Continue to review full report at Codecov](https://codecov.io/gh/apache/incubator-airflow/pull/4357?src=pr=continue). > **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta) > `Δ = absolute (impact)`, `ø = not affected`, `? = missing data` > Powered by [Codecov](https://codecov.io/gh/apache/incubator-airflow/pull/4357?src=pr=footer). Last update [701b877...8badf67](https://codecov.io/gh/apache/incubator-airflow/pull/4357?src=pr=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments). This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] codecov-io commented on issue #4357: [AIRFLOW-3557] Fix various typos
codecov-io commented on issue #4357: [AIRFLOW-3557] Fix various typos URL: https://github.com/apache/incubator-airflow/pull/4357#issuecomment-449563258 # [Codecov](https://codecov.io/gh/apache/incubator-airflow/pull/4357?src=pr=h1) Report > Merging [#4357](https://codecov.io/gh/apache/incubator-airflow/pull/4357?src=pr=desc) into [master](https://codecov.io/gh/apache/incubator-airflow/commit/701b877a2eb4aac151a643f6eb735abf303a4e5e?src=pr=desc) will **not change** coverage. > The diff coverage is `n/a`. [](https://codecov.io/gh/apache/incubator-airflow/pull/4357?src=pr=tree) ```diff @@ Coverage Diff @@ ## master#4357 +/- ## === Coverage 78.12% 78.12% === Files 202 202 Lines 1648316483 === Hits1287812878 Misses 3605 3605 ``` | [Impacted Files](https://codecov.io/gh/apache/incubator-airflow/pull/4357?src=pr=tree) | Coverage Δ | | |---|---|---| | [airflow/sensors/base\_sensor\_operator.py](https://codecov.io/gh/apache/incubator-airflow/pull/4357/diff?src=pr=tree#diff-YWlyZmxvdy9zZW5zb3JzL2Jhc2Vfc2Vuc29yX29wZXJhdG9yLnB5) | `97.87% <ø> (ø)` | :arrow_up: | | [airflow/executors/celery\_executor.py](https://codecov.io/gh/apache/incubator-airflow/pull/4357/diff?src=pr=tree#diff-YWlyZmxvdy9leGVjdXRvcnMvY2VsZXJ5X2V4ZWN1dG9yLnB5) | `76.92% <ø> (ø)` | :arrow_up: | -- [Continue to review full report at Codecov](https://codecov.io/gh/apache/incubator-airflow/pull/4357?src=pr=continue). > **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta) > `Δ = absolute (impact)`, `ø = not affected`, `? = missing data` > Powered by [Codecov](https://codecov.io/gh/apache/incubator-airflow/pull/4357?src=pr=footer). Last update [701b877...8badf67](https://codecov.io/gh/apache/incubator-airflow/pull/4357?src=pr=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments). This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] codecov-io edited a comment on issue #4357: [AIRFLOW-3557] Fix various typos
codecov-io edited a comment on issue #4357: [AIRFLOW-3557] Fix various typos URL: https://github.com/apache/incubator-airflow/pull/4357#issuecomment-449563258 # [Codecov](https://codecov.io/gh/apache/incubator-airflow/pull/4357?src=pr=h1) Report > Merging [#4357](https://codecov.io/gh/apache/incubator-airflow/pull/4357?src=pr=desc) into [master](https://codecov.io/gh/apache/incubator-airflow/commit/701b877a2eb4aac151a643f6eb735abf303a4e5e?src=pr=desc) will **not change** coverage. > The diff coverage is `n/a`. [](https://codecov.io/gh/apache/incubator-airflow/pull/4357?src=pr=tree) ```diff @@ Coverage Diff @@ ## master#4357 +/- ## === Coverage 78.12% 78.12% === Files 202 202 Lines 1648316483 === Hits1287812878 Misses 3605 3605 ``` | [Impacted Files](https://codecov.io/gh/apache/incubator-airflow/pull/4357?src=pr=tree) | Coverage Δ | | |---|---|---| | [airflow/sensors/base\_sensor\_operator.py](https://codecov.io/gh/apache/incubator-airflow/pull/4357/diff?src=pr=tree#diff-YWlyZmxvdy9zZW5zb3JzL2Jhc2Vfc2Vuc29yX29wZXJhdG9yLnB5) | `97.87% <ø> (ø)` | :arrow_up: | | [airflow/executors/celery\_executor.py](https://codecov.io/gh/apache/incubator-airflow/pull/4357/diff?src=pr=tree#diff-YWlyZmxvdy9leGVjdXRvcnMvY2VsZXJ5X2V4ZWN1dG9yLnB5) | `76.92% <ø> (ø)` | :arrow_up: | -- [Continue to review full report at Codecov](https://codecov.io/gh/apache/incubator-airflow/pull/4357?src=pr=continue). > **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta) > `Δ = absolute (impact)`, `ø = not affected`, `? = missing data` > Powered by [Codecov](https://codecov.io/gh/apache/incubator-airflow/pull/4357?src=pr=footer). Last update [701b877...8badf67](https://codecov.io/gh/apache/incubator-airflow/pull/4357?src=pr=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments). This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] codecov-io commented on issue #4358: [AIRFLOW-XXX] Correct instructions for Travis CI on fork
codecov-io commented on issue #4358: [AIRFLOW-XXX] Correct instructions for Travis CI on fork URL: https://github.com/apache/incubator-airflow/pull/4358#issuecomment-449563026 # [Codecov](https://codecov.io/gh/apache/incubator-airflow/pull/4358?src=pr=h1) Report > Merging [#4358](https://codecov.io/gh/apache/incubator-airflow/pull/4358?src=pr=desc) into [master](https://codecov.io/gh/apache/incubator-airflow/commit/701b877a2eb4aac151a643f6eb735abf303a4e5e?src=pr=desc) will **not change** coverage. > The diff coverage is `n/a`. [](https://codecov.io/gh/apache/incubator-airflow/pull/4358?src=pr=tree) ```diff @@ Coverage Diff @@ ## master#4358 +/- ## === Coverage 78.12% 78.12% === Files 202 202 Lines 1648316483 === Hits1287812878 Misses 3605 3605 ``` -- [Continue to review full report at Codecov](https://codecov.io/gh/apache/incubator-airflow/pull/4358?src=pr=continue). > **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta) > `Δ = absolute (impact)`, `ø = not affected`, `? = missing data` > Powered by [Codecov](https://codecov.io/gh/apache/incubator-airflow/pull/4358?src=pr=footer). Last update [701b877...b4aef93](https://codecov.io/gh/apache/incubator-airflow/pull/4358?src=pr=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments). This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] codecov-io edited a comment on issue #4358: [AIRFLOW-XXX] Correct instructions for Travis CI on fork
codecov-io edited a comment on issue #4358: [AIRFLOW-XXX] Correct instructions for Travis CI on fork URL: https://github.com/apache/incubator-airflow/pull/4358#issuecomment-449563026 # [Codecov](https://codecov.io/gh/apache/incubator-airflow/pull/4358?src=pr=h1) Report > Merging [#4358](https://codecov.io/gh/apache/incubator-airflow/pull/4358?src=pr=desc) into [master](https://codecov.io/gh/apache/incubator-airflow/commit/701b877a2eb4aac151a643f6eb735abf303a4e5e?src=pr=desc) will **not change** coverage. > The diff coverage is `n/a`. [](https://codecov.io/gh/apache/incubator-airflow/pull/4358?src=pr=tree) ```diff @@ Coverage Diff @@ ## master#4358 +/- ## === Coverage 78.12% 78.12% === Files 202 202 Lines 1648316483 === Hits1287812878 Misses 3605 3605 ``` -- [Continue to review full report at Codecov](https://codecov.io/gh/apache/incubator-airflow/pull/4358?src=pr=continue). > **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta) > `Δ = absolute (impact)`, `ø = not affected`, `? = missing data` > Powered by [Codecov](https://codecov.io/gh/apache/incubator-airflow/pull/4358?src=pr=footer). Last update [701b877...b4aef93](https://codecov.io/gh/apache/incubator-airflow/pull/4358?src=pr=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments). This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] BasPH opened a new pull request #4358: [AIRFLOW-XXX] Correct instructions for Travis CI on fork
BasPH opened a new pull request #4358: [AIRFLOW-XXX] Correct instructions for
Travis CI on fork
URL: https://github.com/apache/incubator-airflow/pull/4358
Make sure you have checked _all_ steps below.
### Jira
- [x] My PR addresses the following [Airflow
Jira](https://issues.apache.org/jira/browse/AIRFLOW/) issues and references
them in the PR title. For example, "\[AIRFLOW-XXX\] My Airflow PR"
- https://issues.apache.org/jira/browse/AIRFLOW-XXX
- In case you are fixing a typo in the documentation you can prepend your
commit with \[AIRFLOW-XXX\], code changes always need a Jira issue.
AIRFLOW-XXX
### Description
- [x] Here are some details about my PR, including screenshots of any UI
changes:
The instructions in CONTRIBUTING.md pointed to somebodies private Travis CI
setup, this PR corrects that.
### Tests
- [x] My PR adds the following unit tests __OR__ does not need testing for
this extremely good reason:
No tests required.
### Commits
- [x] My commits all reference Jira issues in their subject lines, and I
have squashed multiple commits if they address the same issue. In addition, my
commits follow the guidelines from "[How to write a good git commit
message](http://chris.beams.io/posts/git-commit/)":
1. Subject is separated from body by a blank line
1. Subject is limited to 50 characters (not including Jira issue reference)
1. Subject does not end with a period
1. Subject uses the imperative mood ("add", not "adding")
1. Body wraps at 72 characters
1. Body explains "what" and "why", not "how"
### Documentation
- [x] In case of new functionality, my PR adds documentation that describes
how to use it.
- When adding new operators/hooks/sensors, the autoclass documentation
generation needs to be added.
- All the public functions and the classes in the PR contain docstrings
that explain what it does
### Code Quality
- [x] Passes `flake8`
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] codecov-io edited a comment on issue #4356: [AIRFLOW-3556] Add cross join set downstream function
codecov-io edited a comment on issue #4356: [AIRFLOW-3556] Add cross join set downstream function URL: https://github.com/apache/incubator-airflow/pull/4356#issuecomment-449515658 # [Codecov](https://codecov.io/gh/apache/incubator-airflow/pull/4356?src=pr=h1) Report > Merging [#4356](https://codecov.io/gh/apache/incubator-airflow/pull/4356?src=pr=desc) into [master](https://codecov.io/gh/apache/incubator-airflow/commit/f2113906d052df818cf14be8c513fada91caf26a?src=pr=desc) will **increase** coverage by `62%`. > The diff coverage is `100%`. [](https://codecov.io/gh/apache/incubator-airflow/pull/4356?src=pr=tree) ```diff @@Coverage Diff @@ ## master#4356 +/- ## == + Coverage 16.13% 78.14% +62% == Files 202 202 Lines 1648316486 +3 == + Hits 266012883 +10223 + Misses 13823 3603 -10220 ``` | [Impacted Files](https://codecov.io/gh/apache/incubator-airflow/pull/4356?src=pr=tree) | Coverage Δ | | |---|---|---| | [airflow/utils/helpers.py](https://codecov.io/gh/apache/incubator-airflow/pull/4356/diff?src=pr=tree#diff-YWlyZmxvdy91dGlscy9oZWxwZXJzLnB5) | `82.6% <100%> (+55.94%)` | :arrow_up: | | [airflow/plugins\_manager.py](https://codecov.io/gh/apache/incubator-airflow/pull/4356/diff?src=pr=tree#diff-YWlyZmxvdy9wbHVnaW5zX21hbmFnZXIucHk=) | `93.1% <0%> (+1.14%)` | :arrow_up: | | [airflow/executors/dask\_executor.py](https://codecov.io/gh/apache/incubator-airflow/pull/4356/diff?src=pr=tree#diff-YWlyZmxvdy9leGVjdXRvcnMvZGFza19leGVjdXRvci5weQ==) | `2% <0%> (+2%)` | :arrow_up: | | [airflow/exceptions.py](https://codecov.io/gh/apache/incubator-airflow/pull/4356/diff?src=pr=tree#diff-YWlyZmxvdy9leGNlcHRpb25zLnB5) | `100% <0%> (+2.85%)` | :arrow_up: | | [airflow/utils/operator\_resources.py](https://codecov.io/gh/apache/incubator-airflow/pull/4356/diff?src=pr=tree#diff-YWlyZmxvdy91dGlscy9vcGVyYXRvcl9yZXNvdXJjZXMucHk=) | `86.95% <0%> (+4.34%)` | :arrow_up: | | [airflow/executors/\_\_init\_\_.py](https://codecov.io/gh/apache/incubator-airflow/pull/4356/diff?src=pr=tree#diff-YWlyZmxvdy9leGVjdXRvcnMvX19pbml0X18ucHk=) | `63.46% <0%> (+5.76%)` | :arrow_up: | | [airflow/utils/module\_loading.py](https://codecov.io/gh/apache/incubator-airflow/pull/4356/diff?src=pr=tree#diff-YWlyZmxvdy91dGlscy9tb2R1bGVfbG9hZGluZy5weQ==) | `100% <0%> (+10.52%)` | :arrow_up: | | [airflow/utils/decorators.py](https://codecov.io/gh/apache/incubator-airflow/pull/4356/diff?src=pr=tree#diff-YWlyZmxvdy91dGlscy9kZWNvcmF0b3JzLnB5) | `91.66% <0%> (+14.58%)` | :arrow_up: | | [airflow/settings.py](https://codecov.io/gh/apache/incubator-airflow/pull/4356/diff?src=pr=tree#diff-YWlyZmxvdy9zZXR0aW5ncy5weQ==) | `80.41% <0%> (+14.68%)` | :arrow_up: | | [airflow/hooks/oracle\_hook.py](https://codecov.io/gh/apache/incubator-airflow/pull/4356/diff?src=pr=tree#diff-YWlyZmxvdy9ob29rcy9vcmFjbGVfaG9vay5weQ==) | `15.47% <0%> (+15.47%)` | :arrow_up: | | ... and [162 more](https://codecov.io/gh/apache/incubator-airflow/pull/4356/diff?src=pr=tree-more) | | -- [Continue to review full report at Codecov](https://codecov.io/gh/apache/incubator-airflow/pull/4356?src=pr=continue). > **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta) > `Δ = absolute (impact)`, `ø = not affected`, `? = missing data` > Powered by [Codecov](https://codecov.io/gh/apache/incubator-airflow/pull/4356?src=pr=footer). Last update [f211390...0bbf833](https://codecov.io/gh/apache/incubator-airflow/pull/4356?src=pr=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments). This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] BasPH opened a new pull request #4356: [AIRFLOW-3556] Add cross join set downstream function
BasPH opened a new pull request #4356: [AIRFLOW-3556] Add cross join set
downstream function
URL: https://github.com/apache/incubator-airflow/pull/4356
Make sure you have checked _all_ steps below.
### Jira
- [x] My PR addresses the following [Airflow
Jira](https://issues.apache.org/jira/browse/AIRFLOW/) issues and references
them in the PR title. For example, "\[AIRFLOW-XXX\] My Airflow PR"
- https://issues.apache.org/jira/browse/AIRFLOW-3556
- In case you are fixing a typo in the documentation you can prepend your
commit with \[AIRFLOW-XXX\], code changes always need a Jira issue.
### Description
- [x] Here are some details about my PR, including screenshots of any UI
changes:
Add function to set "cross join style" downstream dependencies between two
list of tasks. For example:
```
cross_downstream(from_tasks=[t1, t2, t3], to_tasks=[t4, t5, t6])
Sets dependencies:
t1 --> t4
\ /
t2 -X> t5
/ \
t3 --> t6
Equivalent to:
t1.set_downstream(t4)
t1.set_downstream(t5)
t1.set_downstream(t6)
t2.set_downstream(t4)
t2.set_downstream(t5)
t2.set_downstream(t6)
t3.set_downstream(t4)
t3.set_downstream(t5)
t3.set_downstream(t6)
```
### Tests
- [x] My PR adds the following unit tests __OR__ does not need testing for
this extremely good reason:
HelpersTest.test_cross_downstream()
### Commits
- [x] My commits all reference Jira issues in their subject lines, and I
have squashed multiple commits if they address the same issue. In addition, my
commits follow the guidelines from "[How to write a good git commit
message](http://chris.beams.io/posts/git-commit/)":
1. Subject is separated from body by a blank line
1. Subject is limited to 50 characters (not including Jira issue reference)
1. Subject does not end with a period
1. Subject uses the imperative mood ("add", not "adding")
1. Body wraps at 72 characters
1. Body explains "what" and "why", not "how"
### Documentation
- [x] In case of new functionality, my PR adds documentation that describes
how to use it.
- When adding new operators/hooks/sensors, the autoclass documentation
generation needs to be added.
- All the public functions and the classes in the PR contain docstrings
that explain what it does
### Code Quality
- [x] Passes `flake8`
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] BasPH opened a new pull request #4357: [AIRFLOW-3557] Fix various typos
BasPH opened a new pull request #4357: [AIRFLOW-3557] Fix various typos
URL: https://github.com/apache/incubator-airflow/pull/4357
Make sure you have checked _all_ steps below.
### Jira
- [x] My PR addresses the following [Airflow
Jira](https://issues.apache.org/jira/browse/AIRFLOW/) issues and references
them in the PR title. For example, "\[AIRFLOW-XXX\] My Airflow PR"
- https://issues.apache.org/jira/browse/AIRFLOW-3557
- In case you are fixing a typo in the documentation you can prepend your
commit with \[AIRFLOW-XXX\], code changes always need a Jira issue.
### Description
- [x] Here are some details about my PR, including screenshots of any UI
changes:
Check source code with [misspell](https://github.com/client9/misspell) and
fixed various typos.
### Tests
- [x] My PR adds the following unit tests __OR__ does not need testing for
this extremely good reason:
No changes to code.
### Commits
- [x] My commits all reference Jira issues in their subject lines, and I
have squashed multiple commits if they address the same issue. In addition, my
commits follow the guidelines from "[How to write a good git commit
message](http://chris.beams.io/posts/git-commit/)":
1. Subject is separated from body by a blank line
1. Subject is limited to 50 characters (not including Jira issue reference)
1. Subject does not end with a period
1. Subject uses the imperative mood ("add", not "adding")
1. Body wraps at 72 characters
1. Body explains "what" and "why", not "how"
### Documentation
- [x] In case of new functionality, my PR adds documentation that describes
how to use it.
- When adding new operators/hooks/sensors, the autoclass documentation
generation needs to be added.
- All the public functions and the classes in the PR contain docstrings
that explain what it does
No new functionality.
### Code Quality
- [x] Passes `flake8`
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] BasPH closed pull request #4356: [AIRFLOW-3556] Add cross join set downstream function
BasPH closed pull request #4356: [AIRFLOW-3556] Add cross join set downstream
function
URL: https://github.com/apache/incubator-airflow/pull/4356
This is a PR merged from a forked repository.
As GitHub hides the original diff on merge, it is displayed below for
the sake of provenance:
As this is a foreign pull request (from a fork), the diff is supplied
below (as it won't show otherwise due to GitHub magic):
diff --git a/airflow/utils/helpers.py b/airflow/utils/helpers.py
index 328147c1cf..5f8c88879c 100644
--- a/airflow/utils/helpers.py
+++ b/airflow/utils/helpers.py
@@ -169,6 +169,37 @@ def chain(*tasks):
up_task.set_downstream(down_task)
+def cross_downstream(from_tasks, to_tasks):
+"""
+Set downstream dependencies for all tasks in from_tasks to all tasks in
to_tasks.
+E.g.: cross_downstream(from_tasks=[t1, t2, t3], to_tasks=[t4, t5, t6])
+Is equivalent to:
+
+t1 --> t4
+ \ /
+t2 -X> t5
+ / \
+t3 --> t6
+
+t1.set_downstream(t4)
+t1.set_downstream(t5)
+t1.set_downstream(t6)
+t2.set_downstream(t4)
+t2.set_downstream(t5)
+t2.set_downstream(t6)
+t3.set_downstream(t4)
+t3.set_downstream(t5)
+t3.set_downstream(t6)
+
+:param from_tasks: List of tasks to start from.
+:type from_tasks: List[airflow.models.BaseOperator]
+:param to_tasks: List of tasks to set as downstream dependencies.
+:type to_tasks: List[airflow.models.BaseOperator]
+"""
+for task in from_tasks:
+task.set_downstream(to_tasks)
+
+
def pprinttable(rows):
"""Returns a pretty ascii table from tuples
diff --git a/tests/utils/test_helpers.py b/tests/utils/test_helpers.py
index 4cb3e1a1fc..837a79acba 100644
--- a/tests/utils/test_helpers.py
+++ b/tests/utils/test_helpers.py
@@ -20,11 +20,16 @@
import logging
import multiprocessing
import os
-import psutil
import signal
import time
import unittest
+from datetime import datetime
+
+import psutil
+import six
+from airflow import DAG
+from airflow.operators.dummy_operator import DummyOperator
from airflow.utils import helpers
@@ -210,6 +215,16 @@ def test_is_container(self):
# Pass an object that is not iter nor a string.
self.assertFalse(helpers.is_container(10))
+def test_cross_downstream(self):
+"""Test if all dependencies between tasks are all set correctly."""
+dag = DAG(dag_id="test_dag", start_date=datetime.now())
+start_tasks = [DummyOperator(task_id="t{i}".format(i=i), dag=dag) for
i in range(1, 4)]
+end_tasks = [DummyOperator(task_id="t{i}".format(i=i), dag=dag) for i
in range(4, 7)]
+helpers.cross_downstream(from_tasks=start_tasks, to_tasks=end_tasks)
+
+for start_task in start_tasks:
+six.assertCountEqual(self,
start_task.get_direct_relatives(upstream=False), end_tasks)
+
if __name__ == '__main__':
unittest.main()
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] feng-tao closed pull request #4352: [AIRFLOW-3555] Remove lxml dependency
feng-tao closed pull request #4352: [AIRFLOW-3555] Remove lxml dependency URL: https://github.com/apache/incubator-airflow/pull/4352 This is a PR merged from a forked repository. As GitHub hides the original diff on merge, it is displayed below for the sake of provenance: As this is a foreign pull request (from a fork), the diff is supplied below (as it won't show otherwise due to GitHub magic): diff --git a/setup.py b/setup.py index 56b9bb2873..f1e97d6b8f 100644 --- a/setup.py +++ b/setup.py @@ -316,7 +316,6 @@ def do_setup(): 'iso8601>=0.1.12', 'json-merge-patch==0.2', 'jinja2>=2.7.3, <=2.10.0', -'lxml>=4.0.0', 'markdown>=2.5.2, <3.0', 'pandas>=0.17.1, <1.0.0', 'pendulum==1.4.4', This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] feng-tao commented on issue #4352: [AIRFLOW-3555] Remove lxml dependency
feng-tao commented on issue #4352: [AIRFLOW-3555] Remove lxml dependency URL: https://github.com/apache/incubator-airflow/pull/4352#issuecomment-449547301 Thanks @jcao219 This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] codecov-io edited a comment on issue #4349: [AIRFLOW-850] Add a PythonSensor
codecov-io edited a comment on issue #4349: [AIRFLOW-850] Add a PythonSensor URL: https://github.com/apache/incubator-airflow/pull/4349#issuecomment-449545622 # [Codecov](https://codecov.io/gh/apache/incubator-airflow/pull/4349?src=pr=h1) Report > Merging [#4349](https://codecov.io/gh/apache/incubator-airflow/pull/4349?src=pr=desc) into [master](https://codecov.io/gh/apache/incubator-airflow/commit/f2113906d052df818cf14be8c513fada91caf26a?src=pr=desc) will **increase** coverage by `61.98%`. > The diff coverage is `n/a`. [](https://codecov.io/gh/apache/incubator-airflow/pull/4349?src=pr=tree) ```diff @@ Coverage Diff @@ ## master#4349 +/- ## === + Coverage 16.13% 78.12% +61.98% === Files 202 202 Lines 1648316483 === + Hits 266012877+10217 + Misses 13823 3606-10217 ``` | [Impacted Files](https://codecov.io/gh/apache/incubator-airflow/pull/4349?src=pr=tree) | Coverage Δ | | |---|---|---| | [airflow/plugins\_manager.py](https://codecov.io/gh/apache/incubator-airflow/pull/4349/diff?src=pr=tree#diff-YWlyZmxvdy9wbHVnaW5zX21hbmFnZXIucHk=) | `93.1% <0%> (+1.14%)` | :arrow_up: | | [airflow/executors/dask\_executor.py](https://codecov.io/gh/apache/incubator-airflow/pull/4349/diff?src=pr=tree#diff-YWlyZmxvdy9leGVjdXRvcnMvZGFza19leGVjdXRvci5weQ==) | `2% <0%> (+2%)` | :arrow_up: | | [airflow/exceptions.py](https://codecov.io/gh/apache/incubator-airflow/pull/4349/diff?src=pr=tree#diff-YWlyZmxvdy9leGNlcHRpb25zLnB5) | `100% <0%> (+2.85%)` | :arrow_up: | | [airflow/utils/operator\_resources.py](https://codecov.io/gh/apache/incubator-airflow/pull/4349/diff?src=pr=tree#diff-YWlyZmxvdy91dGlscy9vcGVyYXRvcl9yZXNvdXJjZXMucHk=) | `86.95% <0%> (+4.34%)` | :arrow_up: | | [airflow/executors/\_\_init\_\_.py](https://codecov.io/gh/apache/incubator-airflow/pull/4349/diff?src=pr=tree#diff-YWlyZmxvdy9leGVjdXRvcnMvX19pbml0X18ucHk=) | `63.46% <0%> (+5.76%)` | :arrow_up: | | [airflow/utils/module\_loading.py](https://codecov.io/gh/apache/incubator-airflow/pull/4349/diff?src=pr=tree#diff-YWlyZmxvdy91dGlscy9tb2R1bGVfbG9hZGluZy5weQ==) | `100% <0%> (+10.52%)` | :arrow_up: | | [airflow/utils/decorators.py](https://codecov.io/gh/apache/incubator-airflow/pull/4349/diff?src=pr=tree#diff-YWlyZmxvdy91dGlscy9kZWNvcmF0b3JzLnB5) | `91.66% <0%> (+14.58%)` | :arrow_up: | | [airflow/settings.py](https://codecov.io/gh/apache/incubator-airflow/pull/4349/diff?src=pr=tree#diff-YWlyZmxvdy9zZXR0aW5ncy5weQ==) | `80.41% <0%> (+14.68%)` | :arrow_up: | | [airflow/hooks/oracle\_hook.py](https://codecov.io/gh/apache/incubator-airflow/pull/4349/diff?src=pr=tree#diff-YWlyZmxvdy9ob29rcy9vcmFjbGVfaG9vay5weQ==) | `15.47% <0%> (+15.47%)` | :arrow_up: | | [airflow/\_\_init\_\_.py](https://codecov.io/gh/apache/incubator-airflow/pull/4349/diff?src=pr=tree#diff-YWlyZmxvdy9fX2luaXRfXy5weQ==) | `80% <0%> (+15.55%)` | :arrow_up: | | ... and [161 more](https://codecov.io/gh/apache/incubator-airflow/pull/4349/diff?src=pr=tree-more) | | -- [Continue to review full report at Codecov](https://codecov.io/gh/apache/incubator-airflow/pull/4349?src=pr=continue). > **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta) > `Δ = absolute (impact)`, `ø = not affected`, `? = missing data` > Powered by [Codecov](https://codecov.io/gh/apache/incubator-airflow/pull/4349?src=pr=footer). Last update [f211390...21e8df4](https://codecov.io/gh/apache/incubator-airflow/pull/4349?src=pr=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments). This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] codecov-io edited a comment on issue #4349: [AIRFLOW-850] Add a PythonSensor
codecov-io edited a comment on issue #4349: [AIRFLOW-850] Add a PythonSensor URL: https://github.com/apache/incubator-airflow/pull/4349#issuecomment-449545622 # [Codecov](https://codecov.io/gh/apache/incubator-airflow/pull/4349?src=pr=h1) Report > Merging [#4349](https://codecov.io/gh/apache/incubator-airflow/pull/4349?src=pr=desc) into [master](https://codecov.io/gh/apache/incubator-airflow/commit/f2113906d052df818cf14be8c513fada91caf26a?src=pr=desc) will **increase** coverage by `61.98%`. > The diff coverage is `n/a`. [](https://codecov.io/gh/apache/incubator-airflow/pull/4349?src=pr=tree) ```diff @@ Coverage Diff @@ ## master#4349 +/- ## === + Coverage 16.13% 78.12% +61.98% === Files 202 202 Lines 1648316483 === + Hits 266012877+10217 + Misses 13823 3606-10217 ``` | [Impacted Files](https://codecov.io/gh/apache/incubator-airflow/pull/4349?src=pr=tree) | Coverage Δ | | |---|---|---| | [airflow/plugins\_manager.py](https://codecov.io/gh/apache/incubator-airflow/pull/4349/diff?src=pr=tree#diff-YWlyZmxvdy9wbHVnaW5zX21hbmFnZXIucHk=) | `93.1% <0%> (+1.14%)` | :arrow_up: | | [airflow/executors/dask\_executor.py](https://codecov.io/gh/apache/incubator-airflow/pull/4349/diff?src=pr=tree#diff-YWlyZmxvdy9leGVjdXRvcnMvZGFza19leGVjdXRvci5weQ==) | `2% <0%> (+2%)` | :arrow_up: | | [airflow/exceptions.py](https://codecov.io/gh/apache/incubator-airflow/pull/4349/diff?src=pr=tree#diff-YWlyZmxvdy9leGNlcHRpb25zLnB5) | `100% <0%> (+2.85%)` | :arrow_up: | | [airflow/utils/operator\_resources.py](https://codecov.io/gh/apache/incubator-airflow/pull/4349/diff?src=pr=tree#diff-YWlyZmxvdy91dGlscy9vcGVyYXRvcl9yZXNvdXJjZXMucHk=) | `86.95% <0%> (+4.34%)` | :arrow_up: | | [airflow/executors/\_\_init\_\_.py](https://codecov.io/gh/apache/incubator-airflow/pull/4349/diff?src=pr=tree#diff-YWlyZmxvdy9leGVjdXRvcnMvX19pbml0X18ucHk=) | `63.46% <0%> (+5.76%)` | :arrow_up: | | [airflow/utils/module\_loading.py](https://codecov.io/gh/apache/incubator-airflow/pull/4349/diff?src=pr=tree#diff-YWlyZmxvdy91dGlscy9tb2R1bGVfbG9hZGluZy5weQ==) | `100% <0%> (+10.52%)` | :arrow_up: | | [airflow/utils/decorators.py](https://codecov.io/gh/apache/incubator-airflow/pull/4349/diff?src=pr=tree#diff-YWlyZmxvdy91dGlscy9kZWNvcmF0b3JzLnB5) | `91.66% <0%> (+14.58%)` | :arrow_up: | | [airflow/settings.py](https://codecov.io/gh/apache/incubator-airflow/pull/4349/diff?src=pr=tree#diff-YWlyZmxvdy9zZXR0aW5ncy5weQ==) | `80.41% <0%> (+14.68%)` | :arrow_up: | | [airflow/hooks/oracle\_hook.py](https://codecov.io/gh/apache/incubator-airflow/pull/4349/diff?src=pr=tree#diff-YWlyZmxvdy9ob29rcy9vcmFjbGVfaG9vay5weQ==) | `15.47% <0%> (+15.47%)` | :arrow_up: | | [airflow/\_\_init\_\_.py](https://codecov.io/gh/apache/incubator-airflow/pull/4349/diff?src=pr=tree#diff-YWlyZmxvdy9fX2luaXRfXy5weQ==) | `80% <0%> (+15.55%)` | :arrow_up: | | ... and [161 more](https://codecov.io/gh/apache/incubator-airflow/pull/4349/diff?src=pr=tree-more) | | -- [Continue to review full report at Codecov](https://codecov.io/gh/apache/incubator-airflow/pull/4349?src=pr=continue). > **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta) > `Δ = absolute (impact)`, `ø = not affected`, `? = missing data` > Powered by [Codecov](https://codecov.io/gh/apache/incubator-airflow/pull/4349?src=pr=footer). Last update [f211390...21e8df4](https://codecov.io/gh/apache/incubator-airflow/pull/4349?src=pr=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments). This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] codecov-io commented on issue #4349: [AIRFLOW-850] Add a PythonSensor
codecov-io commented on issue #4349: [AIRFLOW-850] Add a PythonSensor URL: https://github.com/apache/incubator-airflow/pull/4349#issuecomment-449545622 # [Codecov](https://codecov.io/gh/apache/incubator-airflow/pull/4349?src=pr=h1) Report > Merging [#4349](https://codecov.io/gh/apache/incubator-airflow/pull/4349?src=pr=desc) into [master](https://codecov.io/gh/apache/incubator-airflow/commit/f2113906d052df818cf14be8c513fada91caf26a?src=pr=desc) will **increase** coverage by `61.98%`. > The diff coverage is `n/a`. [](https://codecov.io/gh/apache/incubator-airflow/pull/4349?src=pr=tree) ```diff @@ Coverage Diff @@ ## master#4349 +/- ## === + Coverage 16.13% 78.12% +61.98% === Files 202 202 Lines 1648316483 === + Hits 266012877+10217 + Misses 13823 3606-10217 ``` | [Impacted Files](https://codecov.io/gh/apache/incubator-airflow/pull/4349?src=pr=tree) | Coverage Δ | | |---|---|---| | [airflow/plugins\_manager.py](https://codecov.io/gh/apache/incubator-airflow/pull/4349/diff?src=pr=tree#diff-YWlyZmxvdy9wbHVnaW5zX21hbmFnZXIucHk=) | `93.1% <0%> (+1.14%)` | :arrow_up: | | [airflow/executors/dask\_executor.py](https://codecov.io/gh/apache/incubator-airflow/pull/4349/diff?src=pr=tree#diff-YWlyZmxvdy9leGVjdXRvcnMvZGFza19leGVjdXRvci5weQ==) | `2% <0%> (+2%)` | :arrow_up: | | [airflow/exceptions.py](https://codecov.io/gh/apache/incubator-airflow/pull/4349/diff?src=pr=tree#diff-YWlyZmxvdy9leGNlcHRpb25zLnB5) | `100% <0%> (+2.85%)` | :arrow_up: | | [airflow/utils/operator\_resources.py](https://codecov.io/gh/apache/incubator-airflow/pull/4349/diff?src=pr=tree#diff-YWlyZmxvdy91dGlscy9vcGVyYXRvcl9yZXNvdXJjZXMucHk=) | `86.95% <0%> (+4.34%)` | :arrow_up: | | [airflow/executors/\_\_init\_\_.py](https://codecov.io/gh/apache/incubator-airflow/pull/4349/diff?src=pr=tree#diff-YWlyZmxvdy9leGVjdXRvcnMvX19pbml0X18ucHk=) | `63.46% <0%> (+5.76%)` | :arrow_up: | | [airflow/utils/module\_loading.py](https://codecov.io/gh/apache/incubator-airflow/pull/4349/diff?src=pr=tree#diff-YWlyZmxvdy91dGlscy9tb2R1bGVfbG9hZGluZy5weQ==) | `100% <0%> (+10.52%)` | :arrow_up: | | [airflow/utils/decorators.py](https://codecov.io/gh/apache/incubator-airflow/pull/4349/diff?src=pr=tree#diff-YWlyZmxvdy91dGlscy9kZWNvcmF0b3JzLnB5) | `91.66% <0%> (+14.58%)` | :arrow_up: | | [airflow/settings.py](https://codecov.io/gh/apache/incubator-airflow/pull/4349/diff?src=pr=tree#diff-YWlyZmxvdy9zZXR0aW5ncy5weQ==) | `80.41% <0%> (+14.68%)` | :arrow_up: | | [airflow/hooks/oracle\_hook.py](https://codecov.io/gh/apache/incubator-airflow/pull/4349/diff?src=pr=tree#diff-YWlyZmxvdy9ob29rcy9vcmFjbGVfaG9vay5weQ==) | `15.47% <0%> (+15.47%)` | :arrow_up: | | [airflow/\_\_init\_\_.py](https://codecov.io/gh/apache/incubator-airflow/pull/4349/diff?src=pr=tree#diff-YWlyZmxvdy9fX2luaXRfXy5weQ==) | `80% <0%> (+15.55%)` | :arrow_up: | | ... and [161 more](https://codecov.io/gh/apache/incubator-airflow/pull/4349/diff?src=pr=tree-more) | | -- [Continue to review full report at Codecov](https://codecov.io/gh/apache/incubator-airflow/pull/4349?src=pr=continue). > **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta) > `Δ = absolute (impact)`, `ø = not affected`, `? = missing data` > Powered by [Codecov](https://codecov.io/gh/apache/incubator-airflow/pull/4349?src=pr=footer). Last update [f211390...21e8df4](https://codecov.io/gh/apache/incubator-airflow/pull/4349?src=pr=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments). This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] feng-tao commented on a change in pull request #4349: [AIRFLOW-850] Add a PythonSensor
feng-tao commented on a change in pull request #4349: [AIRFLOW-850] Add a PythonSensor URL: https://github.com/apache/incubator-airflow/pull/4349#discussion_r243725398 ## File path: airflow/contrib/sensors/python_sensor.py ## @@ -0,0 +1,76 @@ +# -*- coding: utf-8 -*- +# +# Licensed to the Apache Software Foundation (ASF) under one +# or more contributor license agreements. See the NOTICE file +# distributed with this work for additional information +# regarding copyright ownership. The ASF licenses this file +# to you under the Apache License, Version 2.0 (the +# "License"); you may not use this file except in compliance +# with the License. You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, +# software distributed under the License is distributed on an +# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY +# KIND, either express or implied. See the License for the +# specific language governing permissions and limitations +# under the License. + +from airflow.sensors.base_sensor_operator import BaseSensorOperator +from airflow.utils.decorators import apply_defaults + + +class PythonSensor(BaseSensorOperator): +""" +Waits for a Python callable to return True Review comment: updated This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] feng-tao commented on issue #4349: [AIRFLOW-850] Add a PythonSensor
feng-tao commented on issue #4349: [AIRFLOW-850] Add a PythonSensor URL: https://github.com/apache/incubator-airflow/pull/4349#issuecomment-449543965 hey @ashb , update the pr with an example. PTAL. This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] feng-tao commented on a change in pull request #4349: [AIRFLOW-850] Add a PythonSensor
feng-tao commented on a change in pull request #4349: [AIRFLOW-850] Add a
PythonSensor
URL: https://github.com/apache/incubator-airflow/pull/4349#discussion_r243725161
##
File path: airflow/contrib/sensors/python_sensor.py
##
@@ -0,0 +1,76 @@
+# -*- coding: utf-8 -*-
+#
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+
+from airflow.sensors.base_sensor_operator import BaseSensorOperator
+from airflow.utils.decorators import apply_defaults
+
+
+class PythonSensor(BaseSensorOperator):
+"""
+Waits for a Python callable to return True
+
+:param python_callable: A reference to an object that is callable
+:type python_callable: python callable
+:param op_kwargs: a dictionary of keyword arguments that will get unpacked
+in your function
+:type op_kwargs: dict
+:param op_args: a list of positional arguments that will get unpacked when
+calling your callable
+:type op_args: list
+:param provide_context: if set to true, Airflow will pass a set of
+keyword arguments that can be used in your function. This set of
+kwargs correspond exactly to what you can use in your jinja
+templates. For this to work, you need to define `**kwargs` in your
+function header.
+:type provide_context: bool
+:param templates_dict: a dictionary where the values are templates that
+will get templated by the Airflow engine sometime between
+``__init__`` and ``execute`` takes place and are made available
+in your callable's context after the template has been applied
+:type templates_dict: dict of str
+"""
+
+template_fields = ('templates_dict',)
+template_ext = tuple()
+
+@apply_defaults
+def __init__(
+self,
+python_callable,
+op_args=None,
+op_kwargs=None,
+provide_context=False,
+templates_dict=None,
+*args, **kwargs):
+super(PythonSensor, self).__init__(*args, **kwargs)
+self.python_callable = python_callable
+self.op_args = op_args or []
+self.op_kwargs = op_kwargs or {}
+self.provide_context = provide_context
+self.templates_dict = templates_dict
+
+def poke(self, context):
+if self.provide_context:
+context.update(self.op_kwargs)
+context['templates_dict'] = self.templates_dict
+self.op_kwargs = context
Review comment:
I think we need to have a way to access the context for the sensor. Not sure
if there is a better way to do so. Besides, I think it is ok to do that as it
is not an expensive operation.
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] stale[bot] closed pull request #3997: [AIRFLOW-3153] send dag last_run to statsd
stale[bot] closed pull request #3997: [AIRFLOW-3153] send dag last_run to statsd
URL: https://github.com/apache/incubator-airflow/pull/3997
This is a PR merged from a forked repository.
As GitHub hides the original diff on merge, it is displayed below for
the sake of provenance:
As this is a foreign pull request (from a fork), the diff is supplied
below (as it won't show otherwise due to GitHub magic):
diff --git a/airflow/jobs.py b/airflow/jobs.py
index da1089d690..94ec4458d8 100644
--- a/airflow/jobs.py
+++ b/airflow/jobs.py
@@ -580,7 +580,8 @@ def __init__(
self.using_sqlite = False
if 'sqlite' in conf.get('core', 'sql_alchemy_conn'):
if self.max_threads > 1:
-self.log.error("Cannot use more than 1 thread when using
sqlite. Setting max_threads to 1")
+self.log.error("Cannot use more than 1 thread when using
sqlite. "
+ "Setting max_threads to 1")
self.max_threads = 1
self.using_sqlite = True
@@ -1026,7 +1027,8 @@ def _change_state_for_tis_without_dagrun(self,
if tis_changed > 0:
self.log.warning(
-"Set %s task instances to state=%s as their associated DagRun
was not in RUNNING state",
+"Set %s task instances to state=%s "
+"as their associated DagRun was not in RUNNING state",
tis_changed, new_state
)
@@ -1201,7 +1203,8 @@ def _find_executable_task_instances(self, simple_dag_bag,
states, session=None):
" this task has been reached.",
task_instance)
continue
else:
-task_concurrency_map[(task_instance.dag_id,
task_instance.task_id)] += 1
+task_concurrency_map[(task_instance.dag_id,
+ task_instance.task_id)] += 1
if self.executor.has_task(task_instance):
self.log.debug(
@@ -1505,6 +1508,8 @@ def _log_file_processing_stats(self,
"Last Run"]
rows = []
+dags_folder = conf.get('core', 'dags_folder').rstrip(os.sep)
+
for file_path in known_file_paths:
last_runtime = processor_manager.get_last_runtime(file_path)
processor_pid = processor_manager.get_pid(file_path)
@@ -1513,6 +1518,16 @@ def _log_file_processing_stats(self,
if processor_start_time else None)
last_run = processor_manager.get_last_finish_time(file_path)
+file_name = file_path[len(dags_folder) + 1:]
+dag_name = os.path.splitext(file_name)[0].replace(os.sep, '.')
+if last_runtime is not None:
+Stats.gauge('last_runtime.{}'.format(dag_name), last_runtime)
+if last_run is not None:
+unixtime = last_run.strftime("%s")
+seconds_ago = (timezone.utcnow() - last_run).total_seconds()
+Stats.gauge('last_run.unixtime.{}'.format(dag_name), unixtime)
+Stats.gauge('last_run.seconds_ago.{}'.format(dag_name),
seconds_ago)
+
rows.append((file_path,
processor_pid,
runtime,
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] stale[bot] commented on issue #4143: [AIRFLOW-689] Okta Authentication
stale[bot] commented on issue #4143: [AIRFLOW-689] Okta Authentication URL: https://github.com/apache/incubator-airflow/pull/4143#issuecomment-449538342 This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions. This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] mdziemianko commented on issue #2450: [Airflow-1413] Fix FTPSensor failing on error message with unexpected text.
mdziemianko commented on issue #2450: [Airflow-1413] Fix FTPSensor failing on error message with unexpected text. URL: https://github.com/apache/incubator-airflow/pull/2450#issuecomment-449516038 seems like all checks passed...:) This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] codecov-io commented on issue #4356: [AIRFLOW-3556] Add cross join set downstream function
codecov-io commented on issue #4356: [AIRFLOW-3556] Add cross join set downstream function URL: https://github.com/apache/incubator-airflow/pull/4356#issuecomment-449515658 # [Codecov](https://codecov.io/gh/apache/incubator-airflow/pull/4356?src=pr=h1) Report > Merging [#4356](https://codecov.io/gh/apache/incubator-airflow/pull/4356?src=pr=desc) into [master](https://codecov.io/gh/apache/incubator-airflow/commit/f2113906d052df818cf14be8c513fada91caf26a?src=pr=desc) will **increase** coverage by `62%`. > The diff coverage is `100%`. [](https://codecov.io/gh/apache/incubator-airflow/pull/4356?src=pr=tree) ```diff @@Coverage Diff @@ ## master#4356 +/- ## == + Coverage 16.13% 78.14% +62% == Files 202 202 Lines 1648316486 +3 == + Hits 266012883 +10223 + Misses 13823 3603 -10220 ``` | [Impacted Files](https://codecov.io/gh/apache/incubator-airflow/pull/4356?src=pr=tree) | Coverage Δ | | |---|---|---| | [airflow/utils/helpers.py](https://codecov.io/gh/apache/incubator-airflow/pull/4356/diff?src=pr=tree#diff-YWlyZmxvdy91dGlscy9oZWxwZXJzLnB5) | `82.6% <100%> (+55.94%)` | :arrow_up: | | [airflow/plugins\_manager.py](https://codecov.io/gh/apache/incubator-airflow/pull/4356/diff?src=pr=tree#diff-YWlyZmxvdy9wbHVnaW5zX21hbmFnZXIucHk=) | `93.1% <0%> (+1.14%)` | :arrow_up: | | [airflow/executors/dask\_executor.py](https://codecov.io/gh/apache/incubator-airflow/pull/4356/diff?src=pr=tree#diff-YWlyZmxvdy9leGVjdXRvcnMvZGFza19leGVjdXRvci5weQ==) | `2% <0%> (+2%)` | :arrow_up: | | [airflow/exceptions.py](https://codecov.io/gh/apache/incubator-airflow/pull/4356/diff?src=pr=tree#diff-YWlyZmxvdy9leGNlcHRpb25zLnB5) | `100% <0%> (+2.85%)` | :arrow_up: | | [airflow/utils/operator\_resources.py](https://codecov.io/gh/apache/incubator-airflow/pull/4356/diff?src=pr=tree#diff-YWlyZmxvdy91dGlscy9vcGVyYXRvcl9yZXNvdXJjZXMucHk=) | `86.95% <0%> (+4.34%)` | :arrow_up: | | [airflow/executors/\_\_init\_\_.py](https://codecov.io/gh/apache/incubator-airflow/pull/4356/diff?src=pr=tree#diff-YWlyZmxvdy9leGVjdXRvcnMvX19pbml0X18ucHk=) | `63.46% <0%> (+5.76%)` | :arrow_up: | | [airflow/utils/module\_loading.py](https://codecov.io/gh/apache/incubator-airflow/pull/4356/diff?src=pr=tree#diff-YWlyZmxvdy91dGlscy9tb2R1bGVfbG9hZGluZy5weQ==) | `100% <0%> (+10.52%)` | :arrow_up: | | [airflow/utils/decorators.py](https://codecov.io/gh/apache/incubator-airflow/pull/4356/diff?src=pr=tree#diff-YWlyZmxvdy91dGlscy9kZWNvcmF0b3JzLnB5) | `91.66% <0%> (+14.58%)` | :arrow_up: | | [airflow/settings.py](https://codecov.io/gh/apache/incubator-airflow/pull/4356/diff?src=pr=tree#diff-YWlyZmxvdy9zZXR0aW5ncy5weQ==) | `80.41% <0%> (+14.68%)` | :arrow_up: | | [airflow/hooks/oracle\_hook.py](https://codecov.io/gh/apache/incubator-airflow/pull/4356/diff?src=pr=tree#diff-YWlyZmxvdy9ob29rcy9vcmFjbGVfaG9vay5weQ==) | `15.47% <0%> (+15.47%)` | :arrow_up: | | ... and [162 more](https://codecov.io/gh/apache/incubator-airflow/pull/4356/diff?src=pr=tree-more) | | -- [Continue to review full report at Codecov](https://codecov.io/gh/apache/incubator-airflow/pull/4356?src=pr=continue). > **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta) > `Δ = absolute (impact)`, `ø = not affected`, `? = missing data` > Powered by [Codecov](https://codecov.io/gh/apache/incubator-airflow/pull/4356?src=pr=footer). Last update [f211390...0bbf833](https://codecov.io/gh/apache/incubator-airflow/pull/4356?src=pr=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments). This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] codecov-io edited a comment on issue #4356: [AIRFLOW-3556] Add cross join set downstream function
codecov-io edited a comment on issue #4356: [AIRFLOW-3556] Add cross join set downstream function URL: https://github.com/apache/incubator-airflow/pull/4356#issuecomment-449515658 # [Codecov](https://codecov.io/gh/apache/incubator-airflow/pull/4356?src=pr=h1) Report > Merging [#4356](https://codecov.io/gh/apache/incubator-airflow/pull/4356?src=pr=desc) into [master](https://codecov.io/gh/apache/incubator-airflow/commit/f2113906d052df818cf14be8c513fada91caf26a?src=pr=desc) will **increase** coverage by `62%`. > The diff coverage is `100%`. [](https://codecov.io/gh/apache/incubator-airflow/pull/4356?src=pr=tree) ```diff @@Coverage Diff @@ ## master#4356 +/- ## == + Coverage 16.13% 78.14% +62% == Files 202 202 Lines 1648316486 +3 == + Hits 266012883 +10223 + Misses 13823 3603 -10220 ``` | [Impacted Files](https://codecov.io/gh/apache/incubator-airflow/pull/4356?src=pr=tree) | Coverage Δ | | |---|---|---| | [airflow/utils/helpers.py](https://codecov.io/gh/apache/incubator-airflow/pull/4356/diff?src=pr=tree#diff-YWlyZmxvdy91dGlscy9oZWxwZXJzLnB5) | `82.6% <100%> (+55.94%)` | :arrow_up: | | [airflow/plugins\_manager.py](https://codecov.io/gh/apache/incubator-airflow/pull/4356/diff?src=pr=tree#diff-YWlyZmxvdy9wbHVnaW5zX21hbmFnZXIucHk=) | `93.1% <0%> (+1.14%)` | :arrow_up: | | [airflow/executors/dask\_executor.py](https://codecov.io/gh/apache/incubator-airflow/pull/4356/diff?src=pr=tree#diff-YWlyZmxvdy9leGVjdXRvcnMvZGFza19leGVjdXRvci5weQ==) | `2% <0%> (+2%)` | :arrow_up: | | [airflow/exceptions.py](https://codecov.io/gh/apache/incubator-airflow/pull/4356/diff?src=pr=tree#diff-YWlyZmxvdy9leGNlcHRpb25zLnB5) | `100% <0%> (+2.85%)` | :arrow_up: | | [airflow/utils/operator\_resources.py](https://codecov.io/gh/apache/incubator-airflow/pull/4356/diff?src=pr=tree#diff-YWlyZmxvdy91dGlscy9vcGVyYXRvcl9yZXNvdXJjZXMucHk=) | `86.95% <0%> (+4.34%)` | :arrow_up: | | [airflow/executors/\_\_init\_\_.py](https://codecov.io/gh/apache/incubator-airflow/pull/4356/diff?src=pr=tree#diff-YWlyZmxvdy9leGVjdXRvcnMvX19pbml0X18ucHk=) | `63.46% <0%> (+5.76%)` | :arrow_up: | | [airflow/utils/module\_loading.py](https://codecov.io/gh/apache/incubator-airflow/pull/4356/diff?src=pr=tree#diff-YWlyZmxvdy91dGlscy9tb2R1bGVfbG9hZGluZy5weQ==) | `100% <0%> (+10.52%)` | :arrow_up: | | [airflow/utils/decorators.py](https://codecov.io/gh/apache/incubator-airflow/pull/4356/diff?src=pr=tree#diff-YWlyZmxvdy91dGlscy9kZWNvcmF0b3JzLnB5) | `91.66% <0%> (+14.58%)` | :arrow_up: | | [airflow/settings.py](https://codecov.io/gh/apache/incubator-airflow/pull/4356/diff?src=pr=tree#diff-YWlyZmxvdy9zZXR0aW5ncy5weQ==) | `80.41% <0%> (+14.68%)` | :arrow_up: | | [airflow/hooks/oracle\_hook.py](https://codecov.io/gh/apache/incubator-airflow/pull/4356/diff?src=pr=tree#diff-YWlyZmxvdy9ob29rcy9vcmFjbGVfaG9vay5weQ==) | `15.47% <0%> (+15.47%)` | :arrow_up: | | ... and [162 more](https://codecov.io/gh/apache/incubator-airflow/pull/4356/diff?src=pr=tree-more) | | -- [Continue to review full report at Codecov](https://codecov.io/gh/apache/incubator-airflow/pull/4356?src=pr=continue). > **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta) > `Δ = absolute (impact)`, `ø = not affected`, `? = missing data` > Powered by [Codecov](https://codecov.io/gh/apache/incubator-airflow/pull/4356?src=pr=footer). Last update [f211390...0bbf833](https://codecov.io/gh/apache/incubator-airflow/pull/4356?src=pr=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments). This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] ohld commented on issue #4352: [AIRFLOW-3555] Remove lxml dependency
ohld commented on issue #4352: [AIRFLOW-3555] Remove lxml dependency URL: https://github.com/apache/incubator-airflow/pull/4352#issuecomment-449513366 LGTM This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] codecov-io edited a comment on issue #3533: [AIRFLOW-161] New redirect route and extra links
codecov-io edited a comment on issue #3533: [AIRFLOW-161] New redirect route and extra links URL: https://github.com/apache/incubator-airflow/pull/3533#issuecomment-400106002 # [Codecov](https://codecov.io/gh/apache/incubator-airflow/pull/3533?src=pr=h1) Report > Merging [#3533](https://codecov.io/gh/apache/incubator-airflow/pull/3533?src=pr=desc) into [master](https://codecov.io/gh/apache/incubator-airflow/commit/e703d6beeb379ee88ef5e7df495e8a785666f8af?src=pr=desc) will **increase** coverage by `1.06%`. > The diff coverage is `91.04%`. [](https://codecov.io/gh/apache/incubator-airflow/pull/3533?src=pr=tree) ```diff @@Coverage Diff @@ ## master#3533 +/- ## == + Coverage 76.67% 77.73% +1.06% == Files 199 204 +5 Lines 1618615913 -273 == - Hits1241012370 -40 + Misses 3776 3543 -233 ``` | [Impacted Files](https://codecov.io/gh/apache/incubator-airflow/pull/3533?src=pr=tree) | Coverage Δ | | |---|---|---| | [airflow/models.py](https://codecov.io/gh/apache/incubator-airflow/pull/3533/diff?src=pr=tree#diff-YWlyZmxvdy9tb2RlbHMucHk=) | `88.8% <100%> (-3.25%)` | :arrow_down: | | [airflow/www/views.py](https://codecov.io/gh/apache/incubator-airflow/pull/3533/diff?src=pr=tree#diff-YWlyZmxvdy93d3cvdmlld3MucHk=) | `69.45% <90.62%> (+0.72%)` | :arrow_up: | | [airflow/www\_rbac/views.py](https://codecov.io/gh/apache/incubator-airflow/pull/3533/diff?src=pr=tree#diff-YWlyZmxvdy93d3dfcmJhYy92aWV3cy5weQ==) | `73.14% <90.62%> (+0.75%)` | :arrow_up: | | [airflow/operators/slack\_operator.py](https://codecov.io/gh/apache/incubator-airflow/pull/3533/diff?src=pr=tree#diff-YWlyZmxvdy9vcGVyYXRvcnMvc2xhY2tfb3BlcmF0b3IucHk=) | `0% <0%> (-97.37%)` | :arrow_down: | | [airflow/sensors/s3\_key\_sensor.py](https://codecov.io/gh/apache/incubator-airflow/pull/3533/diff?src=pr=tree#diff-YWlyZmxvdy9zZW5zb3JzL3MzX2tleV9zZW5zb3IucHk=) | `31.03% <0%> (-68.97%)` | :arrow_down: | | [airflow/sensors/s3\_prefix\_sensor.py](https://codecov.io/gh/apache/incubator-airflow/pull/3533/diff?src=pr=tree#diff-YWlyZmxvdy9zZW5zb3JzL3MzX3ByZWZpeF9zZW5zb3IucHk=) | `41.17% <0%> (-58.83%)` | :arrow_down: | | [airflow/example\_dags/example\_python\_operator.py](https://codecov.io/gh/apache/incubator-airflow/pull/3533/diff?src=pr=tree#diff-YWlyZmxvdy9leGFtcGxlX2RhZ3MvZXhhbXBsZV9weXRob25fb3BlcmF0b3IucHk=) | `78.94% <0%> (-15.79%)` | :arrow_down: | | [airflow/utils/helpers.py](https://codecov.io/gh/apache/incubator-airflow/pull/3533/diff?src=pr=tree#diff-YWlyZmxvdy91dGlscy9oZWxwZXJzLnB5) | `71.34% <0%> (-13.04%)` | :arrow_down: | | [airflow/hooks/mysql\_hook.py](https://codecov.io/gh/apache/incubator-airflow/pull/3533/diff?src=pr=tree#diff-YWlyZmxvdy9ob29rcy9teXNxbF9ob29rLnB5) | `78% <0%> (-12%)` | :arrow_down: | | [airflow/sensors/sql\_sensor.py](https://codecov.io/gh/apache/incubator-airflow/pull/3533/diff?src=pr=tree#diff-YWlyZmxvdy9zZW5zb3JzL3NxbF9zZW5zb3IucHk=) | `90.47% <0%> (-9.53%)` | :arrow_down: | | ... and [86 more](https://codecov.io/gh/apache/incubator-airflow/pull/3533/diff?src=pr=tree-more) | | -- [Continue to review full report at Codecov](https://codecov.io/gh/apache/incubator-airflow/pull/3533?src=pr=continue). > **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta) > `Δ = absolute (impact)`, `ø = not affected`, `? = missing data` > Powered by [Codecov](https://codecov.io/gh/apache/incubator-airflow/pull/3533?src=pr=footer). Last update [e703d6b...80a042d](https://codecov.io/gh/apache/incubator-airflow/pull/3533?src=pr=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments). This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] codecov-io edited a comment on issue #4353: [AIRFLOW-3480] Added Database Deploy/Update/Delete operators
codecov-io edited a comment on issue #4353: [AIRFLOW-3480] Added Database Deploy/Update/Delete operators URL: https://github.com/apache/incubator-airflow/pull/4353#issuecomment-449454398 # [Codecov](https://codecov.io/gh/apache/incubator-airflow/pull/4353?src=pr=h1) Report > Merging [#4353](https://codecov.io/gh/apache/incubator-airflow/pull/4353?src=pr=desc) into [master](https://codecov.io/gh/apache/incubator-airflow/commit/f2113906d052df818cf14be8c513fada91caf26a?src=pr=desc) will **increase** coverage by `61.99%`. > The diff coverage is `n/a`. [](https://codecov.io/gh/apache/incubator-airflow/pull/4353?src=pr=tree) ```diff @@ Coverage Diff @@ ## master#4353 +/- ## === + Coverage 16.13% 78.12% +61.99% === Files 202 202 Lines 1648316483 === + Hits 266012878+10218 + Misses 13823 3605-10218 ``` | [Impacted Files](https://codecov.io/gh/apache/incubator-airflow/pull/4353?src=pr=tree) | Coverage Δ | | |---|---|---| | [airflow/plugins\_manager.py](https://codecov.io/gh/apache/incubator-airflow/pull/4353/diff?src=pr=tree#diff-YWlyZmxvdy9wbHVnaW5zX21hbmFnZXIucHk=) | `93.1% <0%> (+1.14%)` | :arrow_up: | | [airflow/executors/dask\_executor.py](https://codecov.io/gh/apache/incubator-airflow/pull/4353/diff?src=pr=tree#diff-YWlyZmxvdy9leGVjdXRvcnMvZGFza19leGVjdXRvci5weQ==) | `2% <0%> (+2%)` | :arrow_up: | | [airflow/exceptions.py](https://codecov.io/gh/apache/incubator-airflow/pull/4353/diff?src=pr=tree#diff-YWlyZmxvdy9leGNlcHRpb25zLnB5) | `100% <0%> (+2.85%)` | :arrow_up: | | [airflow/utils/operator\_resources.py](https://codecov.io/gh/apache/incubator-airflow/pull/4353/diff?src=pr=tree#diff-YWlyZmxvdy91dGlscy9vcGVyYXRvcl9yZXNvdXJjZXMucHk=) | `86.95% <0%> (+4.34%)` | :arrow_up: | | [airflow/executors/\_\_init\_\_.py](https://codecov.io/gh/apache/incubator-airflow/pull/4353/diff?src=pr=tree#diff-YWlyZmxvdy9leGVjdXRvcnMvX19pbml0X18ucHk=) | `63.46% <0%> (+5.76%)` | :arrow_up: | | [airflow/utils/module\_loading.py](https://codecov.io/gh/apache/incubator-airflow/pull/4353/diff?src=pr=tree#diff-YWlyZmxvdy91dGlscy9tb2R1bGVfbG9hZGluZy5weQ==) | `100% <0%> (+10.52%)` | :arrow_up: | | [airflow/utils/decorators.py](https://codecov.io/gh/apache/incubator-airflow/pull/4353/diff?src=pr=tree#diff-YWlyZmxvdy91dGlscy9kZWNvcmF0b3JzLnB5) | `91.66% <0%> (+14.58%)` | :arrow_up: | | [airflow/settings.py](https://codecov.io/gh/apache/incubator-airflow/pull/4353/diff?src=pr=tree#diff-YWlyZmxvdy9zZXR0aW5ncy5weQ==) | `80.41% <0%> (+14.68%)` | :arrow_up: | | [airflow/hooks/oracle\_hook.py](https://codecov.io/gh/apache/incubator-airflow/pull/4353/diff?src=pr=tree#diff-YWlyZmxvdy9ob29rcy9vcmFjbGVfaG9vay5weQ==) | `15.47% <0%> (+15.47%)` | :arrow_up: | | [airflow/\_\_init\_\_.py](https://codecov.io/gh/apache/incubator-airflow/pull/4353/diff?src=pr=tree#diff-YWlyZmxvdy9fX2luaXRfXy5weQ==) | `80% <0%> (+15.55%)` | :arrow_up: | | ... and [161 more](https://codecov.io/gh/apache/incubator-airflow/pull/4353/diff?src=pr=tree-more) | | -- [Continue to review full report at Codecov](https://codecov.io/gh/apache/incubator-airflow/pull/4353?src=pr=continue). > **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta) > `Δ = absolute (impact)`, `ø = not affected`, `? = missing data` > Powered by [Codecov](https://codecov.io/gh/apache/incubator-airflow/pull/4353?src=pr=footer). Last update [f211390...e1c9a24](https://codecov.io/gh/apache/incubator-airflow/pull/4353?src=pr=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments). This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] codecov-io edited a comment on issue #4353: [AIRFLOW-3480] Added Database Deploy/Update/Delete operators
codecov-io edited a comment on issue #4353: [AIRFLOW-3480] Added Database Deploy/Update/Delete operators URL: https://github.com/apache/incubator-airflow/pull/4353#issuecomment-449454398 # [Codecov](https://codecov.io/gh/apache/incubator-airflow/pull/4353?src=pr=h1) Report > Merging [#4353](https://codecov.io/gh/apache/incubator-airflow/pull/4353?src=pr=desc) into [master](https://codecov.io/gh/apache/incubator-airflow/commit/f2113906d052df818cf14be8c513fada91caf26a?src=pr=desc) will **increase** coverage by `61.99%`. > The diff coverage is `n/a`. [](https://codecov.io/gh/apache/incubator-airflow/pull/4353?src=pr=tree) ```diff @@ Coverage Diff @@ ## master#4353 +/- ## === + Coverage 16.13% 78.12% +61.99% === Files 202 202 Lines 1648316483 === + Hits 266012878+10218 + Misses 13823 3605-10218 ``` | [Impacted Files](https://codecov.io/gh/apache/incubator-airflow/pull/4353?src=pr=tree) | Coverage Δ | | |---|---|---| | [airflow/plugins\_manager.py](https://codecov.io/gh/apache/incubator-airflow/pull/4353/diff?src=pr=tree#diff-YWlyZmxvdy9wbHVnaW5zX21hbmFnZXIucHk=) | `93.1% <0%> (+1.14%)` | :arrow_up: | | [airflow/executors/dask\_executor.py](https://codecov.io/gh/apache/incubator-airflow/pull/4353/diff?src=pr=tree#diff-YWlyZmxvdy9leGVjdXRvcnMvZGFza19leGVjdXRvci5weQ==) | `2% <0%> (+2%)` | :arrow_up: | | [airflow/exceptions.py](https://codecov.io/gh/apache/incubator-airflow/pull/4353/diff?src=pr=tree#diff-YWlyZmxvdy9leGNlcHRpb25zLnB5) | `100% <0%> (+2.85%)` | :arrow_up: | | [airflow/utils/operator\_resources.py](https://codecov.io/gh/apache/incubator-airflow/pull/4353/diff?src=pr=tree#diff-YWlyZmxvdy91dGlscy9vcGVyYXRvcl9yZXNvdXJjZXMucHk=) | `86.95% <0%> (+4.34%)` | :arrow_up: | | [airflow/executors/\_\_init\_\_.py](https://codecov.io/gh/apache/incubator-airflow/pull/4353/diff?src=pr=tree#diff-YWlyZmxvdy9leGVjdXRvcnMvX19pbml0X18ucHk=) | `63.46% <0%> (+5.76%)` | :arrow_up: | | [airflow/utils/module\_loading.py](https://codecov.io/gh/apache/incubator-airflow/pull/4353/diff?src=pr=tree#diff-YWlyZmxvdy91dGlscy9tb2R1bGVfbG9hZGluZy5weQ==) | `100% <0%> (+10.52%)` | :arrow_up: | | [airflow/utils/decorators.py](https://codecov.io/gh/apache/incubator-airflow/pull/4353/diff?src=pr=tree#diff-YWlyZmxvdy91dGlscy9kZWNvcmF0b3JzLnB5) | `91.66% <0%> (+14.58%)` | :arrow_up: | | [airflow/settings.py](https://codecov.io/gh/apache/incubator-airflow/pull/4353/diff?src=pr=tree#diff-YWlyZmxvdy9zZXR0aW5ncy5weQ==) | `80.41% <0%> (+14.68%)` | :arrow_up: | | [airflow/hooks/oracle\_hook.py](https://codecov.io/gh/apache/incubator-airflow/pull/4353/diff?src=pr=tree#diff-YWlyZmxvdy9ob29rcy9vcmFjbGVfaG9vay5weQ==) | `15.47% <0%> (+15.47%)` | :arrow_up: | | [airflow/\_\_init\_\_.py](https://codecov.io/gh/apache/incubator-airflow/pull/4353/diff?src=pr=tree#diff-YWlyZmxvdy9fX2luaXRfXy5weQ==) | `80% <0%> (+15.55%)` | :arrow_up: | | ... and [161 more](https://codecov.io/gh/apache/incubator-airflow/pull/4353/diff?src=pr=tree-more) | | -- [Continue to review full report at Codecov](https://codecov.io/gh/apache/incubator-airflow/pull/4353?src=pr=continue). > **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta) > `Δ = absolute (impact)`, `ø = not affected`, `? = missing data` > Powered by [Codecov](https://codecov.io/gh/apache/incubator-airflow/pull/4353?src=pr=footer). Last update [f211390...e1c9a24](https://codecov.io/gh/apache/incubator-airflow/pull/4353?src=pr=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments). This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] BasPH opened a new pull request #4356: [AIRFLOW-3556] Add cross join set dependency function
BasPH opened a new pull request #4356: [AIRFLOW-3556] Add cross join set
dependency function
URL: https://github.com/apache/incubator-airflow/pull/4356
Make sure you have checked _all_ steps below.
### Jira
- [x] My PR addresses the following [Airflow
Jira](https://issues.apache.org/jira/browse/AIRFLOW/) issues and references
them in the PR title. For example, "\[AIRFLOW-XXX\] My Airflow PR"
- https://issues.apache.org/jira/browse/AIRFLOW-3556
- In case you are fixing a typo in the documentation you can prepend your
commit with \[AIRFLOW-XXX\], code changes always need a Jira issue.
### Description
- [x] Here are some details about my PR, including screenshots of any UI
changes:
Add function to set "cross join style" downstream dependencies between two
list of tasks. For example:
```
cross_downstream(from_tasks=[t1, t2, t3], to_tasks=[t4, t5, t6])
Sets dependencies:
t1 --> t4
\ /
t2 -X> t5
/ \
t3 --> t6
Equivalent to:
t1.set_downstream(t4)
t1.set_downstream(t5)
t1.set_downstream(t6)
t2.set_downstream(t4)
t2.set_downstream(t5)
t2.set_downstream(t6)
t3.set_downstream(t4)
t3.set_downstream(t5)
t3.set_downstream(t6)
```
### Tests
- [x] My PR adds the following unit tests __OR__ does not need testing for
this extremely good reason:
HelpersTest.test_cross_downstream()
### Commits
- [x] My commits all reference Jira issues in their subject lines, and I
have squashed multiple commits if they address the same issue. In addition, my
commits follow the guidelines from "[How to write a good git commit
message](http://chris.beams.io/posts/git-commit/)":
1. Subject is separated from body by a blank line
1. Subject is limited to 50 characters (not including Jira issue reference)
1. Subject does not end with a period
1. Subject uses the imperative mood ("add", not "adding")
1. Body wraps at 72 characters
1. Body explains "what" and "why", not "how"
### Documentation
- [x] In case of new functionality, my PR adds documentation that describes
how to use it.
- When adding new operators/hooks/sensors, the autoclass documentation
generation needs to be added.
- All the public functions and the classes in the PR contain docstrings
that explain what it does
### Code Quality
- [ ] Passes `flake8`
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] codecov-io edited a comment on issue #4225: [AIRFLOW-3383] Rotate fernet keys.
codecov-io edited a comment on issue #4225: [AIRFLOW-3383] Rotate fernet keys. URL: https://github.com/apache/incubator-airflow/pull/4225#issuecomment-441103479 # [Codecov](https://codecov.io/gh/apache/incubator-airflow/pull/4225?src=pr=h1) Report > Merging [#4225](https://codecov.io/gh/apache/incubator-airflow/pull/4225?src=pr=desc) into [master](https://codecov.io/gh/apache/incubator-airflow/commit/f2113906d052df818cf14be8c513fada91caf26a?src=pr=desc) will **increase** coverage by `62%`. > The diff coverage is `66.66%`. [](https://codecov.io/gh/apache/incubator-airflow/pull/4225?src=pr=tree) ```diff @@Coverage Diff @@ ## master#4225 +/- ## == + Coverage 16.13% 78.14% +62% == Files 202 202 Lines 1648316502 +19 == + Hits 266012895 +10235 + Misses 13823 3607 -10216 ``` | [Impacted Files](https://codecov.io/gh/apache/incubator-airflow/pull/4225?src=pr=tree) | Coverage Δ | | |---|---|---| | [airflow/models/\_\_init\_\_.py](https://codecov.io/gh/apache/incubator-airflow/pull/4225/diff?src=pr=tree#diff-YWlyZmxvdy9tb2RlbHMvX19pbml0X18ucHk=) | `92.87% <100%> (+64.93%)` | :arrow_up: | | [airflow/bin/cli.py](https://codecov.io/gh/apache/incubator-airflow/pull/4225/diff?src=pr=tree#diff-YWlyZmxvdy9iaW4vY2xpLnB5) | `64.06% <14.28%> (+50.24%)` | :arrow_up: | | [airflow/models/connection.py](https://codecov.io/gh/apache/incubator-airflow/pull/4225/diff?src=pr=tree#diff-YWlyZmxvdy9tb2RlbHMvY29ubmVjdGlvbi5weQ==) | `88.41% <83.33%> (+65.62%)` | :arrow_up: | | [airflow/plugins\_manager.py](https://codecov.io/gh/apache/incubator-airflow/pull/4225/diff?src=pr=tree#diff-YWlyZmxvdy9wbHVnaW5zX21hbmFnZXIucHk=) | `93.1% <0%> (+1.14%)` | :arrow_up: | | [airflow/executors/dask\_executor.py](https://codecov.io/gh/apache/incubator-airflow/pull/4225/diff?src=pr=tree#diff-YWlyZmxvdy9leGVjdXRvcnMvZGFza19leGVjdXRvci5weQ==) | `2% <0%> (+2%)` | :arrow_up: | | [airflow/exceptions.py](https://codecov.io/gh/apache/incubator-airflow/pull/4225/diff?src=pr=tree#diff-YWlyZmxvdy9leGNlcHRpb25zLnB5) | `100% <0%> (+2.85%)` | :arrow_up: | | [airflow/utils/operator\_resources.py](https://codecov.io/gh/apache/incubator-airflow/pull/4225/diff?src=pr=tree#diff-YWlyZmxvdy91dGlscy9vcGVyYXRvcl9yZXNvdXJjZXMucHk=) | `86.95% <0%> (+4.34%)` | :arrow_up: | | [airflow/executors/\_\_init\_\_.py](https://codecov.io/gh/apache/incubator-airflow/pull/4225/diff?src=pr=tree#diff-YWlyZmxvdy9leGVjdXRvcnMvX19pbml0X18ucHk=) | `63.46% <0%> (+5.76%)` | :arrow_up: | | [airflow/utils/module\_loading.py](https://codecov.io/gh/apache/incubator-airflow/pull/4225/diff?src=pr=tree#diff-YWlyZmxvdy91dGlscy9tb2R1bGVfbG9hZGluZy5weQ==) | `100% <0%> (+10.52%)` | :arrow_up: | | [airflow/utils/decorators.py](https://codecov.io/gh/apache/incubator-airflow/pull/4225/diff?src=pr=tree#diff-YWlyZmxvdy91dGlscy9kZWNvcmF0b3JzLnB5) | `91.66% <0%> (+14.58%)` | :arrow_up: | | ... and [164 more](https://codecov.io/gh/apache/incubator-airflow/pull/4225/diff?src=pr=tree-more) | | -- [Continue to review full report at Codecov](https://codecov.io/gh/apache/incubator-airflow/pull/4225?src=pr=continue). > **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta) > `Δ = absolute (impact)`, `ø = not affected`, `? = missing data` > Powered by [Codecov](https://codecov.io/gh/apache/incubator-airflow/pull/4225?src=pr=footer). Last update [f211390...222c1d9](https://codecov.io/gh/apache/incubator-airflow/pull/4225?src=pr=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments). This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] codecov-io edited a comment on issue #4225: [AIRFLOW-3383] Rotate fernet keys.
codecov-io edited a comment on issue #4225: [AIRFLOW-3383] Rotate fernet keys. URL: https://github.com/apache/incubator-airflow/pull/4225#issuecomment-441103479 # [Codecov](https://codecov.io/gh/apache/incubator-airflow/pull/4225?src=pr=h1) Report > Merging [#4225](https://codecov.io/gh/apache/incubator-airflow/pull/4225?src=pr=desc) into [master](https://codecov.io/gh/apache/incubator-airflow/commit/f2113906d052df818cf14be8c513fada91caf26a?src=pr=desc) will **increase** coverage by `62%`. > The diff coverage is `66.66%`. [](https://codecov.io/gh/apache/incubator-airflow/pull/4225?src=pr=tree) ```diff @@Coverage Diff @@ ## master#4225 +/- ## == + Coverage 16.13% 78.14% +62% == Files 202 202 Lines 1648316502 +19 == + Hits 266012895 +10235 + Misses 13823 3607 -10216 ``` | [Impacted Files](https://codecov.io/gh/apache/incubator-airflow/pull/4225?src=pr=tree) | Coverage Δ | | |---|---|---| | [airflow/models/\_\_init\_\_.py](https://codecov.io/gh/apache/incubator-airflow/pull/4225/diff?src=pr=tree#diff-YWlyZmxvdy9tb2RlbHMvX19pbml0X18ucHk=) | `92.87% <100%> (+64.93%)` | :arrow_up: | | [airflow/bin/cli.py](https://codecov.io/gh/apache/incubator-airflow/pull/4225/diff?src=pr=tree#diff-YWlyZmxvdy9iaW4vY2xpLnB5) | `64.06% <14.28%> (+50.24%)` | :arrow_up: | | [airflow/models/connection.py](https://codecov.io/gh/apache/incubator-airflow/pull/4225/diff?src=pr=tree#diff-YWlyZmxvdy9tb2RlbHMvY29ubmVjdGlvbi5weQ==) | `88.41% <83.33%> (+65.62%)` | :arrow_up: | | [airflow/plugins\_manager.py](https://codecov.io/gh/apache/incubator-airflow/pull/4225/diff?src=pr=tree#diff-YWlyZmxvdy9wbHVnaW5zX21hbmFnZXIucHk=) | `93.1% <0%> (+1.14%)` | :arrow_up: | | [airflow/executors/dask\_executor.py](https://codecov.io/gh/apache/incubator-airflow/pull/4225/diff?src=pr=tree#diff-YWlyZmxvdy9leGVjdXRvcnMvZGFza19leGVjdXRvci5weQ==) | `2% <0%> (+2%)` | :arrow_up: | | [airflow/exceptions.py](https://codecov.io/gh/apache/incubator-airflow/pull/4225/diff?src=pr=tree#diff-YWlyZmxvdy9leGNlcHRpb25zLnB5) | `100% <0%> (+2.85%)` | :arrow_up: | | [airflow/utils/operator\_resources.py](https://codecov.io/gh/apache/incubator-airflow/pull/4225/diff?src=pr=tree#diff-YWlyZmxvdy91dGlscy9vcGVyYXRvcl9yZXNvdXJjZXMucHk=) | `86.95% <0%> (+4.34%)` | :arrow_up: | | [airflow/executors/\_\_init\_\_.py](https://codecov.io/gh/apache/incubator-airflow/pull/4225/diff?src=pr=tree#diff-YWlyZmxvdy9leGVjdXRvcnMvX19pbml0X18ucHk=) | `63.46% <0%> (+5.76%)` | :arrow_up: | | [airflow/utils/module\_loading.py](https://codecov.io/gh/apache/incubator-airflow/pull/4225/diff?src=pr=tree#diff-YWlyZmxvdy91dGlscy9tb2R1bGVfbG9hZGluZy5weQ==) | `100% <0%> (+10.52%)` | :arrow_up: | | [airflow/utils/decorators.py](https://codecov.io/gh/apache/incubator-airflow/pull/4225/diff?src=pr=tree#diff-YWlyZmxvdy91dGlscy9kZWNvcmF0b3JzLnB5) | `91.66% <0%> (+14.58%)` | :arrow_up: | | ... and [164 more](https://codecov.io/gh/apache/incubator-airflow/pull/4225/diff?src=pr=tree-more) | | -- [Continue to review full report at Codecov](https://codecov.io/gh/apache/incubator-airflow/pull/4225?src=pr=continue). > **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta) > `Δ = absolute (impact)`, `ø = not affected`, `? = missing data` > Powered by [Codecov](https://codecov.io/gh/apache/incubator-airflow/pull/4225?src=pr=footer). Last update [f211390...222c1d9](https://codecov.io/gh/apache/incubator-airflow/pull/4225?src=pr=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments). This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] codecov-io edited a comment on issue #4225: [AIRFLOW-3383] Rotate fernet keys.
codecov-io edited a comment on issue #4225: [AIRFLOW-3383] Rotate fernet keys. URL: https://github.com/apache/incubator-airflow/pull/4225#issuecomment-441103479 # [Codecov](https://codecov.io/gh/apache/incubator-airflow/pull/4225?src=pr=h1) Report > Merging [#4225](https://codecov.io/gh/apache/incubator-airflow/pull/4225?src=pr=desc) into [master](https://codecov.io/gh/apache/incubator-airflow/commit/f2113906d052df818cf14be8c513fada91caf26a?src=pr=desc) will **increase** coverage by `62%`. > The diff coverage is `66.66%`. [](https://codecov.io/gh/apache/incubator-airflow/pull/4225?src=pr=tree) ```diff @@Coverage Diff @@ ## master#4225 +/- ## == + Coverage 16.13% 78.14% +62% == Files 202 202 Lines 1648316502 +19 == + Hits 266012895 +10235 + Misses 13823 3607 -10216 ``` | [Impacted Files](https://codecov.io/gh/apache/incubator-airflow/pull/4225?src=pr=tree) | Coverage Δ | | |---|---|---| | [airflow/models/\_\_init\_\_.py](https://codecov.io/gh/apache/incubator-airflow/pull/4225/diff?src=pr=tree#diff-YWlyZmxvdy9tb2RlbHMvX19pbml0X18ucHk=) | `92.87% <100%> (+64.93%)` | :arrow_up: | | [airflow/bin/cli.py](https://codecov.io/gh/apache/incubator-airflow/pull/4225/diff?src=pr=tree#diff-YWlyZmxvdy9iaW4vY2xpLnB5) | `64.06% <14.28%> (+50.24%)` | :arrow_up: | | [airflow/models/connection.py](https://codecov.io/gh/apache/incubator-airflow/pull/4225/diff?src=pr=tree#diff-YWlyZmxvdy9tb2RlbHMvY29ubmVjdGlvbi5weQ==) | `88.41% <83.33%> (+65.62%)` | :arrow_up: | | [airflow/plugins\_manager.py](https://codecov.io/gh/apache/incubator-airflow/pull/4225/diff?src=pr=tree#diff-YWlyZmxvdy9wbHVnaW5zX21hbmFnZXIucHk=) | `93.1% <0%> (+1.14%)` | :arrow_up: | | [airflow/executors/dask\_executor.py](https://codecov.io/gh/apache/incubator-airflow/pull/4225/diff?src=pr=tree#diff-YWlyZmxvdy9leGVjdXRvcnMvZGFza19leGVjdXRvci5weQ==) | `2% <0%> (+2%)` | :arrow_up: | | [airflow/exceptions.py](https://codecov.io/gh/apache/incubator-airflow/pull/4225/diff?src=pr=tree#diff-YWlyZmxvdy9leGNlcHRpb25zLnB5) | `100% <0%> (+2.85%)` | :arrow_up: | | [airflow/utils/operator\_resources.py](https://codecov.io/gh/apache/incubator-airflow/pull/4225/diff?src=pr=tree#diff-YWlyZmxvdy91dGlscy9vcGVyYXRvcl9yZXNvdXJjZXMucHk=) | `86.95% <0%> (+4.34%)` | :arrow_up: | | [airflow/executors/\_\_init\_\_.py](https://codecov.io/gh/apache/incubator-airflow/pull/4225/diff?src=pr=tree#diff-YWlyZmxvdy9leGVjdXRvcnMvX19pbml0X18ucHk=) | `63.46% <0%> (+5.76%)` | :arrow_up: | | [airflow/utils/module\_loading.py](https://codecov.io/gh/apache/incubator-airflow/pull/4225/diff?src=pr=tree#diff-YWlyZmxvdy91dGlscy9tb2R1bGVfbG9hZGluZy5weQ==) | `100% <0%> (+10.52%)` | :arrow_up: | | [airflow/utils/decorators.py](https://codecov.io/gh/apache/incubator-airflow/pull/4225/diff?src=pr=tree#diff-YWlyZmxvdy91dGlscy9kZWNvcmF0b3JzLnB5) | `91.66% <0%> (+14.58%)` | :arrow_up: | | ... and [164 more](https://codecov.io/gh/apache/incubator-airflow/pull/4225/diff?src=pr=tree-more) | | -- [Continue to review full report at Codecov](https://codecov.io/gh/apache/incubator-airflow/pull/4225?src=pr=continue). > **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta) > `Δ = absolute (impact)`, `ø = not affected`, `? = missing data` > Powered by [Codecov](https://codecov.io/gh/apache/incubator-airflow/pull/4225?src=pr=footer). Last update [f211390...222c1d9](https://codecov.io/gh/apache/incubator-airflow/pull/4225?src=pr=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments). This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] codecov-io edited a comment on issue #4225: [AIRFLOW-3383] Rotate fernet keys.
codecov-io edited a comment on issue #4225: [AIRFLOW-3383] Rotate fernet keys. URL: https://github.com/apache/incubator-airflow/pull/4225#issuecomment-441103479 # [Codecov](https://codecov.io/gh/apache/incubator-airflow/pull/4225?src=pr=h1) Report > Merging [#4225](https://codecov.io/gh/apache/incubator-airflow/pull/4225?src=pr=desc) into [master](https://codecov.io/gh/apache/incubator-airflow/commit/f2113906d052df818cf14be8c513fada91caf26a?src=pr=desc) will **increase** coverage by `62%`. > The diff coverage is `66.66%`. [](https://codecov.io/gh/apache/incubator-airflow/pull/4225?src=pr=tree) ```diff @@Coverage Diff @@ ## master#4225 +/- ## == + Coverage 16.13% 78.14% +62% == Files 202 202 Lines 1648316502 +19 == + Hits 266012895 +10235 + Misses 13823 3607 -10216 ``` | [Impacted Files](https://codecov.io/gh/apache/incubator-airflow/pull/4225?src=pr=tree) | Coverage Δ | | |---|---|---| | [airflow/models/\_\_init\_\_.py](https://codecov.io/gh/apache/incubator-airflow/pull/4225/diff?src=pr=tree#diff-YWlyZmxvdy9tb2RlbHMvX19pbml0X18ucHk=) | `92.87% <100%> (+64.93%)` | :arrow_up: | | [airflow/bin/cli.py](https://codecov.io/gh/apache/incubator-airflow/pull/4225/diff?src=pr=tree#diff-YWlyZmxvdy9iaW4vY2xpLnB5) | `64.06% <14.28%> (+50.24%)` | :arrow_up: | | [airflow/models/connection.py](https://codecov.io/gh/apache/incubator-airflow/pull/4225/diff?src=pr=tree#diff-YWlyZmxvdy9tb2RlbHMvY29ubmVjdGlvbi5weQ==) | `88.41% <83.33%> (+65.62%)` | :arrow_up: | | [airflow/plugins\_manager.py](https://codecov.io/gh/apache/incubator-airflow/pull/4225/diff?src=pr=tree#diff-YWlyZmxvdy9wbHVnaW5zX21hbmFnZXIucHk=) | `93.1% <0%> (+1.14%)` | :arrow_up: | | [airflow/executors/dask\_executor.py](https://codecov.io/gh/apache/incubator-airflow/pull/4225/diff?src=pr=tree#diff-YWlyZmxvdy9leGVjdXRvcnMvZGFza19leGVjdXRvci5weQ==) | `2% <0%> (+2%)` | :arrow_up: | | [airflow/exceptions.py](https://codecov.io/gh/apache/incubator-airflow/pull/4225/diff?src=pr=tree#diff-YWlyZmxvdy9leGNlcHRpb25zLnB5) | `100% <0%> (+2.85%)` | :arrow_up: | | [airflow/utils/operator\_resources.py](https://codecov.io/gh/apache/incubator-airflow/pull/4225/diff?src=pr=tree#diff-YWlyZmxvdy91dGlscy9vcGVyYXRvcl9yZXNvdXJjZXMucHk=) | `86.95% <0%> (+4.34%)` | :arrow_up: | | [airflow/executors/\_\_init\_\_.py](https://codecov.io/gh/apache/incubator-airflow/pull/4225/diff?src=pr=tree#diff-YWlyZmxvdy9leGVjdXRvcnMvX19pbml0X18ucHk=) | `63.46% <0%> (+5.76%)` | :arrow_up: | | [airflow/utils/module\_loading.py](https://codecov.io/gh/apache/incubator-airflow/pull/4225/diff?src=pr=tree#diff-YWlyZmxvdy91dGlscy9tb2R1bGVfbG9hZGluZy5weQ==) | `100% <0%> (+10.52%)` | :arrow_up: | | [airflow/utils/decorators.py](https://codecov.io/gh/apache/incubator-airflow/pull/4225/diff?src=pr=tree#diff-YWlyZmxvdy91dGlscy9kZWNvcmF0b3JzLnB5) | `91.66% <0%> (+14.58%)` | :arrow_up: | | ... and [164 more](https://codecov.io/gh/apache/incubator-airflow/pull/4225/diff?src=pr=tree-more) | | -- [Continue to review full report at Codecov](https://codecov.io/gh/apache/incubator-airflow/pull/4225?src=pr=continue). > **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta) > `Δ = absolute (impact)`, `ø = not affected`, `? = missing data` > Powered by [Codecov](https://codecov.io/gh/apache/incubator-airflow/pull/4225?src=pr=footer). Last update [f211390...222c1d9](https://codecov.io/gh/apache/incubator-airflow/pull/4225?src=pr=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments). This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] C4 opened a new pull request #4355: [AIRFLOW-XXX] Update log level and return value
C4 opened a new pull request #4355: [AIRFLOW-XXX] Update log level and return value URL: https://github.com/apache/incubator-airflow/pull/4355 ### Jira N/A ### Description After using this operator, it's very useful to see which file is getting downloaded. Changed the debug logging to info. Also, return the filepath into xcom so that downstream tasks can use it. This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] jmcarp commented on issue #4225: [AIRFLOW-3383] Rotate fernet keys.
jmcarp commented on issue #4225: [AIRFLOW-3383] Rotate fernet keys. URL: https://github.com/apache/incubator-airflow/pull/4225#issuecomment-44947 Sorry @ashb, I was in a hurry earlier and totally misread your point about encrypting variables. I pushed up some changes to rotate keys on encrypted variables too. This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] bjoernpollex-sc commented on issue #730: [AIRFLOW-594] Load plugins and workflows from packages
bjoernpollex-sc commented on issue #730: [AIRFLOW-594] Load plugins and workflows from packages URL: https://github.com/apache/incubator-airflow/pull/730#issuecomment-449470619 @mahrz24 @kaxil @mik-laj @gwax I've just stumbled across this as I was thinking to implement something similar. I'd be happy to pick up work on this again, I agree that it seems to be almost complete. The first thing to do seems to be to rebase onto current `master` (or is a merge also an option?). At a first glance, there's a few conflicts, but nothing that looks hard to resolve. I'm happy to do this, unless someone tells me it's fruitless. This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] bjoernpollex-sc edited a comment on issue #730: [AIRFLOW-594] Load plugins and workflows from packages
bjoernpollex-sc edited a comment on issue #730: [AIRFLOW-594] Load plugins and workflows from packages URL: https://github.com/apache/incubator-airflow/pull/730#issuecomment-449470619 @mahrz24 @kaxil @mik-laj @gwax I've just stumbled across this as I was thinking of implementing something similar. I'd be happy to pick up work on this again, I agree that it seems to be almost complete. The first thing to do seems to be to rebase onto current `master` (or is a merge also an option?). At a first glance, there's a few conflicts, but nothing that looks hard to resolve. I'm happy to do this, unless someone tells me it's fruitless. This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] DariuszAniszewski opened a new pull request #4354: [AIRFLOW-3446] Add Google Cloud BigTable operators
DariuszAniszewski opened a new pull request #4354: [AIRFLOW-3446] Add Google
Cloud BigTable operators
URL: https://github.com/apache/incubator-airflow/pull/4354
Make sure you have checked _all_ steps below.
### Jira
- [x] My PR addresses the following
[AIRFLOW-3446](https://issues.apache.org/jira/browse/AIRFLOW-3446/) issues and
references them in the PR title.
### Description
- [x] Here are some details about my PR, including screenshots of any UI
changes:
New operators allows:
* creating and deleting instance
* creating and deleting table
* updating cluster
* waiting for table replication (sensor)
### Tests
- [x] My PR adds the following unit tests:
* tests/contrib/operators/test_gcp_bigtable_operator.py
### Commits
- [x] My commits all reference Jira issues in their subject lines, and I
have squashed multiple commits if they address the same issue. In addition, my
commits follow the guidelines from "[How to write a good git commit
message](http://chris.beams.io/posts/git-commit/)":
1. Subject is separated from body by a blank line
1. Subject is limited to 50 characters (not including Jira issue reference)
1. Subject does not end with a period
1. Subject uses the imperative mood ("add", not "adding")
1. Body wraps at 72 characters
1. Body explains "what" and "why", not "how"
### Documentation
- [x] In case of new functionality, my PR adds documentation that describes
how to use it.
- When adding new operators/hooks/sensors, the autoclass documentation
generation needs to be added.
- All the public functions and the classes in the PR contain docstrings
that explain what it does
### Code Quality
- [x] Passes `flake8`
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] codecov-io edited a comment on issue #4225: [AIRFLOW-3383] Rotate fernet keys.
codecov-io edited a comment on issue #4225: [AIRFLOW-3383] Rotate fernet keys. URL: https://github.com/apache/incubator-airflow/pull/4225#issuecomment-441103479 # [Codecov](https://codecov.io/gh/apache/incubator-airflow/pull/4225?src=pr=h1) Report > Merging [#4225](https://codecov.io/gh/apache/incubator-airflow/pull/4225?src=pr=desc) into [master](https://codecov.io/gh/apache/incubator-airflow/commit/f2113906d052df818cf14be8c513fada91caf26a?src=pr=desc) will **increase** coverage by `61.98%`. > The diff coverage is `60.86%`. [](https://codecov.io/gh/apache/incubator-airflow/pull/4225?src=pr=tree) ```diff @@ Coverage Diff @@ ## master#4225 +/- ## === + Coverage 16.13% 78.12% +61.98% === Files 202 202 Lines 1648316504 +21 === + Hits 266012894+10234 + Misses 13823 3610-10213 ``` | [Impacted Files](https://codecov.io/gh/apache/incubator-airflow/pull/4225?src=pr=tree) | Coverage Δ | | |---|---|---| | [airflow/models/\_\_init\_\_.py](https://codecov.io/gh/apache/incubator-airflow/pull/4225/diff?src=pr=tree#diff-YWlyZmxvdy9tb2RlbHMvX19pbml0X18ucHk=) | `92.82% <100%> (+64.89%)` | :arrow_up: | | [airflow/bin/cli.py](https://codecov.io/gh/apache/incubator-airflow/pull/4225/diff?src=pr=tree#diff-YWlyZmxvdy9iaW4vY2xpLnB5) | `63.91% <11.11%> (+50.08%)` | :arrow_up: | | [airflow/models/connection.py](https://codecov.io/gh/apache/incubator-airflow/pull/4225/diff?src=pr=tree#diff-YWlyZmxvdy9tb2RlbHMvY29ubmVjdGlvbi5weQ==) | `88.41% <83.33%> (+65.62%)` | :arrow_up: | | [airflow/plugins\_manager.py](https://codecov.io/gh/apache/incubator-airflow/pull/4225/diff?src=pr=tree#diff-YWlyZmxvdy9wbHVnaW5zX21hbmFnZXIucHk=) | `93.1% <0%> (+1.14%)` | :arrow_up: | | [airflow/executors/dask\_executor.py](https://codecov.io/gh/apache/incubator-airflow/pull/4225/diff?src=pr=tree#diff-YWlyZmxvdy9leGVjdXRvcnMvZGFza19leGVjdXRvci5weQ==) | `2% <0%> (+2%)` | :arrow_up: | | [airflow/exceptions.py](https://codecov.io/gh/apache/incubator-airflow/pull/4225/diff?src=pr=tree#diff-YWlyZmxvdy9leGNlcHRpb25zLnB5) | `100% <0%> (+2.85%)` | :arrow_up: | | [airflow/utils/operator\_resources.py](https://codecov.io/gh/apache/incubator-airflow/pull/4225/diff?src=pr=tree#diff-YWlyZmxvdy91dGlscy9vcGVyYXRvcl9yZXNvdXJjZXMucHk=) | `86.95% <0%> (+4.34%)` | :arrow_up: | | [airflow/executors/\_\_init\_\_.py](https://codecov.io/gh/apache/incubator-airflow/pull/4225/diff?src=pr=tree#diff-YWlyZmxvdy9leGVjdXRvcnMvX19pbml0X18ucHk=) | `63.46% <0%> (+5.76%)` | :arrow_up: | | [airflow/utils/module\_loading.py](https://codecov.io/gh/apache/incubator-airflow/pull/4225/diff?src=pr=tree#diff-YWlyZmxvdy91dGlscy9tb2R1bGVfbG9hZGluZy5weQ==) | `100% <0%> (+10.52%)` | :arrow_up: | | [airflow/utils/decorators.py](https://codecov.io/gh/apache/incubator-airflow/pull/4225/diff?src=pr=tree#diff-YWlyZmxvdy91dGlscy9kZWNvcmF0b3JzLnB5) | `91.66% <0%> (+14.58%)` | :arrow_up: | | ... and [164 more](https://codecov.io/gh/apache/incubator-airflow/pull/4225/diff?src=pr=tree-more) | | -- [Continue to review full report at Codecov](https://codecov.io/gh/apache/incubator-airflow/pull/4225?src=pr=continue). > **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta) > `Δ = absolute (impact)`, `ø = not affected`, `? = missing data` > Powered by [Codecov](https://codecov.io/gh/apache/incubator-airflow/pull/4225?src=pr=footer). Last update [f211390...9e7c1b4](https://codecov.io/gh/apache/incubator-airflow/pull/4225?src=pr=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments). This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] codecov-io edited a comment on issue #4225: [AIRFLOW-3383] Rotate fernet keys.
codecov-io edited a comment on issue #4225: [AIRFLOW-3383] Rotate fernet keys. URL: https://github.com/apache/incubator-airflow/pull/4225#issuecomment-441103479 # [Codecov](https://codecov.io/gh/apache/incubator-airflow/pull/4225?src=pr=h1) Report > Merging [#4225](https://codecov.io/gh/apache/incubator-airflow/pull/4225?src=pr=desc) into [master](https://codecov.io/gh/apache/incubator-airflow/commit/f2113906d052df818cf14be8c513fada91caf26a?src=pr=desc) will **increase** coverage by `61.98%`. > The diff coverage is `60.86%`. [](https://codecov.io/gh/apache/incubator-airflow/pull/4225?src=pr=tree) ```diff @@ Coverage Diff @@ ## master#4225 +/- ## === + Coverage 16.13% 78.12% +61.98% === Files 202 202 Lines 1648316504 +21 === + Hits 266012894+10234 + Misses 13823 3610-10213 ``` | [Impacted Files](https://codecov.io/gh/apache/incubator-airflow/pull/4225?src=pr=tree) | Coverage Δ | | |---|---|---| | [airflow/models/\_\_init\_\_.py](https://codecov.io/gh/apache/incubator-airflow/pull/4225/diff?src=pr=tree#diff-YWlyZmxvdy9tb2RlbHMvX19pbml0X18ucHk=) | `92.82% <100%> (+64.89%)` | :arrow_up: | | [airflow/bin/cli.py](https://codecov.io/gh/apache/incubator-airflow/pull/4225/diff?src=pr=tree#diff-YWlyZmxvdy9iaW4vY2xpLnB5) | `63.91% <11.11%> (+50.08%)` | :arrow_up: | | [airflow/models/connection.py](https://codecov.io/gh/apache/incubator-airflow/pull/4225/diff?src=pr=tree#diff-YWlyZmxvdy9tb2RlbHMvY29ubmVjdGlvbi5weQ==) | `88.41% <83.33%> (+65.62%)` | :arrow_up: | | [airflow/plugins\_manager.py](https://codecov.io/gh/apache/incubator-airflow/pull/4225/diff?src=pr=tree#diff-YWlyZmxvdy9wbHVnaW5zX21hbmFnZXIucHk=) | `93.1% <0%> (+1.14%)` | :arrow_up: | | [airflow/executors/dask\_executor.py](https://codecov.io/gh/apache/incubator-airflow/pull/4225/diff?src=pr=tree#diff-YWlyZmxvdy9leGVjdXRvcnMvZGFza19leGVjdXRvci5weQ==) | `2% <0%> (+2%)` | :arrow_up: | | [airflow/exceptions.py](https://codecov.io/gh/apache/incubator-airflow/pull/4225/diff?src=pr=tree#diff-YWlyZmxvdy9leGNlcHRpb25zLnB5) | `100% <0%> (+2.85%)` | :arrow_up: | | [airflow/utils/operator\_resources.py](https://codecov.io/gh/apache/incubator-airflow/pull/4225/diff?src=pr=tree#diff-YWlyZmxvdy91dGlscy9vcGVyYXRvcl9yZXNvdXJjZXMucHk=) | `86.95% <0%> (+4.34%)` | :arrow_up: | | [airflow/executors/\_\_init\_\_.py](https://codecov.io/gh/apache/incubator-airflow/pull/4225/diff?src=pr=tree#diff-YWlyZmxvdy9leGVjdXRvcnMvX19pbml0X18ucHk=) | `63.46% <0%> (+5.76%)` | :arrow_up: | | [airflow/utils/module\_loading.py](https://codecov.io/gh/apache/incubator-airflow/pull/4225/diff?src=pr=tree#diff-YWlyZmxvdy91dGlscy9tb2R1bGVfbG9hZGluZy5weQ==) | `100% <0%> (+10.52%)` | :arrow_up: | | [airflow/utils/decorators.py](https://codecov.io/gh/apache/incubator-airflow/pull/4225/diff?src=pr=tree#diff-YWlyZmxvdy91dGlscy9kZWNvcmF0b3JzLnB5) | `91.66% <0%> (+14.58%)` | :arrow_up: | | ... and [164 more](https://codecov.io/gh/apache/incubator-airflow/pull/4225/diff?src=pr=tree-more) | | -- [Continue to review full report at Codecov](https://codecov.io/gh/apache/incubator-airflow/pull/4225?src=pr=continue). > **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta) > `Δ = absolute (impact)`, `ø = not affected`, `? = missing data` > Powered by [Codecov](https://codecov.io/gh/apache/incubator-airflow/pull/4225?src=pr=footer). Last update [f211390...9e7c1b4](https://codecov.io/gh/apache/incubator-airflow/pull/4225?src=pr=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments). This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] codecov-io commented on issue #4353: [AIRFLOW-3480] Added Database Deploy/Update/Delete operators
codecov-io commented on issue #4353: [AIRFLOW-3480] Added Database Deploy/Update/Delete operators URL: https://github.com/apache/incubator-airflow/pull/4353#issuecomment-449454398 # [Codecov](https://codecov.io/gh/apache/incubator-airflow/pull/4353?src=pr=h1) Report > Merging [#4353](https://codecov.io/gh/apache/incubator-airflow/pull/4353?src=pr=desc) into [master](https://codecov.io/gh/apache/incubator-airflow/commit/f2113906d052df818cf14be8c513fada91caf26a?src=pr=desc) will **increase** coverage by `61.99%`. > The diff coverage is `n/a`. [](https://codecov.io/gh/apache/incubator-airflow/pull/4353?src=pr=tree) ```diff @@ Coverage Diff @@ ## master#4353 +/- ## === + Coverage 16.13% 78.12% +61.99% === Files 202 202 Lines 1648316483 === + Hits 266012878+10218 + Misses 13823 3605-10218 ``` | [Impacted Files](https://codecov.io/gh/apache/incubator-airflow/pull/4353?src=pr=tree) | Coverage Δ | | |---|---|---| | [airflow/plugins\_manager.py](https://codecov.io/gh/apache/incubator-airflow/pull/4353/diff?src=pr=tree#diff-YWlyZmxvdy9wbHVnaW5zX21hbmFnZXIucHk=) | `93.1% <0%> (+1.14%)` | :arrow_up: | | [airflow/executors/dask\_executor.py](https://codecov.io/gh/apache/incubator-airflow/pull/4353/diff?src=pr=tree#diff-YWlyZmxvdy9leGVjdXRvcnMvZGFza19leGVjdXRvci5weQ==) | `2% <0%> (+2%)` | :arrow_up: | | [airflow/exceptions.py](https://codecov.io/gh/apache/incubator-airflow/pull/4353/diff?src=pr=tree#diff-YWlyZmxvdy9leGNlcHRpb25zLnB5) | `100% <0%> (+2.85%)` | :arrow_up: | | [airflow/utils/operator\_resources.py](https://codecov.io/gh/apache/incubator-airflow/pull/4353/diff?src=pr=tree#diff-YWlyZmxvdy91dGlscy9vcGVyYXRvcl9yZXNvdXJjZXMucHk=) | `86.95% <0%> (+4.34%)` | :arrow_up: | | [airflow/executors/\_\_init\_\_.py](https://codecov.io/gh/apache/incubator-airflow/pull/4353/diff?src=pr=tree#diff-YWlyZmxvdy9leGVjdXRvcnMvX19pbml0X18ucHk=) | `63.46% <0%> (+5.76%)` | :arrow_up: | | [airflow/utils/module\_loading.py](https://codecov.io/gh/apache/incubator-airflow/pull/4353/diff?src=pr=tree#diff-YWlyZmxvdy91dGlscy9tb2R1bGVfbG9hZGluZy5weQ==) | `100% <0%> (+10.52%)` | :arrow_up: | | [airflow/utils/decorators.py](https://codecov.io/gh/apache/incubator-airflow/pull/4353/diff?src=pr=tree#diff-YWlyZmxvdy91dGlscy9kZWNvcmF0b3JzLnB5) | `91.66% <0%> (+14.58%)` | :arrow_up: | | [airflow/settings.py](https://codecov.io/gh/apache/incubator-airflow/pull/4353/diff?src=pr=tree#diff-YWlyZmxvdy9zZXR0aW5ncy5weQ==) | `80.41% <0%> (+14.68%)` | :arrow_up: | | [airflow/hooks/oracle\_hook.py](https://codecov.io/gh/apache/incubator-airflow/pull/4353/diff?src=pr=tree#diff-YWlyZmxvdy9ob29rcy9vcmFjbGVfaG9vay5weQ==) | `15.47% <0%> (+15.47%)` | :arrow_up: | | [airflow/\_\_init\_\_.py](https://codecov.io/gh/apache/incubator-airflow/pull/4353/diff?src=pr=tree#diff-YWlyZmxvdy9fX2luaXRfXy5weQ==) | `80% <0%> (+15.55%)` | :arrow_up: | | ... and [161 more](https://codecov.io/gh/apache/incubator-airflow/pull/4353/diff?src=pr=tree-more) | | -- [Continue to review full report at Codecov](https://codecov.io/gh/apache/incubator-airflow/pull/4353?src=pr=continue). > **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta) > `Δ = absolute (impact)`, `ø = not affected`, `? = missing data` > Powered by [Codecov](https://codecov.io/gh/apache/incubator-airflow/pull/4353?src=pr=footer). Last update [f211390...4b3348a](https://codecov.io/gh/apache/incubator-airflow/pull/4353?src=pr=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments). This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] codecov-io edited a comment on issue #4352: [AIRFLOW-3555] Remove lxml dependency
codecov-io edited a comment on issue #4352: [AIRFLOW-3555] Remove lxml dependency URL: https://github.com/apache/incubator-airflow/pull/4352#issuecomment-449450870 # [Codecov](https://codecov.io/gh/apache/incubator-airflow/pull/4352?src=pr=h1) Report > Merging [#4352](https://codecov.io/gh/apache/incubator-airflow/pull/4352?src=pr=desc) into [master](https://codecov.io/gh/apache/incubator-airflow/commit/f2113906d052df818cf14be8c513fada91caf26a?src=pr=desc) will **increase** coverage by `61.99%`. > The diff coverage is `n/a`. [](https://codecov.io/gh/apache/incubator-airflow/pull/4352?src=pr=tree) ```diff @@ Coverage Diff @@ ## master#4352 +/- ## === + Coverage 16.13% 78.12% +61.99% === Files 202 202 Lines 1648316483 === + Hits 266012878+10218 + Misses 13823 3605-10218 ``` | [Impacted Files](https://codecov.io/gh/apache/incubator-airflow/pull/4352?src=pr=tree) | Coverage Δ | | |---|---|---| | [airflow/plugins\_manager.py](https://codecov.io/gh/apache/incubator-airflow/pull/4352/diff?src=pr=tree#diff-YWlyZmxvdy9wbHVnaW5zX21hbmFnZXIucHk=) | `93.1% <0%> (+1.14%)` | :arrow_up: | | [airflow/executors/dask\_executor.py](https://codecov.io/gh/apache/incubator-airflow/pull/4352/diff?src=pr=tree#diff-YWlyZmxvdy9leGVjdXRvcnMvZGFza19leGVjdXRvci5weQ==) | `2% <0%> (+2%)` | :arrow_up: | | [airflow/exceptions.py](https://codecov.io/gh/apache/incubator-airflow/pull/4352/diff?src=pr=tree#diff-YWlyZmxvdy9leGNlcHRpb25zLnB5) | `100% <0%> (+2.85%)` | :arrow_up: | | [airflow/utils/operator\_resources.py](https://codecov.io/gh/apache/incubator-airflow/pull/4352/diff?src=pr=tree#diff-YWlyZmxvdy91dGlscy9vcGVyYXRvcl9yZXNvdXJjZXMucHk=) | `86.95% <0%> (+4.34%)` | :arrow_up: | | [airflow/executors/\_\_init\_\_.py](https://codecov.io/gh/apache/incubator-airflow/pull/4352/diff?src=pr=tree#diff-YWlyZmxvdy9leGVjdXRvcnMvX19pbml0X18ucHk=) | `63.46% <0%> (+5.76%)` | :arrow_up: | | [airflow/utils/module\_loading.py](https://codecov.io/gh/apache/incubator-airflow/pull/4352/diff?src=pr=tree#diff-YWlyZmxvdy91dGlscy9tb2R1bGVfbG9hZGluZy5weQ==) | `100% <0%> (+10.52%)` | :arrow_up: | | [airflow/utils/decorators.py](https://codecov.io/gh/apache/incubator-airflow/pull/4352/diff?src=pr=tree#diff-YWlyZmxvdy91dGlscy9kZWNvcmF0b3JzLnB5) | `91.66% <0%> (+14.58%)` | :arrow_up: | | [airflow/settings.py](https://codecov.io/gh/apache/incubator-airflow/pull/4352/diff?src=pr=tree#diff-YWlyZmxvdy9zZXR0aW5ncy5weQ==) | `80.41% <0%> (+14.68%)` | :arrow_up: | | [airflow/hooks/oracle\_hook.py](https://codecov.io/gh/apache/incubator-airflow/pull/4352/diff?src=pr=tree#diff-YWlyZmxvdy9ob29rcy9vcmFjbGVfaG9vay5weQ==) | `15.47% <0%> (+15.47%)` | :arrow_up: | | [airflow/\_\_init\_\_.py](https://codecov.io/gh/apache/incubator-airflow/pull/4352/diff?src=pr=tree#diff-YWlyZmxvdy9fX2luaXRfXy5weQ==) | `80% <0%> (+15.55%)` | :arrow_up: | | ... and [161 more](https://codecov.io/gh/apache/incubator-airflow/pull/4352/diff?src=pr=tree-more) | | -- [Continue to review full report at Codecov](https://codecov.io/gh/apache/incubator-airflow/pull/4352?src=pr=continue). > **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta) > `Δ = absolute (impact)`, `ø = not affected`, `? = missing data` > Powered by [Codecov](https://codecov.io/gh/apache/incubator-airflow/pull/4352?src=pr=footer). Last update [f211390...b375413](https://codecov.io/gh/apache/incubator-airflow/pull/4352?src=pr=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments). This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] codecov-io edited a comment on issue #4352: [AIRFLOW-3555] Remove lxml dependency
codecov-io edited a comment on issue #4352: [AIRFLOW-3555] Remove lxml dependency URL: https://github.com/apache/incubator-airflow/pull/4352#issuecomment-449450870 # [Codecov](https://codecov.io/gh/apache/incubator-airflow/pull/4352?src=pr=h1) Report > Merging [#4352](https://codecov.io/gh/apache/incubator-airflow/pull/4352?src=pr=desc) into [master](https://codecov.io/gh/apache/incubator-airflow/commit/f2113906d052df818cf14be8c513fada91caf26a?src=pr=desc) will **increase** coverage by `61.99%`. > The diff coverage is `n/a`. [](https://codecov.io/gh/apache/incubator-airflow/pull/4352?src=pr=tree) ```diff @@ Coverage Diff @@ ## master#4352 +/- ## === + Coverage 16.13% 78.12% +61.99% === Files 202 202 Lines 1648316483 === + Hits 266012878+10218 + Misses 13823 3605-10218 ``` | [Impacted Files](https://codecov.io/gh/apache/incubator-airflow/pull/4352?src=pr=tree) | Coverage Δ | | |---|---|---| | [airflow/plugins\_manager.py](https://codecov.io/gh/apache/incubator-airflow/pull/4352/diff?src=pr=tree#diff-YWlyZmxvdy9wbHVnaW5zX21hbmFnZXIucHk=) | `93.1% <0%> (+1.14%)` | :arrow_up: | | [airflow/executors/dask\_executor.py](https://codecov.io/gh/apache/incubator-airflow/pull/4352/diff?src=pr=tree#diff-YWlyZmxvdy9leGVjdXRvcnMvZGFza19leGVjdXRvci5weQ==) | `2% <0%> (+2%)` | :arrow_up: | | [airflow/exceptions.py](https://codecov.io/gh/apache/incubator-airflow/pull/4352/diff?src=pr=tree#diff-YWlyZmxvdy9leGNlcHRpb25zLnB5) | `100% <0%> (+2.85%)` | :arrow_up: | | [airflow/utils/operator\_resources.py](https://codecov.io/gh/apache/incubator-airflow/pull/4352/diff?src=pr=tree#diff-YWlyZmxvdy91dGlscy9vcGVyYXRvcl9yZXNvdXJjZXMucHk=) | `86.95% <0%> (+4.34%)` | :arrow_up: | | [airflow/executors/\_\_init\_\_.py](https://codecov.io/gh/apache/incubator-airflow/pull/4352/diff?src=pr=tree#diff-YWlyZmxvdy9leGVjdXRvcnMvX19pbml0X18ucHk=) | `63.46% <0%> (+5.76%)` | :arrow_up: | | [airflow/utils/module\_loading.py](https://codecov.io/gh/apache/incubator-airflow/pull/4352/diff?src=pr=tree#diff-YWlyZmxvdy91dGlscy9tb2R1bGVfbG9hZGluZy5weQ==) | `100% <0%> (+10.52%)` | :arrow_up: | | [airflow/utils/decorators.py](https://codecov.io/gh/apache/incubator-airflow/pull/4352/diff?src=pr=tree#diff-YWlyZmxvdy91dGlscy9kZWNvcmF0b3JzLnB5) | `91.66% <0%> (+14.58%)` | :arrow_up: | | [airflow/settings.py](https://codecov.io/gh/apache/incubator-airflow/pull/4352/diff?src=pr=tree#diff-YWlyZmxvdy9zZXR0aW5ncy5weQ==) | `80.41% <0%> (+14.68%)` | :arrow_up: | | [airflow/hooks/oracle\_hook.py](https://codecov.io/gh/apache/incubator-airflow/pull/4352/diff?src=pr=tree#diff-YWlyZmxvdy9ob29rcy9vcmFjbGVfaG9vay5weQ==) | `15.47% <0%> (+15.47%)` | :arrow_up: | | [airflow/\_\_init\_\_.py](https://codecov.io/gh/apache/incubator-airflow/pull/4352/diff?src=pr=tree#diff-YWlyZmxvdy9fX2luaXRfXy5weQ==) | `80% <0%> (+15.55%)` | :arrow_up: | | ... and [161 more](https://codecov.io/gh/apache/incubator-airflow/pull/4352/diff?src=pr=tree-more) | | -- [Continue to review full report at Codecov](https://codecov.io/gh/apache/incubator-airflow/pull/4352?src=pr=continue). > **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta) > `Δ = absolute (impact)`, `ø = not affected`, `? = missing data` > Powered by [Codecov](https://codecov.io/gh/apache/incubator-airflow/pull/4352?src=pr=footer). Last update [f211390...b375413](https://codecov.io/gh/apache/incubator-airflow/pull/4352?src=pr=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments). This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] codecov-io commented on issue #4352: [AIRFLOW-3555] Remove lxml dependency
codecov-io commented on issue #4352: [AIRFLOW-3555] Remove lxml dependency URL: https://github.com/apache/incubator-airflow/pull/4352#issuecomment-449450870 # [Codecov](https://codecov.io/gh/apache/incubator-airflow/pull/4352?src=pr=h1) Report > Merging [#4352](https://codecov.io/gh/apache/incubator-airflow/pull/4352?src=pr=desc) into [master](https://codecov.io/gh/apache/incubator-airflow/commit/f2113906d052df818cf14be8c513fada91caf26a?src=pr=desc) will **increase** coverage by `61.99%`. > The diff coverage is `n/a`. [](https://codecov.io/gh/apache/incubator-airflow/pull/4352?src=pr=tree) ```diff @@ Coverage Diff @@ ## master#4352 +/- ## === + Coverage 16.13% 78.12% +61.99% === Files 202 202 Lines 1648316483 === + Hits 266012878+10218 + Misses 13823 3605-10218 ``` | [Impacted Files](https://codecov.io/gh/apache/incubator-airflow/pull/4352?src=pr=tree) | Coverage Δ | | |---|---|---| | [airflow/plugins\_manager.py](https://codecov.io/gh/apache/incubator-airflow/pull/4352/diff?src=pr=tree#diff-YWlyZmxvdy9wbHVnaW5zX21hbmFnZXIucHk=) | `93.1% <0%> (+1.14%)` | :arrow_up: | | [airflow/executors/dask\_executor.py](https://codecov.io/gh/apache/incubator-airflow/pull/4352/diff?src=pr=tree#diff-YWlyZmxvdy9leGVjdXRvcnMvZGFza19leGVjdXRvci5weQ==) | `2% <0%> (+2%)` | :arrow_up: | | [airflow/exceptions.py](https://codecov.io/gh/apache/incubator-airflow/pull/4352/diff?src=pr=tree#diff-YWlyZmxvdy9leGNlcHRpb25zLnB5) | `100% <0%> (+2.85%)` | :arrow_up: | | [airflow/utils/operator\_resources.py](https://codecov.io/gh/apache/incubator-airflow/pull/4352/diff?src=pr=tree#diff-YWlyZmxvdy91dGlscy9vcGVyYXRvcl9yZXNvdXJjZXMucHk=) | `86.95% <0%> (+4.34%)` | :arrow_up: | | [airflow/executors/\_\_init\_\_.py](https://codecov.io/gh/apache/incubator-airflow/pull/4352/diff?src=pr=tree#diff-YWlyZmxvdy9leGVjdXRvcnMvX19pbml0X18ucHk=) | `63.46% <0%> (+5.76%)` | :arrow_up: | | [airflow/utils/module\_loading.py](https://codecov.io/gh/apache/incubator-airflow/pull/4352/diff?src=pr=tree#diff-YWlyZmxvdy91dGlscy9tb2R1bGVfbG9hZGluZy5weQ==) | `100% <0%> (+10.52%)` | :arrow_up: | | [airflow/utils/decorators.py](https://codecov.io/gh/apache/incubator-airflow/pull/4352/diff?src=pr=tree#diff-YWlyZmxvdy91dGlscy9kZWNvcmF0b3JzLnB5) | `91.66% <0%> (+14.58%)` | :arrow_up: | | [airflow/settings.py](https://codecov.io/gh/apache/incubator-airflow/pull/4352/diff?src=pr=tree#diff-YWlyZmxvdy9zZXR0aW5ncy5weQ==) | `80.41% <0%> (+14.68%)` | :arrow_up: | | [airflow/hooks/oracle\_hook.py](https://codecov.io/gh/apache/incubator-airflow/pull/4352/diff?src=pr=tree#diff-YWlyZmxvdy9ob29rcy9vcmFjbGVfaG9vay5weQ==) | `15.47% <0%> (+15.47%)` | :arrow_up: | | [airflow/\_\_init\_\_.py](https://codecov.io/gh/apache/incubator-airflow/pull/4352/diff?src=pr=tree#diff-YWlyZmxvdy9fX2luaXRfXy5weQ==) | `80% <0%> (+15.55%)` | :arrow_up: | | ... and [161 more](https://codecov.io/gh/apache/incubator-airflow/pull/4352/diff?src=pr=tree-more) | | -- [Continue to review full report at Codecov](https://codecov.io/gh/apache/incubator-airflow/pull/4352?src=pr=continue). > **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta) > `Δ = absolute (impact)`, `ø = not affected`, `? = missing data` > Powered by [Codecov](https://codecov.io/gh/apache/incubator-airflow/pull/4352?src=pr=footer). Last update [f211390...b375413](https://codecov.io/gh/apache/incubator-airflow/pull/4352?src=pr=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments). This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] potiuk opened a new pull request #4353: [AIRFLOW-3480] Added Database Deploy/Update/Delete operators
potiuk opened a new pull request #4353: [AIRFLOW-3480] Added Database
Deploy/Update/Delete operators
URL: https://github.com/apache/incubator-airflow/pull/4353
Make sure you have checked _all_ steps below.
### Jira
- [x] My PR addresses the following [Airflow
Jira](https://issues.apache.org/jira/browse/AIRFLOW/AIRFLOW-3480) issue and
references them in the PR title.
### Description
- [x] Here are some details about my PR, including screenshots of any UI
changes:
Added Database Deploy/Update/Delete operators for Google Cloud Spanner
### Tests
- [x] My PR adds the following unit tests:
- test_database_create
- test_database_create_with_pre_existing_db
- test_database_create_ex_if_param_missing(parameterised)
- test_database_update
- test_database_update_ex_if_param_missing(parameterised)
- test_database_update_ex_if_database_not_exist
- test_database_delete
- test_database_delete_exits_and_succeeds_if_database_does_not_exist
- test_database_delete_ex_if_param_missing (parameterised)
### Commits
- [x] My commits all reference Jira issues in their subject lines, and I
have squashed multiple commits if they address the same issue. In addition, my
commits follow the guidelines from "[How to write a good git commit
message](http://chris.beams.io/posts/git-commit/)":
1. Subject is separated from body by a blank line
1. Subject is limited to 50 characters (not including Jira issue reference)
1. Subject does not end with a period
1. Subject uses the imperative mood ("add", not "adding")
1. Body wraps at 72 characters
1. Body explains "what" and "why", not "how"
### Documentation
- [x] In case of new functionality, my PR adds documentation that describes
how to use it.
- When adding new operators/hooks/sensors, the autoclass documentation
generation needs to be added.
- All the public functions and the classes in the PR contain docstrings
that explain what it does
### Code Quality
- [x] Passes `flake8`
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] jcao219 opened a new pull request #4352: [AIRFLOW-3555] Remove lxml dependency
jcao219 opened a new pull request #4352: [AIRFLOW-3555] Remove lxml dependency
URL: https://github.com/apache/incubator-airflow/pull/4352
Make sure you have checked _all_ steps below.
### Jira
- [x] My PR addresses the following [Airflow
Jira](https://issues.apache.org/jira/browse/AIRFLOW/) issues and references
them in the PR title. For example, "\[AIRFLOW-XXX\] My Airflow PR"
- https://issues.apache.org/jira/browse/AIRFLOW-3555
### Description
- [x] The lxml dependency is no longer needed except for when running tests.
### Tests
- [x] My PR adds the following unit tests __OR__ does not need testing for
this extremely good reason: Dependency clean up
### Commits
- [x] My commits all reference Jira issues in their subject lines, and I
have squashed multiple commits if they address the same issue. In addition, my
commits follow the guidelines from "[How to write a good git commit
message](http://chris.beams.io/posts/git-commit/)":
1. Subject is separated from body by a blank line
1. Subject is limited to 50 characters (not including Jira issue reference)
1. Subject does not end with a period
1. Subject uses the imperative mood ("add", not "adding")
1. Body wraps at 72 characters
1. Body explains "what" and "why", not "how"
### Documentation
- [x] In case of new functionality, my PR adds documentation that describes
how to use it.
- When adding new operators/hooks/sensors, the autoclass documentation
generation needs to be added.
- All the public functions and the classes in the PR contain docstrings
that explain what it does
### Code Quality
- [x] Passes `flake8`
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] codecov-io edited a comment on issue #4350: [AIRFLOW-3527] Cloud SQL Proxy has shorter path for UNIX socket
codecov-io edited a comment on issue #4350: [AIRFLOW-3527] Cloud SQL Proxy has shorter path for UNIX socket URL: https://github.com/apache/incubator-airflow/pull/4350#issuecomment-449404187 # [Codecov](https://codecov.io/gh/apache/incubator-airflow/pull/4350?src=pr=h1) Report > Merging [#4350](https://codecov.io/gh/apache/incubator-airflow/pull/4350?src=pr=desc) into [master](https://codecov.io/gh/apache/incubator-airflow/commit/f2113906d052df818cf14be8c513fada91caf26a?src=pr=desc) will **increase** coverage by `61.99%`. > The diff coverage is `n/a`. [](https://codecov.io/gh/apache/incubator-airflow/pull/4350?src=pr=tree) ```diff @@ Coverage Diff @@ ## master#4350 +/- ## === + Coverage 16.13% 78.12% +61.99% === Files 202 202 Lines 1648316483 === + Hits 266012878+10218 + Misses 13823 3605-10218 ``` | [Impacted Files](https://codecov.io/gh/apache/incubator-airflow/pull/4350?src=pr=tree) | Coverage Δ | | |---|---|---| | [airflow/plugins\_manager.py](https://codecov.io/gh/apache/incubator-airflow/pull/4350/diff?src=pr=tree#diff-YWlyZmxvdy9wbHVnaW5zX21hbmFnZXIucHk=) | `93.1% <0%> (+1.14%)` | :arrow_up: | | [airflow/executors/dask\_executor.py](https://codecov.io/gh/apache/incubator-airflow/pull/4350/diff?src=pr=tree#diff-YWlyZmxvdy9leGVjdXRvcnMvZGFza19leGVjdXRvci5weQ==) | `2% <0%> (+2%)` | :arrow_up: | | [airflow/exceptions.py](https://codecov.io/gh/apache/incubator-airflow/pull/4350/diff?src=pr=tree#diff-YWlyZmxvdy9leGNlcHRpb25zLnB5) | `100% <0%> (+2.85%)` | :arrow_up: | | [airflow/utils/operator\_resources.py](https://codecov.io/gh/apache/incubator-airflow/pull/4350/diff?src=pr=tree#diff-YWlyZmxvdy91dGlscy9vcGVyYXRvcl9yZXNvdXJjZXMucHk=) | `86.95% <0%> (+4.34%)` | :arrow_up: | | [airflow/executors/\_\_init\_\_.py](https://codecov.io/gh/apache/incubator-airflow/pull/4350/diff?src=pr=tree#diff-YWlyZmxvdy9leGVjdXRvcnMvX19pbml0X18ucHk=) | `63.46% <0%> (+5.76%)` | :arrow_up: | | [airflow/utils/module\_loading.py](https://codecov.io/gh/apache/incubator-airflow/pull/4350/diff?src=pr=tree#diff-YWlyZmxvdy91dGlscy9tb2R1bGVfbG9hZGluZy5weQ==) | `100% <0%> (+10.52%)` | :arrow_up: | | [airflow/utils/decorators.py](https://codecov.io/gh/apache/incubator-airflow/pull/4350/diff?src=pr=tree#diff-YWlyZmxvdy91dGlscy9kZWNvcmF0b3JzLnB5) | `91.66% <0%> (+14.58%)` | :arrow_up: | | [airflow/settings.py](https://codecov.io/gh/apache/incubator-airflow/pull/4350/diff?src=pr=tree#diff-YWlyZmxvdy9zZXR0aW5ncy5weQ==) | `80.41% <0%> (+14.68%)` | :arrow_up: | | [airflow/hooks/oracle\_hook.py](https://codecov.io/gh/apache/incubator-airflow/pull/4350/diff?src=pr=tree#diff-YWlyZmxvdy9ob29rcy9vcmFjbGVfaG9vay5weQ==) | `15.47% <0%> (+15.47%)` | :arrow_up: | | [airflow/\_\_init\_\_.py](https://codecov.io/gh/apache/incubator-airflow/pull/4350/diff?src=pr=tree#diff-YWlyZmxvdy9fX2luaXRfXy5weQ==) | `80% <0%> (+15.55%)` | :arrow_up: | | ... and [161 more](https://codecov.io/gh/apache/incubator-airflow/pull/4350/diff?src=pr=tree-more) | | -- [Continue to review full report at Codecov](https://codecov.io/gh/apache/incubator-airflow/pull/4350?src=pr=continue). > **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta) > `Δ = absolute (impact)`, `ø = not affected`, `? = missing data` > Powered by [Codecov](https://codecov.io/gh/apache/incubator-airflow/pull/4350?src=pr=footer). Last update [f211390...ccb3721](https://codecov.io/gh/apache/incubator-airflow/pull/4350?src=pr=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments). This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] omusavi commented on issue #4121: [AIRFLOW-2568] Azure Container Instances operator
omusavi commented on issue #4121: [AIRFLOW-2568] Azure Container Instances operator URL: https://github.com/apache/incubator-airflow/pull/4121#issuecomment-449416747 Thanks to all the previous commenters! The PR has now been updated to include full unit testing for all the hooks. I have also rebased against the latest master and pulled in the connections.py changes. Thanks @Fokko and @feluelle for the comments, please review when you get a chance. Happy holidays! This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] codecov-io commented on issue #4351: [AIRFLOW-3554] Remove contrib folder from code cov omit list
codecov-io commented on issue #4351: [AIRFLOW-3554] Remove contrib folder from code cov omit list URL: https://github.com/apache/incubator-airflow/pull/4351#issuecomment-449412224 # [Codecov](https://codecov.io/gh/apache/incubator-airflow/pull/4351?src=pr=h1) Report > Merging [#4351](https://codecov.io/gh/apache/incubator-airflow/pull/4351?src=pr=desc) into [master](https://codecov.io/gh/apache/incubator-airflow/commit/f2113906d052df818cf14be8c513fada91caf26a?src=pr=desc) will **increase** coverage by `57.57%`. > The diff coverage is `n/a`. [](https://codecov.io/gh/apache/incubator-airflow/pull/4351?src=pr=tree) ```diff @@Coverage Diff @@ ## master #4351 +/- ## == + Coverage 16.13% 73.7% +57.57% == Files 202 411 +209 Lines 16483 28786+12303 == + Hits 2660 21218+18558 + Misses 138237568 -6255 ``` | [Impacted Files](https://codecov.io/gh/apache/incubator-airflow/pull/4351?src=pr=tree) | Coverage Δ | | |---|---|---| | [...ow/contrib/example\_dags/example\_qubole\_operator.py](https://codecov.io/gh/apache/incubator-airflow/pull/4351/diff?src=pr=tree#diff-YWlyZmxvdy9jb250cmliL2V4YW1wbGVfZGFncy9leGFtcGxlX3F1Ym9sZV9vcGVyYXRvci5weQ==) | `0% <0%> (ø)` | | | [airflow/contrib/operators/s3\_list\_operator.py](https://codecov.io/gh/apache/incubator-airflow/pull/4351/diff?src=pr=tree#diff-YWlyZmxvdy9jb250cmliL29wZXJhdG9ycy9zM19saXN0X29wZXJhdG9yLnB5) | `100% <0%> (ø)` | | | [...low/contrib/operators/wasb\_delete\_blob\_operator.py](https://codecov.io/gh/apache/incubator-airflow/pull/4351/diff?src=pr=tree#diff-YWlyZmxvdy9jb250cmliL29wZXJhdG9ycy93YXNiX2RlbGV0ZV9ibG9iX29wZXJhdG9yLnB5) | `100% <0%> (ø)` | | | [airflow/contrib/hooks/vertica\_hook.py](https://codecov.io/gh/apache/incubator-airflow/pull/4351/diff?src=pr=tree#diff-YWlyZmxvdy9jb250cmliL2hvb2tzL3ZlcnRpY2FfaG9vay5weQ==) | `50% <0%> (ø)` | | | [...flow/contrib/example\_dags/example\_qubole\_sensor.py](https://codecov.io/gh/apache/incubator-airflow/pull/4351/diff?src=pr=tree#diff-YWlyZmxvdy9jb250cmliL2V4YW1wbGVfZGFncy9leGFtcGxlX3F1Ym9sZV9zZW5zb3IucHk=) | `0% <0%> (ø)` | | | [airflow/contrib/sensors/pubsub\_sensor.py](https://codecov.io/gh/apache/incubator-airflow/pull/4351/diff?src=pr=tree#diff-YWlyZmxvdy9jb250cmliL3NlbnNvcnMvcHVic3ViX3NlbnNvci5weQ==) | `100% <0%> (ø)` | | | [airflow/contrib/hooks/qubole\_hook.py](https://codecov.io/gh/apache/incubator-airflow/pull/4351/diff?src=pr=tree#diff-YWlyZmxvdy9jb250cmliL2hvb2tzL3F1Ym9sZV9ob29rLnB5) | `53.65% <0%> (ø)` | | | [airflow/contrib/hooks/databricks\_hook.py](https://codecov.io/gh/apache/incubator-airflow/pull/4351/diff?src=pr=tree#diff-YWlyZmxvdy9jb250cmliL2hvb2tzL2RhdGFicmlja3NfaG9vay5weQ==) | `95.61% <0%> (ø)` | | | [airflow/contrib/hooks/aws\_firehose\_hook.py](https://codecov.io/gh/apache/incubator-airflow/pull/4351/diff?src=pr=tree#diff-YWlyZmxvdy9jb250cmliL2hvb2tzL2F3c19maXJlaG9zZV9ob29rLnB5) | `100% <0%> (ø)` | | | [airflow/contrib/sensors/wasb\_sensor.py](https://codecov.io/gh/apache/incubator-airflow/pull/4351/diff?src=pr=tree#diff-YWlyZmxvdy9jb250cmliL3NlbnNvcnMvd2FzYl9zZW5zb3IucHk=) | `100% <0%> (ø)` | | | ... and [370 more](https://codecov.io/gh/apache/incubator-airflow/pull/4351/diff?src=pr=tree-more) | | -- [Continue to review full report at Codecov](https://codecov.io/gh/apache/incubator-airflow/pull/4351?src=pr=continue). > **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta) > `Δ = absolute (impact)`, `ø = not affected`, `? = missing data` > Powered by [Codecov](https://codecov.io/gh/apache/incubator-airflow/pull/4351?src=pr=footer). Last update [f211390...cc6a49a](https://codecov.io/gh/apache/incubator-airflow/pull/4351?src=pr=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments). This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] codecov-io edited a comment on issue #4351: [AIRFLOW-3554] Remove contrib folder from code cov omit list
codecov-io edited a comment on issue #4351: [AIRFLOW-3554] Remove contrib folder from code cov omit list URL: https://github.com/apache/incubator-airflow/pull/4351#issuecomment-449412224 # [Codecov](https://codecov.io/gh/apache/incubator-airflow/pull/4351?src=pr=h1) Report > Merging [#4351](https://codecov.io/gh/apache/incubator-airflow/pull/4351?src=pr=desc) into [master](https://codecov.io/gh/apache/incubator-airflow/commit/f2113906d052df818cf14be8c513fada91caf26a?src=pr=desc) will **increase** coverage by `57.57%`. > The diff coverage is `n/a`. [](https://codecov.io/gh/apache/incubator-airflow/pull/4351?src=pr=tree) ```diff @@Coverage Diff @@ ## master #4351 +/- ## == + Coverage 16.13% 73.7% +57.57% == Files 202 411 +209 Lines 16483 28786+12303 == + Hits 2660 21218+18558 + Misses 138237568 -6255 ``` | [Impacted Files](https://codecov.io/gh/apache/incubator-airflow/pull/4351?src=pr=tree) | Coverage Δ | | |---|---|---| | [...ow/contrib/example\_dags/example\_qubole\_operator.py](https://codecov.io/gh/apache/incubator-airflow/pull/4351/diff?src=pr=tree#diff-YWlyZmxvdy9jb250cmliL2V4YW1wbGVfZGFncy9leGFtcGxlX3F1Ym9sZV9vcGVyYXRvci5weQ==) | `0% <0%> (ø)` | | | [airflow/contrib/operators/s3\_list\_operator.py](https://codecov.io/gh/apache/incubator-airflow/pull/4351/diff?src=pr=tree#diff-YWlyZmxvdy9jb250cmliL29wZXJhdG9ycy9zM19saXN0X29wZXJhdG9yLnB5) | `100% <0%> (ø)` | | | [...low/contrib/operators/wasb\_delete\_blob\_operator.py](https://codecov.io/gh/apache/incubator-airflow/pull/4351/diff?src=pr=tree#diff-YWlyZmxvdy9jb250cmliL29wZXJhdG9ycy93YXNiX2RlbGV0ZV9ibG9iX29wZXJhdG9yLnB5) | `100% <0%> (ø)` | | | [airflow/contrib/hooks/vertica\_hook.py](https://codecov.io/gh/apache/incubator-airflow/pull/4351/diff?src=pr=tree#diff-YWlyZmxvdy9jb250cmliL2hvb2tzL3ZlcnRpY2FfaG9vay5weQ==) | `50% <0%> (ø)` | | | [...flow/contrib/example\_dags/example\_qubole\_sensor.py](https://codecov.io/gh/apache/incubator-airflow/pull/4351/diff?src=pr=tree#diff-YWlyZmxvdy9jb250cmliL2V4YW1wbGVfZGFncy9leGFtcGxlX3F1Ym9sZV9zZW5zb3IucHk=) | `0% <0%> (ø)` | | | [airflow/contrib/sensors/pubsub\_sensor.py](https://codecov.io/gh/apache/incubator-airflow/pull/4351/diff?src=pr=tree#diff-YWlyZmxvdy9jb250cmliL3NlbnNvcnMvcHVic3ViX3NlbnNvci5weQ==) | `100% <0%> (ø)` | | | [airflow/contrib/hooks/qubole\_hook.py](https://codecov.io/gh/apache/incubator-airflow/pull/4351/diff?src=pr=tree#diff-YWlyZmxvdy9jb250cmliL2hvb2tzL3F1Ym9sZV9ob29rLnB5) | `53.65% <0%> (ø)` | | | [airflow/contrib/hooks/databricks\_hook.py](https://codecov.io/gh/apache/incubator-airflow/pull/4351/diff?src=pr=tree#diff-YWlyZmxvdy9jb250cmliL2hvb2tzL2RhdGFicmlja3NfaG9vay5weQ==) | `95.61% <0%> (ø)` | | | [airflow/contrib/hooks/aws\_firehose\_hook.py](https://codecov.io/gh/apache/incubator-airflow/pull/4351/diff?src=pr=tree#diff-YWlyZmxvdy9jb250cmliL2hvb2tzL2F3c19maXJlaG9zZV9ob29rLnB5) | `100% <0%> (ø)` | | | [airflow/contrib/sensors/wasb\_sensor.py](https://codecov.io/gh/apache/incubator-airflow/pull/4351/diff?src=pr=tree#diff-YWlyZmxvdy9jb250cmliL3NlbnNvcnMvd2FzYl9zZW5zb3IucHk=) | `100% <0%> (ø)` | | | ... and [370 more](https://codecov.io/gh/apache/incubator-airflow/pull/4351/diff?src=pr=tree-more) | | -- [Continue to review full report at Codecov](https://codecov.io/gh/apache/incubator-airflow/pull/4351?src=pr=continue). > **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta) > `Δ = absolute (impact)`, `ø = not affected`, `? = missing data` > Powered by [Codecov](https://codecov.io/gh/apache/incubator-airflow/pull/4351?src=pr=footer). Last update [f211390...cc6a49a](https://codecov.io/gh/apache/incubator-airflow/pull/4351?src=pr=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments). This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] codecov-io commented on issue #4350: [AIRFLOW-3527] Cloud SQL Proxy has shorter path for UNIX socket
codecov-io commented on issue #4350: [AIRFLOW-3527] Cloud SQL Proxy has shorter path for UNIX socket URL: https://github.com/apache/incubator-airflow/pull/4350#issuecomment-449404187 # [Codecov](https://codecov.io/gh/apache/incubator-airflow/pull/4350?src=pr=h1) Report > Merging [#4350](https://codecov.io/gh/apache/incubator-airflow/pull/4350?src=pr=desc) into [master](https://codecov.io/gh/apache/incubator-airflow/commit/f2113906d052df818cf14be8c513fada91caf26a?src=pr=desc) will **increase** coverage by `61.99%`. > The diff coverage is `n/a`. [](https://codecov.io/gh/apache/incubator-airflow/pull/4350?src=pr=tree) ```diff @@ Coverage Diff @@ ## master#4350 +/- ## === + Coverage 16.13% 78.12% +61.99% === Files 202 202 Lines 1648316483 === + Hits 266012878+10218 + Misses 13823 3605-10218 ``` | [Impacted Files](https://codecov.io/gh/apache/incubator-airflow/pull/4350?src=pr=tree) | Coverage Δ | | |---|---|---| | [airflow/plugins\_manager.py](https://codecov.io/gh/apache/incubator-airflow/pull/4350/diff?src=pr=tree#diff-YWlyZmxvdy9wbHVnaW5zX21hbmFnZXIucHk=) | `93.1% <0%> (+1.14%)` | :arrow_up: | | [airflow/executors/dask\_executor.py](https://codecov.io/gh/apache/incubator-airflow/pull/4350/diff?src=pr=tree#diff-YWlyZmxvdy9leGVjdXRvcnMvZGFza19leGVjdXRvci5weQ==) | `2% <0%> (+2%)` | :arrow_up: | | [airflow/exceptions.py](https://codecov.io/gh/apache/incubator-airflow/pull/4350/diff?src=pr=tree#diff-YWlyZmxvdy9leGNlcHRpb25zLnB5) | `100% <0%> (+2.85%)` | :arrow_up: | | [airflow/utils/operator\_resources.py](https://codecov.io/gh/apache/incubator-airflow/pull/4350/diff?src=pr=tree#diff-YWlyZmxvdy91dGlscy9vcGVyYXRvcl9yZXNvdXJjZXMucHk=) | `86.95% <0%> (+4.34%)` | :arrow_up: | | [airflow/executors/\_\_init\_\_.py](https://codecov.io/gh/apache/incubator-airflow/pull/4350/diff?src=pr=tree#diff-YWlyZmxvdy9leGVjdXRvcnMvX19pbml0X18ucHk=) | `63.46% <0%> (+5.76%)` | :arrow_up: | | [airflow/utils/module\_loading.py](https://codecov.io/gh/apache/incubator-airflow/pull/4350/diff?src=pr=tree#diff-YWlyZmxvdy91dGlscy9tb2R1bGVfbG9hZGluZy5weQ==) | `100% <0%> (+10.52%)` | :arrow_up: | | [airflow/utils/decorators.py](https://codecov.io/gh/apache/incubator-airflow/pull/4350/diff?src=pr=tree#diff-YWlyZmxvdy91dGlscy9kZWNvcmF0b3JzLnB5) | `91.66% <0%> (+14.58%)` | :arrow_up: | | [airflow/settings.py](https://codecov.io/gh/apache/incubator-airflow/pull/4350/diff?src=pr=tree#diff-YWlyZmxvdy9zZXR0aW5ncy5weQ==) | `80.41% <0%> (+14.68%)` | :arrow_up: | | [airflow/hooks/oracle\_hook.py](https://codecov.io/gh/apache/incubator-airflow/pull/4350/diff?src=pr=tree#diff-YWlyZmxvdy9ob29rcy9vcmFjbGVfaG9vay5weQ==) | `15.47% <0%> (+15.47%)` | :arrow_up: | | [airflow/\_\_init\_\_.py](https://codecov.io/gh/apache/incubator-airflow/pull/4350/diff?src=pr=tree#diff-YWlyZmxvdy9fX2luaXRfXy5weQ==) | `80% <0%> (+15.55%)` | :arrow_up: | | ... and [161 more](https://codecov.io/gh/apache/incubator-airflow/pull/4350/diff?src=pr=tree-more) | | -- [Continue to review full report at Codecov](https://codecov.io/gh/apache/incubator-airflow/pull/4350?src=pr=continue). > **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta) > `Δ = absolute (impact)`, `ø = not affected`, `? = missing data` > Powered by [Codecov](https://codecov.io/gh/apache/incubator-airflow/pull/4350?src=pr=footer). Last update [f211390...16b422c](https://codecov.io/gh/apache/incubator-airflow/pull/4350?src=pr=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments). This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] feluelle opened a new pull request #4351: [AIRFLOW-3554] Remove contrib folder from code cov omit list
feluelle opened a new pull request #4351: [AIRFLOW-3554] Remove contrib folder
from code cov omit list
URL: https://github.com/apache/incubator-airflow/pull/4351
Make sure you have checked _all_ steps below.
### Jira
- [x] My PR addresses the following [Airflow
Jira](https://issues.apache.org/jira/browse/AIRFLOW/) issues and references
them in the PR title. For example, "\[AIRFLOW-XXX\] My Airflow PR"
- https://issues.apache.org/jira/browse/AIRFLOW-3554
- In case you are fixing a typo in the documentation you can prepend your
commit with \[AIRFLOW-XXX\], code changes always need a Jira issue.
### Description
- [x] Here are some details about my PR, including screenshots of any UI
changes:
Currently the `contrib` folder is not being processed by codecov.
That means that contributors won't see a code coverage of their implemented
code in this folder.
To generally improve code/test coverage for this project I would recommend
to enable coverage for the `contrib` folder, too.
### Tests
- [x] My PR adds the following unit tests __OR__ does not need testing for
this extremely good reason:
### Commits
- [x] My commits all reference Jira issues in their subject lines, and I
have squashed multiple commits if they address the same issue. In addition, my
commits follow the guidelines from "[How to write a good git commit
message](http://chris.beams.io/posts/git-commit/)":
1. Subject is separated from body by a blank line
1. Subject is limited to 50 characters (not including Jira issue reference)
1. Subject does not end with a period
1. Subject uses the imperative mood ("add", not "adding")
1. Body wraps at 72 characters
1. Body explains "what" and "why", not "how"
### Documentation
- [x] In case of new functionality, my PR adds documentation that describes
how to use it.
- When adding new operators/hooks/sensors, the autoclass documentation
generation needs to be added.
- All the public functions and the classes in the PR contain docstrings
that explain what it does
### Code Quality
- [x] Passes `flake8`
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] ashb commented on a change in pull request #4349: [AIRFLOW-850] Add a PythonSensor
ashb commented on a change in pull request #4349: [AIRFLOW-850] Add a
PythonSensor
URL: https://github.com/apache/incubator-airflow/pull/4349#discussion_r243594087
##
File path: airflow/contrib/sensors/python_sensor.py
##
@@ -0,0 +1,76 @@
+# -*- coding: utf-8 -*-
+#
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+
+from airflow.sensors.base_sensor_operator import BaseSensorOperator
+from airflow.utils.decorators import apply_defaults
+
+
+class PythonSensor(BaseSensorOperator):
+"""
+Waits for a Python callable to return True
+
+:param python_callable: A reference to an object that is callable
+:type python_callable: python callable
+:param op_kwargs: a dictionary of keyword arguments that will get unpacked
+in your function
+:type op_kwargs: dict
+:param op_args: a list of positional arguments that will get unpacked when
+calling your callable
+:type op_args: list
+:param provide_context: if set to true, Airflow will pass a set of
+keyword arguments that can be used in your function. This set of
+kwargs correspond exactly to what you can use in your jinja
+templates. For this to work, you need to define `**kwargs` in your
+function header.
+:type provide_context: bool
+:param templates_dict: a dictionary where the values are templates that
+will get templated by the Airflow engine sometime between
+``__init__`` and ``execute`` takes place and are made available
+in your callable's context after the template has been applied
+:type templates_dict: dict of str
+"""
+
+template_fields = ('templates_dict',)
+template_ext = tuple()
+
+@apply_defaults
+def __init__(
+self,
+python_callable,
+op_args=None,
+op_kwargs=None,
+provide_context=False,
+templates_dict=None,
+*args, **kwargs):
+super(PythonSensor, self).__init__(*args, **kwargs)
+self.python_callable = python_callable
+self.op_args = op_args or []
+self.op_kwargs = op_kwargs or {}
+self.provide_context = provide_context
+self.templates_dict = templates_dict
+
+def poke(self, context):
+if self.provide_context:
+context.update(self.op_kwargs)
+context['templates_dict'] = self.templates_dict
+self.op_kwargs = context
Review comment:
Does it matter that we end up doing this each time we poke?
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] ashb commented on a change in pull request #4349: [AIRFLOW-850] Add a PythonSensor
ashb commented on a change in pull request #4349: [AIRFLOW-850] Add a PythonSensor URL: https://github.com/apache/incubator-airflow/pull/4349#discussion_r243593636 ## File path: airflow/contrib/sensors/python_sensor.py ## @@ -0,0 +1,76 @@ +# -*- coding: utf-8 -*- +# +# Licensed to the Apache Software Foundation (ASF) under one +# or more contributor license agreements. See the NOTICE file +# distributed with this work for additional information +# regarding copyright ownership. The ASF licenses this file +# to you under the Apache License, Version 2.0 (the +# "License"); you may not use this file except in compliance +# with the License. You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, +# software distributed under the License is distributed on an +# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY +# KIND, either express or implied. See the License for the +# specific language governing permissions and limitations +# under the License. + +from airflow.sensors.base_sensor_operator import BaseSensorOperator +from airflow.utils.decorators import apply_defaults + + +class PythonSensor(BaseSensorOperator): +""" +Waits for a Python callable to return True Review comment: Adding a short example here in the docs specifying what arguments are passed to the callable, how to access templates_ditc etc would be good I think This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] ashb commented on a change in pull request #4225: [AIRFLOW-3383] Rotate fernet keys.
ashb commented on a change in pull request #4225: [AIRFLOW-3383] Rotate fernet
keys.
URL: https://github.com/apache/incubator-airflow/pull/4225#discussion_r243591064
##
File path: airflow/bin/cli.py
##
@@ -2109,7 +2119,12 @@ class CLIFactory(object):
'func': next_execution,
'help': "Get the next execution datetime of a DAG.",
'args': ('dag_id', 'subdir')
-}
+},
+{
+'func': rotate_fernet_key,
+'help': 'Rotate all encrypted connection credentials and
variables.',
Review comment:
I don't see us rotating the encryption on variables anywhere?
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] potiuk opened a new pull request #4350: [AIRFLOW-3527] Cloud SQL Proxy has shorter path for UNIX socket
potiuk opened a new pull request #4350: [AIRFLOW-3527] Cloud SQL Proxy has
shorter path for UNIX socket
URL: https://github.com/apache/incubator-airflow/pull/4350
Make sure you have checked _all_ steps below.
### Jira
- [ ] My PR addresses the following [Airflow
Jira](https://issues.apache.org/jira/browse/AIRFLOW-3527)
### Description
- [x] Here are some details about my PR, including screenshots of any UI
changes:
There is a limitation of UNIX socket path length as described in
http://pubs.opengroup.org/onlinepubs/9699919799/basedefs/sys_un.h.html#tag_13_67_04
Cloud SQL Proxy uses generated path and it can get longer than the limit
in case of POSTGRES connections especially (POSTGRES adds few characters on
its own). The error returned by sqlproxy in this case is pretty vague
(invalid path) - it makes it difficult to understand the problem by
the user.
This commit fixes it in two ways:
* makes it less likely that the path length will be exceeded
by shorter random string generated for the socket directory.
* raises an Error in case of calculated path is too long
### Tests
- [x] My PR adds the following unit tests:
CloudSqlQueryValidationTest:
* test_create_operator_with_too_long_unix_socket_path
* test_create_operator_with_not_too_long_unix_socket_path
### Commits
- [x] My commits all reference Jira issues in their subject lines, and I
have squashed multiple commits if they address the same issue. In addition, my
commits follow the guidelines from "[How to write a good git commit
message](http://chris.beams.io/posts/git-commit/)":
1. Subject is separated from body by a blank line
1. Subject is limited to 50 characters (not including Jira issue reference)
1. Subject does not end with a period
1. Subject uses the imperative mood ("add", not "adding")
1. Body wraps at 72 characters
1. Body explains "what" and "why", not "how"
### Documentation
- [x] No documentation update is needed.
### Code Quality
- [x] Passes `flake8`
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
