[GitHub] [airflow] feng-tao commented on issue #4973: [AIRFLOW-4155] Allow Public role access to /home
feng-tao commented on issue #4973: [AIRFLOW-4155] Allow Public role access to /home URL: https://github.com/apache/airflow/pull/4973#issuecomment-476982602 ping @ashb This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Resolved] (AIRFLOW-4118) Instrument dagrun duration
[ https://issues.apache.org/jira/browse/AIRFLOW-4118?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chao-Han Tsai resolved AIRFLOW-4118. Resolution: Fixed > Instrument dagrun duration > -- > > Key: AIRFLOW-4118 > URL: https://issues.apache.org/jira/browse/AIRFLOW-4118 > Project: Apache Airflow > Issue Type: New Feature >Reporter: Chao-Han Tsai >Assignee: Chao-Han Tsai >Priority: Major > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[GitHub] [airflow] XD-DENG commented on issue #4982: [AIRFLOW-4160] Fix redirecting of 'Trigger Dag' Button in DAG Page (.vs v1-10-stable)
XD-DENG commented on issue #4982: [AIRFLOW-4160] Fix redirecting of 'Trigger Dag' Button in DAG Page (.vs v1-10-stable) URL: https://github.com/apache/airflow/pull/4982#issuecomment-476961163 @ashb CI passed. Please take a look. Another PR with the same change against master branch (https://github.com/apache/airflow/pull/4981) was already merged by Feng Tao. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [airflow] codecov-io edited a comment on issue #4982: [AIRFLOW-4160] Fix redirecting of 'Trigger Dag' Button in DAG Page (.vs v1-10-stable)
codecov-io edited a comment on issue #4982: [AIRFLOW-4160] Fix redirecting of 'Trigger Dag' Button in DAG Page (.vs v1-10-stable) URL: https://github.com/apache/airflow/pull/4982#issuecomment-476960816 # [Codecov](https://codecov.io/gh/apache/airflow/pull/4982?src=pr=h1) Report > Merging [#4982](https://codecov.io/gh/apache/airflow/pull/4982?src=pr=desc) into [v1-10-stable](https://codecov.io/gh/apache/airflow/commit/2d9b0fb21faeb984be221a5cdd8cf68b41755981?src=pr=desc) will **increase** coverage by `0.19%`. > The diff coverage is `n/a`. [](https://codecov.io/gh/apache/airflow/pull/4982?src=pr=tree) ```diff @@ Coverage Diff@@ ## v1-10-stable#4982 +/- ## + Coverage 77.87% 78.07% +0.19% Files 221 221 Lines 1752818110 +582 + Hits 1365014139 +489 - Misses 3878 3971 +93 ``` | [Impacted Files](https://codecov.io/gh/apache/airflow/pull/4982?src=pr=tree) | Coverage Δ | | |---|---|---| | [airflow/operators/mysql\_operator.py](https://codecov.io/gh/apache/airflow/pull/4982/diff?src=pr=tree#diff-YWlyZmxvdy9vcGVyYXRvcnMvbXlzcWxfb3BlcmF0b3IucHk=) | `79.16% <0%> (-20.84%)` | :arrow_down: | | [airflow/utils/decorators.py](https://codecov.io/gh/apache/airflow/pull/4982/diff?src=pr=tree#diff-YWlyZmxvdy91dGlscy9kZWNvcmF0b3JzLnB5) | `84.48% <0%> (-7.19%)` | :arrow_down: | | [airflow/utils/dag\_processing.py](https://codecov.io/gh/apache/airflow/pull/4982/diff?src=pr=tree#diff-YWlyZmxvdy91dGlscy9kYWdfcHJvY2Vzc2luZy5weQ==) | `56.17% <0%> (-2.81%)` | :arrow_down: | | [airflow/operators/redshift\_to\_s3\_operator.py](https://codecov.io/gh/apache/airflow/pull/4982/diff?src=pr=tree#diff-YWlyZmxvdy9vcGVyYXRvcnMvcmVkc2hpZnRfdG9fczNfb3BlcmF0b3IucHk=) | `92.98% <0%> (-2.58%)` | :arrow_down: | | [airflow/configuration.py](https://codecov.io/gh/apache/airflow/pull/4982/diff?src=pr=tree#diff-YWlyZmxvdy9jb25maWd1cmF0aW9uLnB5) | `89.04% <0%> (-0.18%)` | :arrow_down: | | [airflow/operators/oracle\_operator.py](https://codecov.io/gh/apache/airflow/pull/4982/diff?src=pr=tree#diff-YWlyZmxvdy9vcGVyYXRvcnMvb3JhY2xlX29wZXJhdG9yLnB5) | `0% <0%> (ø)` | :arrow_up: | | [airflow/sensors/hdfs\_sensor.py](https://codecov.io/gh/apache/airflow/pull/4982/diff?src=pr=tree#diff-YWlyZmxvdy9zZW5zb3JzL2hkZnNfc2Vuc29yLnB5) | `100% <0%> (ø)` | :arrow_up: | | [airflow/operators/s3\_to\_redshift\_operator.py](https://codecov.io/gh/apache/airflow/pull/4982/diff?src=pr=tree#diff-YWlyZmxvdy9vcGVyYXRvcnMvczNfdG9fcmVkc2hpZnRfb3BlcmF0b3IucHk=) | `100% <0%> (ø)` | :arrow_up: | | [airflow/models/\_\_init\_\_.py](https://codecov.io/gh/apache/airflow/pull/4982/diff?src=pr=tree#diff-YWlyZmxvdy9tb2RlbHMvX19pbml0X18ucHk=) | `92.72% <0%> (+0.04%)` | :arrow_up: | | [airflow/hooks/hive\_hooks.py](https://codecov.io/gh/apache/airflow/pull/4982/diff?src=pr=tree#diff-YWlyZmxvdy9ob29rcy9oaXZlX2hvb2tzLnB5) | `75.45% <0%> (+0.12%)` | :arrow_up: | | ... and [9 more](https://codecov.io/gh/apache/airflow/pull/4982/diff?src=pr=tree-more) | | -- [Continue to review full report at Codecov](https://codecov.io/gh/apache/airflow/pull/4982?src=pr=continue). > **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta) > `Δ = absolute (impact)`, `ø = not affected`, `? = missing data` > Powered by [Codecov](https://codecov.io/gh/apache/airflow/pull/4982?src=pr=footer). Last update [2d9b0fb...7ac493e](https://codecov.io/gh/apache/airflow/pull/4982?src=pr=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments). This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [airflow] codecov-io commented on issue #4982: [AIRFLOW-4160] Fix redirecting of 'Trigger Dag' Button in DAG Page (.vs v1-10-stable)
codecov-io commented on issue #4982: [AIRFLOW-4160] Fix redirecting of 'Trigger Dag' Button in DAG Page (.vs v1-10-stable) URL: https://github.com/apache/airflow/pull/4982#issuecomment-476960816 # [Codecov](https://codecov.io/gh/apache/airflow/pull/4982?src=pr=h1) Report > Merging [#4982](https://codecov.io/gh/apache/airflow/pull/4982?src=pr=desc) into [v1-10-stable](https://codecov.io/gh/apache/airflow/commit/2d9b0fb21faeb984be221a5cdd8cf68b41755981?src=pr=desc) will **increase** coverage by `0.19%`. > The diff coverage is `n/a`. [](https://codecov.io/gh/apache/airflow/pull/4982?src=pr=tree) ```diff @@ Coverage Diff@@ ## v1-10-stable#4982 +/- ## + Coverage 77.87% 78.07% +0.19% Files 221 221 Lines 1752818110 +582 + Hits 1365014139 +489 - Misses 3878 3971 +93 ``` | [Impacted Files](https://codecov.io/gh/apache/airflow/pull/4982?src=pr=tree) | Coverage Δ | | |---|---|---| | [airflow/operators/mysql\_operator.py](https://codecov.io/gh/apache/airflow/pull/4982/diff?src=pr=tree#diff-YWlyZmxvdy9vcGVyYXRvcnMvbXlzcWxfb3BlcmF0b3IucHk=) | `79.16% <0%> (-20.84%)` | :arrow_down: | | [airflow/utils/decorators.py](https://codecov.io/gh/apache/airflow/pull/4982/diff?src=pr=tree#diff-YWlyZmxvdy91dGlscy9kZWNvcmF0b3JzLnB5) | `84.48% <0%> (-7.19%)` | :arrow_down: | | [airflow/utils/dag\_processing.py](https://codecov.io/gh/apache/airflow/pull/4982/diff?src=pr=tree#diff-YWlyZmxvdy91dGlscy9kYWdfcHJvY2Vzc2luZy5weQ==) | `56.17% <0%> (-2.81%)` | :arrow_down: | | [airflow/operators/redshift\_to\_s3\_operator.py](https://codecov.io/gh/apache/airflow/pull/4982/diff?src=pr=tree#diff-YWlyZmxvdy9vcGVyYXRvcnMvcmVkc2hpZnRfdG9fczNfb3BlcmF0b3IucHk=) | `92.98% <0%> (-2.58%)` | :arrow_down: | | [airflow/configuration.py](https://codecov.io/gh/apache/airflow/pull/4982/diff?src=pr=tree#diff-YWlyZmxvdy9jb25maWd1cmF0aW9uLnB5) | `89.04% <0%> (-0.18%)` | :arrow_down: | | [airflow/operators/oracle\_operator.py](https://codecov.io/gh/apache/airflow/pull/4982/diff?src=pr=tree#diff-YWlyZmxvdy9vcGVyYXRvcnMvb3JhY2xlX29wZXJhdG9yLnB5) | `0% <0%> (ø)` | :arrow_up: | | [airflow/sensors/hdfs\_sensor.py](https://codecov.io/gh/apache/airflow/pull/4982/diff?src=pr=tree#diff-YWlyZmxvdy9zZW5zb3JzL2hkZnNfc2Vuc29yLnB5) | `100% <0%> (ø)` | :arrow_up: | | [airflow/operators/s3\_to\_redshift\_operator.py](https://codecov.io/gh/apache/airflow/pull/4982/diff?src=pr=tree#diff-YWlyZmxvdy9vcGVyYXRvcnMvczNfdG9fcmVkc2hpZnRfb3BlcmF0b3IucHk=) | `100% <0%> (ø)` | :arrow_up: | | [airflow/models/\_\_init\_\_.py](https://codecov.io/gh/apache/airflow/pull/4982/diff?src=pr=tree#diff-YWlyZmxvdy9tb2RlbHMvX19pbml0X18ucHk=) | `92.72% <0%> (+0.04%)` | :arrow_up: | | [airflow/hooks/hive\_hooks.py](https://codecov.io/gh/apache/airflow/pull/4982/diff?src=pr=tree#diff-YWlyZmxvdy9ob29rcy9oaXZlX2hvb2tzLnB5) | `75.45% <0%> (+0.12%)` | :arrow_up: | | ... and [9 more](https://codecov.io/gh/apache/airflow/pull/4982/diff?src=pr=tree-more) | | -- [Continue to review full report at Codecov](https://codecov.io/gh/apache/airflow/pull/4982?src=pr=continue). > **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta) > `Δ = absolute (impact)`, `ø = not affected`, `? = missing data` > Powered by [Codecov](https://codecov.io/gh/apache/airflow/pull/4982?src=pr=footer). Last update [2d9b0fb...7ac493e](https://codecov.io/gh/apache/airflow/pull/4982?src=pr=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments). This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [airflow] jmcarp commented on a change in pull request #4830: [AIRFLOW-3929] Use anchor tag for dag filter link.
jmcarp commented on a change in pull request #4830: [AIRFLOW-3929] Use anchor tag for dag filter link. URL: https://github.com/apache/airflow/pull/4830#discussion_r269384479 ## File path: airflow/www/templates/airflow/dag.html ## @@ -125,12 +125,12 @@ - - + + Review comment: Thanks, updated to drop `type="button"` from all anchors in this template. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Updated] (AIRFLOW-4163) IntervalCheckOperator support relative difference ratio and can ignore zero
[ https://issues.apache.org/jira/browse/AIRFLOW-4163?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chao-Han Tsai updated AIRFLOW-4163: --- Description: - IntervalCheckOperator takes max/min ratio of two values for a metric and returns true if it is less than threshold. Currently if one of the values is 0, it assigns the ratio as None. In python comparison None < Number is always true. We should add an option to fail the task if one of the value is 0. - Currently it only supports Max/Min. It would be useful to support calculating ratio with relative difference. was: IntervalCheckOperator takes max/min ratio of two values for a metric and returns true if it is less than threshold. The bug is if one of the values is 0, it assigns the ratio as None. In python comparison None < Number is always true. We should change that to fail the test Also we should support calculating ratio with relative difference. > IntervalCheckOperator support relative difference ratio and can ignore zero > > > Key: AIRFLOW-4163 > URL: https://issues.apache.org/jira/browse/AIRFLOW-4163 > Project: Apache Airflow > Issue Type: Bug >Reporter: Chao-Han Tsai >Assignee: Chao-Han Tsai >Priority: Major > > - IntervalCheckOperator takes max/min ratio of two values for a metric and > returns true if it is less than threshold. Currently if one of the values is > 0, it assigns the ratio as None. In python comparison None < Number is always > true. We should add an option to fail the task if one of the value is 0. > - Currently it only supports Max/Min. It would be useful to support > calculating ratio with relative difference. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[GitHub] [airflow] codecov-io edited a comment on issue #4926: [AIRFLOW-4104] Add type annotations to common classes.
codecov-io edited a comment on issue #4926: [AIRFLOW-4104] Add type annotations to common classes. URL: https://github.com/apache/airflow/pull/4926#issuecomment-473576798 # [Codecov](https://codecov.io/gh/apache/airflow/pull/4926?src=pr=h1) Report > Merging [#4926](https://codecov.io/gh/apache/airflow/pull/4926?src=pr=desc) into [master](https://codecov.io/gh/apache/airflow/commit/e27950a75ce287c094e550fba07d1c8de5dc4143?src=pr=desc) will **decrease** coverage by `<.01%`. > The diff coverage is `95.45%`. [](https://codecov.io/gh/apache/airflow/pull/4926?src=pr=tree) ```diff @@Coverage Diff @@ ## master#4926 +/- ## == - Coverage 75.75% 75.75% -0.01% == Files 458 458 Lines 2986129862 +1 == Hits2262222622 - Misses 7239 7240 +1 ``` | [Impacted Files](https://codecov.io/gh/apache/airflow/pull/4926?src=pr=tree) | Coverage Δ | | |---|---|---| | [airflow/models/connection.py](https://codecov.io/gh/apache/airflow/pull/4926/diff?src=pr=tree#diff-YWlyZmxvdy9tb2RlbHMvY29ubmVjdGlvbi5weQ==) | `65.53% <0%> (-0.38%)` | :arrow_down: | | [airflow/operators/check\_operator.py](https://codecov.io/gh/apache/airflow/pull/4926/diff?src=pr=tree#diff-YWlyZmxvdy9vcGVyYXRvcnMvY2hlY2tfb3BlcmF0b3IucHk=) | `60.48% <100%> (ø)` | :arrow_up: | | [airflow/hooks/base\_hook.py](https://codecov.io/gh/apache/airflow/pull/4926/diff?src=pr=tree#diff-YWlyZmxvdy9ob29rcy9iYXNlX2hvb2sucHk=) | `92.3% <100%> (+0.15%)` | :arrow_up: | | [airflow/operators/python\_operator.py](https://codecov.io/gh/apache/airflow/pull/4926/diff?src=pr=tree#diff-YWlyZmxvdy9vcGVyYXRvcnMvcHl0aG9uX29wZXJhdG9yLnB5) | `95.83% <100%> (+0.02%)` | :arrow_up: | | [airflow/models/\_\_init\_\_.py](https://codecov.io/gh/apache/airflow/pull/4926/diff?src=pr=tree#diff-YWlyZmxvdy9tb2RlbHMvX19pbml0X18ucHk=) | `93.01% <100%> (-0.01%)` | :arrow_down: | -- [Continue to review full report at Codecov](https://codecov.io/gh/apache/airflow/pull/4926?src=pr=continue). > **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta) > `Δ = absolute (impact)`, `ø = not affected`, `? = missing data` > Powered by [Codecov](https://codecov.io/gh/apache/airflow/pull/4926?src=pr=footer). Last update [e27950a...0e751f4](https://codecov.io/gh/apache/airflow/pull/4926?src=pr=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments). This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [airflow] codecov-io edited a comment on issue #4926: [AIRFLOW-4104] Add type annotations to common classes.
codecov-io edited a comment on issue #4926: [AIRFLOW-4104] Add type annotations to common classes. URL: https://github.com/apache/airflow/pull/4926#issuecomment-473576798 # [Codecov](https://codecov.io/gh/apache/airflow/pull/4926?src=pr=h1) Report > Merging [#4926](https://codecov.io/gh/apache/airflow/pull/4926?src=pr=desc) into [master](https://codecov.io/gh/apache/airflow/commit/e27950a75ce287c094e550fba07d1c8de5dc4143?src=pr=desc) will **decrease** coverage by `<.01%`. > The diff coverage is `95.45%`. [](https://codecov.io/gh/apache/airflow/pull/4926?src=pr=tree) ```diff @@Coverage Diff @@ ## master#4926 +/- ## == - Coverage 75.75% 75.75% -0.01% == Files 458 458 Lines 2986129862 +1 == Hits2262222622 - Misses 7239 7240 +1 ``` | [Impacted Files](https://codecov.io/gh/apache/airflow/pull/4926?src=pr=tree) | Coverage Δ | | |---|---|---| | [airflow/models/connection.py](https://codecov.io/gh/apache/airflow/pull/4926/diff?src=pr=tree#diff-YWlyZmxvdy9tb2RlbHMvY29ubmVjdGlvbi5weQ==) | `65.53% <0%> (-0.38%)` | :arrow_down: | | [airflow/operators/check\_operator.py](https://codecov.io/gh/apache/airflow/pull/4926/diff?src=pr=tree#diff-YWlyZmxvdy9vcGVyYXRvcnMvY2hlY2tfb3BlcmF0b3IucHk=) | `60.48% <100%> (ø)` | :arrow_up: | | [airflow/hooks/base\_hook.py](https://codecov.io/gh/apache/airflow/pull/4926/diff?src=pr=tree#diff-YWlyZmxvdy9ob29rcy9iYXNlX2hvb2sucHk=) | `92.3% <100%> (+0.15%)` | :arrow_up: | | [airflow/operators/python\_operator.py](https://codecov.io/gh/apache/airflow/pull/4926/diff?src=pr=tree#diff-YWlyZmxvdy9vcGVyYXRvcnMvcHl0aG9uX29wZXJhdG9yLnB5) | `95.83% <100%> (+0.02%)` | :arrow_up: | | [airflow/models/\_\_init\_\_.py](https://codecov.io/gh/apache/airflow/pull/4926/diff?src=pr=tree#diff-YWlyZmxvdy9tb2RlbHMvX19pbml0X18ucHk=) | `93.01% <100%> (-0.01%)` | :arrow_down: | -- [Continue to review full report at Codecov](https://codecov.io/gh/apache/airflow/pull/4926?src=pr=continue). > **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta) > `Δ = absolute (impact)`, `ø = not affected`, `? = missing data` > Powered by [Codecov](https://codecov.io/gh/apache/airflow/pull/4926?src=pr=footer). Last update [e27950a...0e751f4](https://codecov.io/gh/apache/airflow/pull/4926?src=pr=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments). This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Commented] (AIRFLOW-4163) IntervalCheckOperator support relative difference ratio and can ignore zero
[
https://issues.apache.org/jira/browse/AIRFLOW-4163?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16802344#comment-16802344
]
ASF GitHub Bot commented on AIRFLOW-4163:
-
milton0825 commented on pull request #4983: [AIRFLOW-4163]
IntervalCheckOperator supports relative diff and not i…
URL: https://github.com/apache/airflow/pull/4983
…gnore 0
Make sure you have checked _all_ steps below.
### Jira
- [ ] My PR addresses the following [Airflow
Jira](https://issues.apache.org/jira/browse/AIRFLOW/) issues and references
them in the PR title. For example, "\[AIRFLOW-XXX\] My Airflow PR"
- https://issues.apache.org/jira/browse/AIRFLOW-XXX
- In case you are fixing a typo in the documentation you can prepend your
commit with \[AIRFLOW-XXX\], code changes always need a Jira issue.
- In case you are proposing a fundamental code change, you need to create
an Airflow Improvement Proposal
([AIP](https://cwiki.apache.org/confluence/display/AIRFLOW/Airflow+Improvements+Proposals)).
- In case you are adding a dependency, check if the license complies with
the [ASF 3rd Party License
Policy](https://www.apache.org/legal/resolved.html#category-x).
### Description
- [ ] Here are some details about my PR, including screenshots of any UI
changes:
### Tests
- [ ] My PR adds the following unit tests __OR__ does not need testing for
this extremely good reason:
### Commits
- [ ] My commits all reference Jira issues in their subject lines, and I
have squashed multiple commits if they address the same issue. In addition, my
commits follow the guidelines from "[How to write a good git commit
message](http://chris.beams.io/posts/git-commit/)":
1. Subject is separated from body by a blank line
1. Subject is limited to 50 characters (not including Jira issue reference)
1. Subject does not end with a period
1. Subject uses the imperative mood ("add", not "adding")
1. Body wraps at 72 characters
1. Body explains "what" and "why", not "how"
### Documentation
- [ ] In case of new functionality, my PR adds documentation that describes
how to use it.
- When adding new operators/hooks/sensors, the autoclass documentation
generation needs to be added.
- All the public functions and the classes in the PR contain docstrings
that explain what it does
- If you implement backwards incompatible changes, please leave a note in
the [Updating.md](https://github.com/apache/airflow/blob/master/UPDATING.md) so
we can assign it to a appropriate release
### Code Quality
- [ ] Passes `flake8`
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
> IntervalCheckOperator support relative difference ratio and can ignore zero
>
>
> Key: AIRFLOW-4163
> URL: https://issues.apache.org/jira/browse/AIRFLOW-4163
> Project: Apache Airflow
> Issue Type: Bug
>Reporter: Chao-Han Tsai
>Assignee: Chao-Han Tsai
>Priority: Major
>
> IntervalCheckOperator takes max/min ratio of two values for a metric and
> returns true if it is less than threshold.
> The bug is if one of the values is 0, it assigns the ratio as None. In python
> comparison None < Number is always true. We should change that to fail the
> test
> Also we should support calculating ratio with relative difference.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[GitHub] [airflow] milton0825 opened a new pull request #4983: [AIRFLOW-4163] IntervalCheckOperator supports relative diff and not i…
milton0825 opened a new pull request #4983: [AIRFLOW-4163]
IntervalCheckOperator supports relative diff and not i…
URL: https://github.com/apache/airflow/pull/4983
…gnore 0
Make sure you have checked _all_ steps below.
### Jira
- [ ] My PR addresses the following [Airflow
Jira](https://issues.apache.org/jira/browse/AIRFLOW/) issues and references
them in the PR title. For example, "\[AIRFLOW-XXX\] My Airflow PR"
- https://issues.apache.org/jira/browse/AIRFLOW-XXX
- In case you are fixing a typo in the documentation you can prepend your
commit with \[AIRFLOW-XXX\], code changes always need a Jira issue.
- In case you are proposing a fundamental code change, you need to create
an Airflow Improvement Proposal
([AIP](https://cwiki.apache.org/confluence/display/AIRFLOW/Airflow+Improvements+Proposals)).
- In case you are adding a dependency, check if the license complies with
the [ASF 3rd Party License
Policy](https://www.apache.org/legal/resolved.html#category-x).
### Description
- [ ] Here are some details about my PR, including screenshots of any UI
changes:
### Tests
- [ ] My PR adds the following unit tests __OR__ does not need testing for
this extremely good reason:
### Commits
- [ ] My commits all reference Jira issues in their subject lines, and I
have squashed multiple commits if they address the same issue. In addition, my
commits follow the guidelines from "[How to write a good git commit
message](http://chris.beams.io/posts/git-commit/)":
1. Subject is separated from body by a blank line
1. Subject is limited to 50 characters (not including Jira issue reference)
1. Subject does not end with a period
1. Subject uses the imperative mood ("add", not "adding")
1. Body wraps at 72 characters
1. Body explains "what" and "why", not "how"
### Documentation
- [ ] In case of new functionality, my PR adds documentation that describes
how to use it.
- When adding new operators/hooks/sensors, the autoclass documentation
generation needs to be added.
- All the public functions and the classes in the PR contain docstrings
that explain what it does
- If you implement backwards incompatible changes, please leave a note in
the [Updating.md](https://github.com/apache/airflow/blob/master/UPDATING.md) so
we can assign it to a appropriate release
### Code Quality
- [ ] Passes `flake8`
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[jira] [Updated] (AIRFLOW-4163) IntervalCheckOperator support relative difference ratio and can ignore zero
[ https://issues.apache.org/jira/browse/AIRFLOW-4163?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chao-Han Tsai updated AIRFLOW-4163: --- Summary: IntervalCheckOperator support relative difference ratio and can ignore zero (was: IntervalCheckOperator support relative difference ratio) > IntervalCheckOperator support relative difference ratio and can ignore zero > > > Key: AIRFLOW-4163 > URL: https://issues.apache.org/jira/browse/AIRFLOW-4163 > Project: Apache Airflow > Issue Type: Bug >Reporter: Chao-Han Tsai >Assignee: Chao-Han Tsai >Priority: Major > > IntervalCheckOperator takes max/min ratio of two values for a metric and > returns true if it is less than threshold. > The bug is if one of the values is 0, it assigns the ratio as None. In python > comparison None < Number is always true. We should change that to fail the > test > Also we should support calculating ratio with relative difference. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (AIRFLOW-4163) IntervalCheckOperator support relative difference ratio
[ https://issues.apache.org/jira/browse/AIRFLOW-4163?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chao-Han Tsai updated AIRFLOW-4163: --- Description: IntervalCheckOperator takes max/min ratio of two values for a metric and returns true if it is less than threshold. The bug is if one of the values is 0, it assigns the ratio as None. In python comparison None < Number is always true. We should change that to fail the test Also we should support calculating ratio with relative difference. was: IntervalCheckOperator takes max/min ratio of two values for a metric and returns true if it is less than threshold. The bug is if one of the values is 0, it assigns the ratio as None. In python comparison None < Number is always true. We should change that to fail the test > IntervalCheckOperator support relative difference ratio > > > Key: AIRFLOW-4163 > URL: https://issues.apache.org/jira/browse/AIRFLOW-4163 > Project: Apache Airflow > Issue Type: Bug >Reporter: Chao-Han Tsai >Assignee: Chao-Han Tsai >Priority: Major > > IntervalCheckOperator takes max/min ratio of two values for a metric and > returns true if it is less than threshold. > The bug is if one of the values is 0, it assigns the ratio as None. In python > comparison None < Number is always true. We should change that to fail the > test > Also we should support calculating ratio with relative difference. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (AIRFLOW-4163) IntervalCheckOperator support relative difference ratio
[ https://issues.apache.org/jira/browse/AIRFLOW-4163?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chao-Han Tsai updated AIRFLOW-4163: --- Summary: IntervalCheckOperator support relative difference ratio (was: IntervalCheckOperator comparison doesn't work for null values) > IntervalCheckOperator support relative difference ratio > > > Key: AIRFLOW-4163 > URL: https://issues.apache.org/jira/browse/AIRFLOW-4163 > Project: Apache Airflow > Issue Type: Bug >Reporter: Chao-Han Tsai >Assignee: Chao-Han Tsai >Priority: Major > > IntervalCheckOperator takes max/min ratio of two values for a metric and > returns true if it is less than threshold. > The bug is if one of the values is 0, it assigns the ratio as None. In python > comparison None < Number is always true. We should change that to fail the > test -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[GitHub] [airflow] potiuk commented on a change in pull request #4938: [AIRFLOW-4117] Multi-staging Image - Travis CI tests [Step 3/3]
potiuk commented on a change in pull request #4938: [AIRFLOW-4117]
Multi-staging Image - Travis CI tests [Step 3/3]
URL: https://github.com/apache/airflow/pull/4938#discussion_r269370193
##
File path: scripts/ci/in_container/run_ci_tests.sh
##
@@ -0,0 +1,128 @@
+#!/usr/bin/env bash
+
+#
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+#http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+
+set -euo pipefail
+
+MY_DIR=$(cd "$(dirname "$0")"; pwd)
+
+if [[ ${AIRFLOW_CI_VERBOSE:="false"} == "true" ]]; then
+set -x
+fi
+
+if [[ -z "${HADOOP_HOME:=}" ]]; then
+echo "HADOOP_HOME not set - abort" >&2
+exit 1
+fi
+
+export HADOOP_HOME
+
+if [[ -z "${AIRFLOW__CORE__SQL_ALCHEMY_CONN:=}" ]]; then
+echo "AIRFLOW__CORE__SQL_ALCHEMY_CONN not set - abort" >&2
+exit 2
+fi
+
+echo "Using ${HADOOP_DISTRO:=} distribution of Hadoop from ${HADOOP_HOME}"
+
+export HADOOP_DISTRO
+
+pwd
+
+echo "Using CI airflow.cfg"
+
+cp -f ${MY_DIR}/airflow_ci.cfg ~/unittests.cfg
+
+AIRFLOW_ROOT="$(cd ${MY_DIR}; cd ../../..; pwd)"
+export AIRFLOW__CORE__DAGS_FOLDER="${AIRFLOW_ROOT}/tests/dags"
+
+# add test/contrib to PYTHONPATH
+export PYTHONPATH=${PYTHONPATH:-${AIRFLOW_ROOT}/tests/test_utils}
+
+echo Backend connection: ${AIRFLOW__CORE__SQL_ALCHEMY_CONN}
+
+# environment
+export AIRFLOW_HOME=${AIRFLOW_HOME:=${HOME}}
+
+echo Airflow home: ${AIRFLOW_HOME}
+
+export AIRFLOW__CORE__UNIT_TEST_MODE=True
+
+# add test/test_utils to PYTHONPATH TODO: Do we need that ??? Looks fishy.
+export PYTHONPATH=${PYTHONPATH}:${MY_DIR}/tests/test_utils
+
+# any argument received is overriding the default nose execution arguments:
+NOSE_ARGS=$@
+
+which airflow
+
+# Fix codecov build path
+# TODO: Check this - this should be made travis-independent
+if [[ ! -h /home/travis/build/apache/airflow ]]; then
+ sudo mkdir -p /home/travis/build/apache
+ sudo ln -s ${AIRFLOW_ROOT} /home/travis/build/apache/airflow
+fi
+
+if [[ -z "${KUBERNETES_VERSION:=}" ]]; then
+ echo "Initializing the DB"
+ yes | airflow initdb || true
+ yes | airflow resetdb || true
+fi
+
+if [[ -z "${NOSE_ARGS}" ]]; then
+NOSE_ARGS="--with-coverage \
+--cover-erase \
+--cover-html \
+--cover-package=airflow \
+--cover-html-dir=airflow/www/static/coverage \
+--with-ignore-docstrings \
+--rednose \
+--with-timer \
+-v \
+--logging-level=INFO"
+echo
+echo "Running ALL Tests"
+echo
+else
+echo
+echo "Running tests with ${NOSE_ARGS}"
+echo
+fi
+
+if [[ -z "${KUBERNETES_VERSION}" ]]; then
+if [[ -z "${KRB5_KTNAME:=}" ]]; then
+echo "KRB5_KTNAME not set - abort" >&2
+exit 3
+fi
+ # kdc init happens in setup_kdc.sh
+ kinit -kt "${KRB5_KTNAME}" airflow
+fi
+
+# For impersonation tests running on SQLite on Travis, make the database world
readable so other
+# users can update it
+AIRFLOW_DB="${HOME}/airflow.db"
+
+if [[ -f "${AIRFLOW_DB}" ]]; then
+ chmod a+rw "${AIRFLOW_DB}"
+ chmod g+rwx "${AIRFLOW_HOME}"
+fi
+
+echo "Starting the tests with the following nose arguments: ${NOSE_ARGS}"
+
+set +
Review comment:
Fixed
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [airflow] potiuk commented on a change in pull request #4938: [AIRFLOW-4117] Multi-staging Image - Travis CI tests [Step 3/3]
potiuk commented on a change in pull request #4938: [AIRFLOW-4117] Multi-staging Image - Travis CI tests [Step 3/3] URL: https://github.com/apache/airflow/pull/4938#discussion_r269370124 ## File path: setup.py ## @@ -294,6 +308,7 @@ def do_setup(): 'configparser>=3.5.0, <3.6.0', 'croniter>=0.3.17, <0.4', 'dill>=0.2.2, <0.3', +'dumb-init>=1.2.2', Review comment: Fixed. Just moved it to devel This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Assigned] (AIRFLOW-4160) Redirecting of "Trigger Dag" Button in DAG Page is not working
[ https://issues.apache.org/jira/browse/AIRFLOW-4160?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiaodong DENG reassigned AIRFLOW-4160: -- Assignee: Xiaodong DENG > Redirecting of "Trigger Dag" Button in DAG Page is not working > -- > > Key: AIRFLOW-4160 > URL: https://issues.apache.org/jira/browse/AIRFLOW-4160 > Project: Apache Airflow > Issue Type: Bug > Components: webapp >Reporter: Xiaodong DENG >Assignee: Xiaodong DENG >Priority: Minor > Fix For: 1.10.3 > > > "Trigger Dag" Button in DAG Page was added in > [https://github.com/apache/airflow/pull/4373] > but it was broken by [https://github.com/apache/airflow/pull/4643] > > This issue is found in the testing process of 1.10.3b1 -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[GitHub] [airflow] potiuk commented on a change in pull request #4938: [AIRFLOW-4117] Multi-staging Image - Travis CI tests [Step 3/3]
potiuk commented on a change in pull request #4938: [AIRFLOW-4117] Multi-staging Image - Travis CI tests [Step 3/3] URL: https://github.com/apache/airflow/pull/4938#discussion_r269368263 ## File path: setup.py ## @@ -153,6 +166,7 @@ def write_version(filename=os.path.join(*['airflow', ] # major update coming soon, clamp to 0.x cloudant = ['cloudant>=0.5.9,<2.0'] +codecov = ['codecov'] Review comment: Done. Moved to devel. Makes sense to keep it there. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [airflow] potiuk commented on a change in pull request #4938: [AIRFLOW-4117] Multi-staging Image - Travis CI tests [Step 3/3]
potiuk commented on a change in pull request #4938: [AIRFLOW-4117]
Multi-staging Image - Travis CI tests [Step 3/3]
URL: https://github.com/apache/airflow/pull/4938#discussion_r269368151
##
File path: scripts/ci/in_container/run_ci.sh
##
@@ -0,0 +1,127 @@
+#!/usr/bin/env bash
+
+#
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+#http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+
+set -euo pipefail
+
+MY_DIR=$(cd "$(dirname "$0")"; pwd)
+
+if [[ ${AIRFLOW_CI_VERBOSE:="false"} == "true" ]]; then
+set -x
+fi
+
+AIRFLOW_ROOT="${MY_DIR}/../.."
+
+PYTHON_VERSION=${PYTHON_VERSION:=3.6}
+ENV=${ENV:=docker}
+BACKEND=${BACKEND:=sqlite}
+KUBERNETES_VERSION=${KUBERNETES_VERSION:=}
+
+RUN_TESTS=${RUN_TESTS:="true"}
+
+ARGS=$@
+
+if [[ -z "${AIRFLOW_HOME:=}" ]]; then
+echo
+echo AIRFLOW_HOME not set
+echo
+exit 1
+fi
+
+# Fix file permissions
+if [[ -d $HOME/.minikube ]]; then
+sudo chown -R airflow.airflow $HOME/.kube $HOME/.minikube
+fi
+
+if [[ ${PYTHON_VERSION} == 3* ]]; then
+PIP=pip3
+else
+PIP=pip2
+fi
+
+if [[ "${ENV}" == "docker" ]]; then
+${MY_DIR}/setup_env.sh
+${MY_DIR}/setup_kdc.sh
+fi
+
+if [[ "${BACKEND}" == "mysql" ]]; then
+export AIRFLOW__CORE__SQL_ALCHEMY_CONN=mysql://root@mysql/airflow
+export AIRFLOW__CELERY__RESULT_BACKEND=db+mysql://root@mysql/airflow
+${MY_DIR}/setup_mysql.sh >/dev/null
+elif [[ "${BACKEND}" == "sqlite" ]]; then
+export AIRFLOW__CORE__SQL_ALCHEMY_CONN=sqlite:///${AIRFLOW_HOME}/airflow.db
+export AIRFLOW__CORE__EXECUTOR=SequentialExecutor
+elif [[ "${BACKEND}" == "postgres" ]]; then
+export
AIRFLOW__CORE__SQL_ALCHEMY_CONN=postgresql+psycopg2://postgres:airflow@postgres/airflow
+export
AIRFLOW__CELERY__RESULT_BACKEND=db+postgresql://postgres:airflow@postgres/airflow
+else
+echo
+echo ""
+echo "Wrong backend ${BACKEND}"
+echo ""
+exit 2
+fi
+
+
+if [[ -z ${KUBERNETES_VERSION} ]]; then
+if [[ "${RUN_TESTS}" == "true" ]]; then
+echo
+echo Running CI tests with ${ARGS}
+echo
+${MY_DIR}/run_ci_tests.sh ${ARGS}
+codecov -e py${PYTHON_VERSION}-backend_${BACKEND}-env_${ENV}
+else
+if [[ -z "${ARGS}" ]]; then
Review comment:
Done.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [airflow] potiuk commented on a change in pull request #4938: [AIRFLOW-4117] Multi-staging Image - Travis CI tests [Step 3/3]
potiuk commented on a change in pull request #4938: [AIRFLOW-4117] Multi-staging Image - Travis CI tests [Step 3/3] URL: https://github.com/apache/airflow/pull/4938#discussion_r269365991 ## File path: scripts/ci/docker-compose.yml ## @@ -18,61 +18,67 @@ version: "2.2" services: mysql: image: mysql:5.6 -restart: always +restart: "no" environment: - MYSQL_ALLOW_EMPTY_PASSWORD=true - MYSQL_ROOT_HOST=% volumes: - ./mysql/conf.d:/etc/mysql/conf.d - postgres: image: postgres:9.6 -restart: always +restart: "no" environment: - POSTGRES_USER=postgres - POSTGRES_PASSWORD=airflow - POSTGRES_DB=airflow - mongo: image: mongo:3 -restart: always +restart: "no" cassandra: image: cassandra:3.0 -restart: always +restart: "no" rabbitmq: image: rabbitmq:3.7 -restart: always +restart: "no" Review comment: OK. Cool. Thanks @gerardo. changing it. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [airflow] potiuk commented on a change in pull request #4937: [AIRFLOW-4116] Multi-staging includes CI image [Step 2/3]
potiuk commented on a change in pull request #4937: [AIRFLOW-4116]
Multi-staging includes CI image [Step 2/3]
URL: https://github.com/apache/airflow/pull/4937#discussion_r269365539
##
File path: Dockerfile
##
@@ -85,14 +85,134 @@ RUN adduser airflow \
&& echo "airflow ALL=(ALL) NOPASSWD: ALL" > /etc/sudoers.d/airflow \
&& chmod 0440 /etc/sudoers.d/airflow
+
+# This is an image with all APT dependencies needed by CI. It is built on top
of the airlfow APT image
+# Parameters:
+# airflow-apt-deps - this is the base image for CI deps image.
+
+FROM airflow-apt-deps as airflow-ci-apt-deps
+
+SHELL ["/bin/bash", "-o", "pipefail", "-e", "-u", "-x", "-c"]
+
+ENV JAVA_HOME=/usr/lib/jvm/java-8-openjdk-amd64/
+
+ARG APT_DEPS_IMAGE
+ENV APT_DEPS_IMAGE=${APT_DEPS_IMAGE}
+
+RUN echo "${APT_DEPS_IMAGE}"
+
+# Note the ifs below might be removed if Buildkit will become usable. It
should skip building this
+# image automatically if it is not used. For now we still go through all
layers below but they are empty
+# Note missing directories on debian-stretch
https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=863199
+RUN if [[ "${APT_DEPS_IMAGE}" == "airflow-ci-apt-deps" ]]; then \
+mkdir -pv /usr/share/man/man1 \
+&& mkdir -pv /usr/share/man/man7 \
+&& apt-get update \
+&& apt-get install --no-install-recommends -y \
+ lsb-release gnupg dirmngr openjdk-8-jdk \
+ vim tmux less unzip net-tools netcat \
+ ldap-utils postgresql-client sqlite3 \
+ krb5-user openssh-client openssh-server \
+ python-selinux \
Review comment:
OK. That's good then. It will be easier a bit then :). I will test it as i
am close to get simpler mysql install following the other problems.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [airflow] potiuk commented on a change in pull request #4936: [AIRFLOW-4115] Multi-staging Aiflow Docker image [Step 1/3]
potiuk commented on a change in pull request #4936: [AIRFLOW-4115] Multi-staging Aiflow Docker image [Step 1/3] URL: https://github.com/apache/airflow/pull/4936#discussion_r269364871 ## File path: .dockerignore ## @@ -0,0 +1,107 @@ +# +# Licensed to the Apache Software Foundation (ASF) under one +# or more contributor license agreements. See the NOTICE file +# distributed with this work for additional information +# regarding copyright ownership. The ASF licenses this file +# to you under the Apache License, Version 2.0 (the +# "License"); you may not use this file except in compliance +# with the License. You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, +# software distributed under the License is distributed on an +# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY +# KIND, either express or implied. See the License for the +# specific language governing permissions and limitations +# under the License. + +# NOTE! This docker ignore uses recommended technique +# Where everything is excluded by default and you deliberately +# Add only those directories/files you need. This is very useful +# To make sure that Docker context is always the same on any machine +# So that generated files are not accidentally added to the context +# This allows Docker's `COPY .` to behave in predictable way + +# Ignore everything +** + +# Allow only these directories +!/airflow +!/dags +!/dev +#!/docs - we do not need docs in the Docker! +!/hooks Review comment: Fixed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [airflow] potiuk commented on a change in pull request #4936: [AIRFLOW-4115] Multi-staging Aiflow Docker image [Step 1/3]
potiuk commented on a change in pull request #4936: [AIRFLOW-4115] Multi-staging Aiflow Docker image [Step 1/3] URL: https://github.com/apache/airflow/pull/4936#discussion_r269364815 ## File path: .dockerignore ## @@ -0,0 +1,107 @@ +# +# Licensed to the Apache Software Foundation (ASF) under one +# or more contributor license agreements. See the NOTICE file +# distributed with this work for additional information +# regarding copyright ownership. The ASF licenses this file +# to you under the Apache License, Version 2.0 (the +# "License"); you may not use this file except in compliance +# with the License. You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, +# software distributed under the License is distributed on an +# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY +# KIND, either express or implied. See the License for the +# specific language governing permissions and limitations +# under the License. + +# NOTE! This docker ignore uses recommended technique +# Where everything is excluded by default and you deliberately +# Add only those directories/files you need. This is very useful +# To make sure that Docker context is always the same on any machine +# So that generated files are not accidentally added to the context +# This allows Docker's `COPY .` to behave in predictable way + +# Ignore everything +** + +# Allow only these directories +!/airflow +!/dags +!/dev +#!/docs - we do not need docs in the Docker! +!/hooks +!/licenses +!/scripts +!/tests + +!.coveragerc +!.rat-excludes +!LICENSE +!MANIFEST.in +!NOTICE + +# !README.md - avoid triggering context change on README change (new companies using Airflow) + +!run_tests.sh +!setup.cfg +!setup.py +!airflow/version.py +!airflow/__init__.py +!airflow/bin/airflow +!airflow/www +!scripts/docker/entrypoint.sh + +# Now - ignore unnecessary files inside allowed directories +# This goes after the allowed directories + +# Git version is dynamically generated +/airflow/git_version + +# Exclude static www files generated bu NPM +/airflow/www/static/coverage +/airflow/www/static/dist +/airflow/www/node_modules + +# Exclude link to docs +/airflow/www/static/docs + + +# Generally exclude all NPM modules +**/node_modules/ Review comment: Fixed This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [airflow] potiuk commented on a change in pull request #4936: [AIRFLOW-4115] Multi-staging Aiflow Docker image [Step 1/3]
potiuk commented on a change in pull request #4936: [AIRFLOW-4115]
Multi-staging Aiflow Docker image [Step 1/3]
URL: https://github.com/apache/airflow/pull/4936#discussion_r269364590
##
File path: Dockerfile
##
@@ -13,46 +13,179 @@
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
+#
+# WARNING: THIS DOCKERFILE IS NOT INTENDED FOR PRODUCTION USE OR DEPLOYMENT.
+#
+# Arguments of the build
+ARG PYTHON_BASE_IMAGE="python:3.6-slim"
+ARG AIRFLOW_VERSION="2.0.0.dev0"
+# Which image is used as dependency for the main image
+ARG APT_DEPS_IMAGE="airflow-apt-deps"
+# Speeds up building the image - cassandra driver without CYTHON saves around
10 minutes
+ARG CASS_DRIVER_NO_CYTHON="1"
+# Build cassandra driver on multiple CPUs
+ARG CASS_DRIVER_BUILD_CONCURRENCY="8"
+# By default PIP install is run without cache to make image smaller
+ARG PIP_NO_CACHE_DIR="true"
+# Additional python deps to install
+ARG ADDITIONAL_PYTHON_DEPS=""
+# PIP version used to install dependencies
+ARG PIP_VERSION="19.0.1"
+# By increasing this number we can do force build of all dependencies
+ARG DEPENDENCIES_EPOCH_NUMBER="1"
+
+# This is base image with APT dependencies needed by Airflow. It is based on a
python slim image
+# Parameters:
+#PYTHON_BASE_IMAGE - base python image (python:x.y-slim)
+
+FROM ${PYTHON_BASE_IMAGE} as airflow-apt-deps
+
+SHELL ["/bin/bash", "-o", "pipefail", "-e", "-u", "-x", "-c"]
+
+ARG PYTHON_BASE_IMAGE
+ARG AIRFLOW_VERSION
+ENV PYTHON_BASE_IMAGE=${PYTHON_BASE_IMAGE}
+ENV AIRFLOW_VERSION=$AIRFLOW_VERSION
+
+# Print versions
+RUN echo "Base image: ${PYTHON_BASE_IMAGE}"
+RUN echo "Airflow version: ${AIRFLOW_VERSION}"
+
+# Make sure noninteractie debian install is used and language variab1les set
+ENV DEBIAN_FRONTEND=noninteractive LANGUAGE=C.UTF-8 LANG=C.UTF-8
LC_ALL=C.UTF-8 \
+LC_CTYPE=C.UTF-8 LC_MESSAGES=C.UTF-8
+
+# Increase the value below to force renstalling of all dependencies
+ENV DEPENDENCIES_EPOCH_NUMBER=${DEPENDENCIES_EPOCH_NUMBER}
+
+# Install curl and gnupg2 - needed to download nodejs in next step
+RUN apt-get update \
+&& apt-get install -y --no-install-recommends \
+ curl gnupg2 \
+&& apt-get autoremove -yqq --purge \
+&& apt-get clean \
+&& rm -rf /var/lib/apt/lists/*
+
+
+# Install basic apt dependencies
+RUN curl -sL https://deb.nodesource.com/setup_10.x | bash - \
+&& apt-get update \
+&& apt-get install -y --no-install-recommends \
+ # Packages to install \
+ libsasl2-dev freetds-bin build-essential sasl2-bin \
+ libsasl2-2 libsasl2-dev libsasl2-modules \
+ default-libmysqlclient-dev apt-utils curl rsync netcat locales \
+ freetds-dev libkrb5-dev libssl-dev libffi-dev libpq-dev git \
+ nodejs gosu sudo \
+&& apt-get autoremove -yqq --purge \
+&& apt-get clean \
+&& rm -rf /var/lib/apt/lists/*
+
+RUN adduser airflow \
+&& echo "airflow ALL=(ALL) NOPASSWD: ALL" > /etc/sudoers.d/airflow \
+&& chmod 0440 /etc/sudoers.d/airflow
+
+
+# This is the target image - it installs PIP and NPN dependencies including
efficient caching
+# mechanisms - it might be used to build the bare airflow build or CI build
+# Parameters:
+#APT_DEPS_IMAGE - image with APT dependencies. It might either be base
deps image with airflow
+# dependencies or CI deps image that contains also
CI-required dependencies
+
+FROM airflow-apt-deps as main
+
+SHELL ["/bin/bash", "-o", "pipefail", "-e", "-u", "-x", "-c"]
+
+WORKDIR /opt/airflow
+
+RUN echo "Airflow version: ${AIRFLOW_VERSION}"
+
+ARG AIRFLOW_HOME=/opt/airflow
+ENV AIRFLOW_HOME=${AIRFLOW_HOME}
-FROM python:3.6-slim
-SHELL ["/bin/bash", "-xc"]
+RUN mkdir -pv ${AIRFLOW_HOME} \
+&& chown -R airflow.airflow ${AIRFLOW_HOME}
-ENV AIRFLOW_HOME=/usr/local/airflow
-ARG AIRFLOW_DEPS="all"
-ARG PYTHON_DEPS=""
-ARG BUILD_DEPS="freetds-dev libkrb5-dev libssl-dev libffi-dev libpq-dev git"
-ARG APT_DEPS="libsasl2-dev freetds-bin build-essential
default-libmysqlclient-dev apt-utils curl rsync netcat locales"
+# Increase the value here to force reinstalling Apache Airflow pip dependencies
+ENV FORCE_REINSTALL_ALL_PIP_DEPENDENCIES=1
-ENV PATH="$HOME/.npm-packages/bin:$PATH"
+# Optimizing installation of Cassandra driver
+ARG CASS_DRIVER_BUILD_CONCURRENCY
+ARG CASS_DRIVER_NO_CYTHON
+ENV CASS_DRIVER_BUILD_CONCURRENCY=${CASS_DRIVER_BUILD_CONCURRENCY}
+ENV
[GitHub] [airflow] nritholtz commented on issue #4903: [AIRFLOW-4069] Add Opsgenie Alert Hook and Operator
nritholtz commented on issue #4903: [AIRFLOW-4069] Add Opsgenie Alert Hook and Operator URL: https://github.com/apache/airflow/pull/4903#issuecomment-476891936 @zhongjiajie Made recommended changes @ashb @mik-laj Removed code.rst changes since #4788 was merged into master. Latest build passed and looks ready to go. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [airflow] codecov-io edited a comment on issue #4903: [AIRFLOW-4069] Add Opsgenie Alert Hook and Operator
codecov-io edited a comment on issue #4903: [AIRFLOW-4069] Add Opsgenie Alert Hook and Operator URL: https://github.com/apache/airflow/pull/4903#issuecomment-471783078 # [Codecov](https://codecov.io/gh/apache/airflow/pull/4903?src=pr=h1) Report > Merging [#4903](https://codecov.io/gh/apache/airflow/pull/4903?src=pr=desc) into [master](https://codecov.io/gh/apache/airflow/commit/e27950a75ce287c094e550fba07d1c8de5dc4143?src=pr=desc) will **increase** coverage by `0.03%`. > The diff coverage is `95.65%`. [](https://codecov.io/gh/apache/airflow/pull/4903?src=pr=tree) ```diff @@Coverage Diff @@ ## master#4903 +/- ## == + Coverage 75.75% 75.78% +0.03% == Files 458 460 +2 Lines 2986129907 +46 == + Hits2262222666 +44 - Misses 7239 7241 +2 ``` | [Impacted Files](https://codecov.io/gh/apache/airflow/pull/4903?src=pr=tree) | Coverage Δ | | |---|---|---| | [airflow/contrib/hooks/opsgenie\_alert\_hook.py](https://codecov.io/gh/apache/airflow/pull/4903/diff?src=pr=tree#diff-YWlyZmxvdy9jb250cmliL2hvb2tzL29wc2dlbmllX2FsZXJ0X2hvb2sucHk=) | `100% <100%> (ø)` | | | [...rflow/contrib/operators/opsgenie\_alert\_operator.py](https://codecov.io/gh/apache/airflow/pull/4903/diff?src=pr=tree#diff-YWlyZmxvdy9jb250cmliL29wZXJhdG9ycy9vcHNnZW5pZV9hbGVydF9vcGVyYXRvci5weQ==) | `93.93% <93.93%> (ø)` | | -- [Continue to review full report at Codecov](https://codecov.io/gh/apache/airflow/pull/4903?src=pr=continue). > **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta) > `Δ = absolute (impact)`, `ø = not affected`, `? = missing data` > Powered by [Codecov](https://codecov.io/gh/apache/airflow/pull/4903?src=pr=footer). Last update [e27950a...a82e3cd](https://codecov.io/gh/apache/airflow/pull/4903?src=pr=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments). This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [airflow] codecov-io edited a comment on issue #4903: [AIRFLOW-4069] Add Opsgenie Alert Hook and Operator
codecov-io edited a comment on issue #4903: [AIRFLOW-4069] Add Opsgenie Alert Hook and Operator URL: https://github.com/apache/airflow/pull/4903#issuecomment-471783078 # [Codecov](https://codecov.io/gh/apache/airflow/pull/4903?src=pr=h1) Report > Merging [#4903](https://codecov.io/gh/apache/airflow/pull/4903?src=pr=desc) into [master](https://codecov.io/gh/apache/airflow/commit/e27950a75ce287c094e550fba07d1c8de5dc4143?src=pr=desc) will **increase** coverage by `0.03%`. > The diff coverage is `95.65%`. [](https://codecov.io/gh/apache/airflow/pull/4903?src=pr=tree) ```diff @@Coverage Diff @@ ## master#4903 +/- ## == + Coverage 75.75% 75.78% +0.03% == Files 458 460 +2 Lines 2986129907 +46 == + Hits2262222666 +44 - Misses 7239 7241 +2 ``` | [Impacted Files](https://codecov.io/gh/apache/airflow/pull/4903?src=pr=tree) | Coverage Δ | | |---|---|---| | [airflow/contrib/hooks/opsgenie\_alert\_hook.py](https://codecov.io/gh/apache/airflow/pull/4903/diff?src=pr=tree#diff-YWlyZmxvdy9jb250cmliL2hvb2tzL29wc2dlbmllX2FsZXJ0X2hvb2sucHk=) | `100% <100%> (ø)` | | | [...rflow/contrib/operators/opsgenie\_alert\_operator.py](https://codecov.io/gh/apache/airflow/pull/4903/diff?src=pr=tree#diff-YWlyZmxvdy9jb250cmliL29wZXJhdG9ycy9vcHNnZW5pZV9hbGVydF9vcGVyYXRvci5weQ==) | `93.93% <93.93%> (ø)` | | -- [Continue to review full report at Codecov](https://codecov.io/gh/apache/airflow/pull/4903?src=pr=continue). > **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta) > `Δ = absolute (impact)`, `ø = not affected`, `? = missing data` > Powered by [Codecov](https://codecov.io/gh/apache/airflow/pull/4903?src=pr=footer). Last update [e27950a...a82e3cd](https://codecov.io/gh/apache/airflow/pull/4903?src=pr=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments). This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Created] (AIRFLOW-4164) Add previous execution date in ISO format macro variable
Trevor Burke created AIRFLOW-4164:
-
Summary: Add previous execution date in ISO format macro variable
Key: AIRFLOW-4164
URL: https://issues.apache.org/jira/browse/AIRFLOW-4164
Project: Apache Airflow
Issue Type: Improvement
Components: models
Affects Versions: 1.10.2
Reporter: Trevor Burke
Assignee: Trevor Burke
It appears, to me, there isn't a macro default variable to access previous
execution date in ISO format, just simple '%Y-%m-%d' format. Adding in a macro
default variable of `prev_ds_iso` would require only four lines added, none
removed across three modules.
If it's already possible to access a more precise version of \{{ prev_ds }} I'm
all ears! Happy to take this quick PR on though.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Assigned] (AIRFLOW-4160) Redirecting of "Trigger Dag" Button in DAG Page is not working
[ https://issues.apache.org/jira/browse/AIRFLOW-4160?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chris Lambertus reassigned AIRFLOW-4160: Assignee: (was: Simon poortman) > Redirecting of "Trigger Dag" Button in DAG Page is not working > -- > > Key: AIRFLOW-4160 > URL: https://issues.apache.org/jira/browse/AIRFLOW-4160 > Project: Apache Airflow > Issue Type: Bug > Components: webapp >Reporter: Xiaodong DENG >Priority: Minor > Fix For: 1.10.3 > > > "Trigger Dag" Button in DAG Page was added in > [https://github.com/apache/airflow/pull/4373] > but it was broken by [https://github.com/apache/airflow/pull/4643] > > This issue is found in the testing process of 1.10.3b1 -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (AIRFLOW-4160) Redirecting of "Trigger Dag" Button in DAG Page is not working
[ https://issues.apache.org/jira/browse/AIRFLOW-4160?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16802145#comment-16802145 ] ASF subversion and git services commented on AIRFLOW-4160: -- Commit 363f41c7746b412cf01ec956dd156f6a5d638015 in airflow's branch refs/heads/fd-increase-pd-size from Xiaodong [ https://gitbox.apache.org/repos/asf?p=airflow.git;h=363f41c ] [AIRFLOW-4160] Fix redirecting of 'Trigger Dag' Button in DAG Page (#4981) > Redirecting of "Trigger Dag" Button in DAG Page is not working > -- > > Key: AIRFLOW-4160 > URL: https://issues.apache.org/jira/browse/AIRFLOW-4160 > Project: Apache Airflow > Issue Type: Bug > Components: webapp >Reporter: Xiaodong DENG >Assignee: Simon poortman >Priority: Minor > Fix For: 1.10.3 > > > "Trigger Dag" Button in DAG Page was added in > [https://github.com/apache/airflow/pull/4373] > but it was broken by [https://github.com/apache/airflow/pull/4643] > > This issue is found in the testing process of 1.10.3b1 -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[GitHub] [airflow] Fokko commented on issue #4896: [AIRFLOW-4061] Remove incubator in CI process
Fokko commented on issue #4896: [AIRFLOW-4061] Remove incubator in CI process URL: https://github.com/apache/airflow/pull/4896#issuecomment-476838246 So it died: ``` ERROR: for airflow-testing manifest for airflowci/airflow-ci:latest not found ERROR: manifest for airflowci/airflow-ci:latest not found The command "docker-compose -f scripts/ci/docker-compose.yml pull --quiet --parallel" failed and exited with 1 during . ``` Lets see if we can do a new release of the image. AIP-3 is pending until Sunday. Lets wait until 1.10.3 to make @ashb 's life easier ❤️ and then make a new release of the CI image. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [airflow] Fokko commented on a change in pull request #4749: [AIRFLOW-3934] Increase standard Dataproc PD size
Fokko commented on a change in pull request #4749: [AIRFLOW-3934] Increase standard Dataproc PD size URL: https://github.com/apache/airflow/pull/4749#discussion_r269301800 ## File path: airflow/contrib/operators/dataproc_operator.py ## @@ -160,10 +160,10 @@ def __init__(self, properties=None, master_machine_type='n1-standard-4', master_disk_type='pd-standard', - master_disk_size=500, + master_disk_size=1024, Review comment: @fenglu-g what are your thoughts? :-) This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Commented] (AIRFLOW-4072) enable GKEPodOperator xcom
[
https://issues.apache.org/jira/browse/AIRFLOW-4072?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16802144#comment-16802144
]
ASF subversion and git services commented on AIRFLOW-4072:
--
Commit f51b426ab18c33de16a6fc6a261e6d3b87b79510 in airflow's branch
refs/heads/fd-increase-pd-size from marengaz
[ https://gitbox.apache.org/repos/asf?p=airflow.git;h=f51b426 ]
[AIRFLOW-4072] enable GKEPodOperator xcom (#4905)
this provides consistent functionality with KubernetesPodOperator
> enable GKEPodOperator xcom
> --
>
> Key: AIRFLOW-4072
> URL: https://issues.apache.org/jira/browse/AIRFLOW-4072
> Project: Apache Airflow
> Issue Type: Bug
>Reporter: Ben Marengo
>Assignee: Ben Marengo
>Priority: Trivial
> Fix For: 2.0.0
>
>
> this provides consistent functionality with {{KubernetesPodOperator}}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (AIRFLOW-3811) Automatic generation of API Reference
[ https://issues.apache.org/jira/browse/AIRFLOW-3811?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16802146#comment-16802146 ] ASF subversion and git services commented on AIRFLOW-3811: -- Commit e27950a75ce287c094e550fba07d1c8de5dc4143 in airflow's branch refs/heads/fd-increase-pd-size from Kamil Breguła [ https://gitbox.apache.org/repos/asf?p=airflow.git;h=e27950a ] [AIRFLOW-3811] automatic generation of API Reference in docs (#4788) Co-authored-by: Jarek Potiuk > Automatic generation of API Reference > - > > Key: AIRFLOW-3811 > URL: https://issues.apache.org/jira/browse/AIRFLOW-3811 > Project: Apache Airflow > Issue Type: Sub-task > Components: docs >Reporter: Kamil Bregula >Assignee: Kamil Bregula >Priority: Minor > Fix For: 2.0.0 > > > Hello community, > We want to introduce the reference API generation based on the source code > that will replace the `code.rst` file. > Greetings -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (AIRFLOW-2227) Allow variable migrations by adding deletion
[ https://issues.apache.org/jira/browse/AIRFLOW-2227?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16802143#comment-16802143 ] ASF subversion and git services commented on AIRFLOW-2227: -- Commit b78193240ad4266a7a22dced3832f51a9dce1897 in airflow's branch refs/heads/fd-increase-pd-size from Jordan Sanders [ https://gitbox.apache.org/repos/asf?p=airflow.git;h=b781932 ] [AIRFLOW-2227] Add delete method to Variable class (#4963) > Allow variable migrations by adding deletion > > > Key: AIRFLOW-2227 > URL: https://issues.apache.org/jira/browse/AIRFLOW-2227 > Project: Apache Airflow > Issue Type: Improvement > Components: models >Reporter: nadav >Priority: Minor > Fix For: 2.0.0 > > > In order to allow using migration (via alembic or customized scripts) one > need the ability to delete variables (delete function). > Another utility function was added to allow retrieval of all the current > variables keys. > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (AIRFLOW-4154) Correct string formatting in jobs.py
[
https://issues.apache.org/jira/browse/AIRFLOW-4154?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16802142#comment-16802142
]
ASF subversion and git services commented on AIRFLOW-4154:
--
Commit dd97dc6fb155d71a9915f602b010dbd9d19de979 in airflow's branch
refs/heads/fd-increase-pd-size from Xiaodong
[ https://gitbox.apache.org/repos/asf?p=airflow.git;h=dd97dc6 ]
[AIRFLOW-4154] Correct string formatting in jobs.py (#4972)
> Correct string formatting in jobs.py
>
>
> Key: AIRFLOW-4154
> URL: https://issues.apache.org/jira/browse/AIRFLOW-4154
> Project: Apache Airflow
> Issue Type: Task
>Affects Versions: 1.10.2
>Reporter: Xiaodong DENG
>Assignee: Xiaodong DENG
>Priority: Minor
> Fix For: 2.0.0, 1.10.3
>
>
> There are improper string formatting, like
>
> * _*["{}".format(x) for x in tis_to_set_to_scheduled])*_
> * using _*.format*_ in log calls
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[GitHub] [airflow] Fokko commented on issue #4749: [AIRFLOW-3934] Increase standard Dataproc PD size
Fokko commented on issue #4749: [AIRFLOW-3934] Increase standard Dataproc PD size URL: https://github.com/apache/airflow/pull/4749#issuecomment-476836849 @kaxil My mistake. I've pushed it to Apache instead of my fork. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [airflow] mrunesson commented on issue #4648: [AIRFLOW-3274] Add run_as_user and fs_group options for Kubernetes
mrunesson commented on issue #4648: [AIRFLOW-3274] Add run_as_user and fs_group options for Kubernetes URL: https://github.com/apache/airflow/pull/4648#issuecomment-476791339 It would be good to consider the possibility to be able to set this per launched pod and also set the pod root filesystem to read only. See https://github.com/mrunesson/airflow/commit/ae2078d89e61fc420958dc337d57b993f2218eb3 for concept implementation. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Assigned] (AIRFLOW-4160) Redirecting of "Trigger Dag" Button in DAG Page is not working
[ https://issues.apache.org/jira/browse/AIRFLOW-4160?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Simon poortman reassigned AIRFLOW-4160: --- Assignee: Simon poortman (was: Xiaodong DENG) > Redirecting of "Trigger Dag" Button in DAG Page is not working > -- > > Key: AIRFLOW-4160 > URL: https://issues.apache.org/jira/browse/AIRFLOW-4160 > Project: Apache Airflow > Issue Type: Bug > Components: webapp >Reporter: Xiaodong DENG >Assignee: Simon poortman >Priority: Minor > Fix For: 1.10.3 > > > "Trigger Dag" Button in DAG Page was added in > [https://github.com/apache/airflow/pull/4373] > but it was broken by [https://github.com/apache/airflow/pull/4643] > > This issue is found in the testing process of 1.10.3b1 -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[GitHub] [airflow] galuszkak edited a comment on issue #4772: [AIRFLOW-3937] KubernetesPodOperator support for envFrom configMapRef…
galuszkak edited a comment on issue #4772: [AIRFLOW-3937] KubernetesPodOperator support for envFrom configMapRef… URL: https://github.com/apache/airflow/pull/4772#issuecomment-476785413 @ashb I agree, I just take literally what @dimberman wanted as he would like this as `private` helper, which I took very literally ;). Thanks for hard work, and a lot of those code review. It was a really good experience for me, although a little longer than expected ;). Again thank You! This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [airflow] galuszkak commented on issue #4772: [AIRFLOW-3937] KubernetesPodOperator support for envFrom configMapRef…
galuszkak commented on issue #4772: [AIRFLOW-3937] KubernetesPodOperator support for envFrom configMapRef… URL: https://github.com/apache/airflow/pull/4772#issuecomment-476785413 @ashb I agree, I just take literally what @dimberman that he would like this as `private` helper, which I took very literally ;). Thanks for hard work, and a lot of those code review. It was a really good experience for me, although a little longer than expected ;). Again thank You! This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Commented] (AIRFLOW-3811) Automatic generation of API Reference
[ https://issues.apache.org/jira/browse/AIRFLOW-3811?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16802009#comment-16802009 ] ASF GitHub Bot commented on AIRFLOW-3811: - ashb commented on pull request #4788: [AIRFLOW-3811][3/3] Add automatic generation of API Reference URL: https://github.com/apache/airflow/pull/4788 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org > Automatic generation of API Reference > - > > Key: AIRFLOW-3811 > URL: https://issues.apache.org/jira/browse/AIRFLOW-3811 > Project: Apache Airflow > Issue Type: Sub-task > Components: docs >Reporter: Kamil Bregula >Assignee: Kamil Bregula >Priority: Minor > > Hello community, > We want to introduce the reference API generation based on the source code > that will replace the `code.rst` file. > Greetings -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (AIRFLOW-3811) Automatic generation of API Reference
[ https://issues.apache.org/jira/browse/AIRFLOW-3811?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16802010#comment-16802010 ] ASF subversion and git services commented on AIRFLOW-3811: -- Commit e27950a75ce287c094e550fba07d1c8de5dc4143 in airflow's branch refs/heads/master from Kamil Breguła [ https://gitbox.apache.org/repos/asf?p=airflow.git;h=e27950a ] [AIRFLOW-3811] automatic generation of API Reference in docs (#4788) Co-authored-by: Jarek Potiuk > Automatic generation of API Reference > - > > Key: AIRFLOW-3811 > URL: https://issues.apache.org/jira/browse/AIRFLOW-3811 > Project: Apache Airflow > Issue Type: Sub-task > Components: docs >Reporter: Kamil Bregula >Assignee: Kamil Bregula >Priority: Minor > Fix For: 2.0.0 > > > Hello community, > We want to introduce the reference API generation based on the source code > that will replace the `code.rst` file. > Greetings -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Resolved] (AIRFLOW-3811) Automatic generation of API Reference
[ https://issues.apache.org/jira/browse/AIRFLOW-3811?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ash Berlin-Taylor resolved AIRFLOW-3811. Resolution: Fixed Fix Version/s: 2.0.0 > Automatic generation of API Reference > - > > Key: AIRFLOW-3811 > URL: https://issues.apache.org/jira/browse/AIRFLOW-3811 > Project: Apache Airflow > Issue Type: Sub-task > Components: docs >Reporter: Kamil Bregula >Assignee: Kamil Bregula >Priority: Minor > Fix For: 2.0.0 > > > Hello community, > We want to introduce the reference API generation based on the source code > that will replace the `code.rst` file. > Greetings -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[GitHub] [airflow] ashb merged pull request #4788: [AIRFLOW-3811][3/3] Add automatic generation of API Reference
ashb merged pull request #4788: [AIRFLOW-3811][3/3] Add automatic generation of API Reference URL: https://github.com/apache/airflow/pull/4788 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [airflow] ashb commented on issue #4772: [AIRFLOW-3937] KubernetesPodOperator support for envFrom configMapRef…
ashb commented on issue #4772: [AIRFLOW-3937] KubernetesPodOperator support for envFrom configMapRef… URL: https://github.com/apache/airflow/pull/4772#issuecomment-476776571 Oh I didn't know that `__foo` actually did something specific. Still: I don't think _anything_ should be private. If an advanced user want's to subclass and change behaviour, let the, This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [airflow] galuszkak commented on issue #4772: [AIRFLOW-3937] KubernetesPodOperator support for envFrom configMapRef…
galuszkak commented on issue #4772: [AIRFLOW-3937] KubernetesPodOperator support for envFrom configMapRef… URL: https://github.com/apache/airflow/pull/4772#issuecomment-476773177 For me `__` always meant private (because it cannot be overwritten), and `_` protected (because it can be overwritten). Mostly because of this: https://docs.python.org/3.7/tutorial/classes.html#private-variables But I get Your point, I think this is just part of preference, how to think about API design. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [airflow] ashb commented on issue #4772: [AIRFLOW-3937] KubernetesPodOperator support for envFrom configMapRef…
ashb commented on issue #4772: [AIRFLOW-3937] KubernetesPodOperator support for envFrom configMapRef… URL: https://github.com/apache/airflow/pull/4772#issuecomment-476770100 Thanks - if I was paying more attention I could have merged it (and squashed it) on Github. The reason Why I changed it from `__` to `_` is that `__` is generally reserved for Python internals. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [airflow] galuszkak commented on issue #4772: [AIRFLOW-3937] KubernetesPodOperator support for envFrom configMapRef…
galuszkak commented on issue #4772: [AIRFLOW-3937] KubernetesPodOperator support for envFrom configMapRef… URL: https://github.com/apache/airflow/pull/4772#issuecomment-476763348 @ashb I've rebase Your changes. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Commented] (AIRFLOW-3659) Create Google Cloud Transfer Service Operators
[ https://issues.apache.org/jira/browse/AIRFLOW-3659?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16801971#comment-16801971 ] ASF subversion and git services commented on AIRFLOW-3659: -- Commit 7860e03c9cda3a2c2abbe0a9446178d08ffc6c0d in airflow's branch refs/heads/v1-10-stable from Kamil Breguła [ https://gitbox.apache.org/repos/asf?p=airflow.git;h=7860e03 ] [AIRFLOW-3659] Create Google Cloud Transfer Service Operators (#4792) Co-authored-by: Antoni Smolinski > Create Google Cloud Transfer Service Operators > -- > > Key: AIRFLOW-3659 > URL: https://issues.apache.org/jira/browse/AIRFLOW-3659 > Project: Apache Airflow > Issue Type: New Feature >Reporter: Jarek Potiuk >Assignee: Kamil Bregula >Priority: Major > Fix For: 2.0.0, 1.10.3 > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (AIRFLOW-3908) Cloud Vision Operators
[ https://issues.apache.org/jira/browse/AIRFLOW-3908?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16801968#comment-16801968 ] ASF subversion and git services commented on AIRFLOW-3908: -- Commit 7d20912d68de07a8bdaeeb2ff529b2823b85993a in airflow's branch refs/heads/v1-10-stable from Kamil Breguła [ https://gitbox.apache.org/repos/asf?p=airflow.git;h=7d20912 ] [AIRFLOW-3908] Add more Google Cloud Vision operators (#4791) > Cloud Vision Operators > -- > > Key: AIRFLOW-3908 > URL: https://issues.apache.org/jira/browse/AIRFLOW-3908 > Project: Apache Airflow > Issue Type: Improvement >Reporter: Kamil Bregula >Assignee: Kamil Bregula >Priority: Major > Fix For: 2.0.0, 1.10.3 > > > Hello > I want to create operators for Cloud Vision: > |*Operator name*|*API URL /description*| > |CloudVisionAnnotateImage, > |[Docs|https://googleapis.github.io/google-cloud-python/latest/vision/gapic/v1/api.html#google.cloud.vision_v1.ImageAnnotatorClient.annotate_image]| > |CloudVisionAddProductToProductSetOperator, > |[Docs|https://googleapis.github.io/google-cloud-python/latest/vision/gapic/v1/api.html#google.cloud.vision_v1.ProductSearchClient.add_product_to_product_set]| > |CloudVisionRemoveProductFromProductSet, > |[Docs|https://googleapis.github.io/google-cloud-python/latest/vision/gapic/v1/api.html#google.cloud.vision_v1.ProductSearchClient.remove_product_from_product_set]| > |CloudVisionReferenceImageCreate|[Docs|https://googleapis.github.io/google-cloud-python/latest/vision/gapic/v1/api.html#google.cloud.vision_v1.ProductSearchClient.create_reference_image]| > Greets -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Resolved] (AIRFLOW-4160) Redirecting of "Trigger Dag" Button in DAG Page is not working
[ https://issues.apache.org/jira/browse/AIRFLOW-4160?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tao Feng resolved AIRFLOW-4160. --- Resolution: Fixed Fix Version/s: 1.10.3 > Redirecting of "Trigger Dag" Button in DAG Page is not working > -- > > Key: AIRFLOW-4160 > URL: https://issues.apache.org/jira/browse/AIRFLOW-4160 > Project: Apache Airflow > Issue Type: Bug > Components: webapp >Reporter: Xiaodong DENG >Assignee: Xiaodong DENG >Priority: Minor > Fix For: 1.10.3 > > > "Trigger Dag" Button in DAG Page was added in > [https://github.com/apache/airflow/pull/4373] > but it was broken by [https://github.com/apache/airflow/pull/4643] > > This issue is found in the testing process of 1.10.3b1 -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (AIRFLOW-3202) Update Airflow Docs with missing AWS hooks/operators
[ https://issues.apache.org/jira/browse/AIRFLOW-3202?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16801969#comment-16801969 ] ASF subversion and git services commented on AIRFLOW-3202: -- Commit ffa63b34c0c614a0ac1d7a61cfec780b5ba47ecd in airflow's branch refs/heads/v1-10-stable from Siddharth Gupta [ https://gitbox.apache.org/repos/asf?p=airflow.git;h=ffa63b3 ] [AIRFLOW-3202] add missing documentation for AWS hooks/operator (#4048) Add missing documentation for AWS hooks/operator > Update Airflow Docs with missing AWS hooks/operators > > > Key: AIRFLOW-3202 > URL: https://issues.apache.org/jira/browse/AIRFLOW-3202 > Project: Apache Airflow > Issue Type: Improvement >Reporter: Siddharth >Priority: Major > Fix For: 2.0.0 > > > Adding missing documentation for: > # AWS Kinesis > # AWS DynamoDB > # AWS Lambda -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (AIRFLOW-2915) Add example DAG for GoogleCloudStorageToBigQueryOperator
[ https://issues.apache.org/jira/browse/AIRFLOW-2915?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16801967#comment-16801967 ] ASF subversion and git services commented on AIRFLOW-2915: -- Commit d8cb43df6dde4d2c668934f070961ce6e0c2cd0e in airflow's branch refs/heads/v1-10-stable from Tim Swast [ https://gitbox.apache.org/repos/asf?p=airflow.git;h=d8cb43d ] [AIRFLOW-2915] Add example DAG for GoogleCloudStorageToBigQueryOperator (#3763) * Don't add DAG if GCP operators aren't installed. * Remove extra blank lines for flake8. > Add example DAG for GoogleCloudStorageToBigQueryOperator > > > Key: AIRFLOW-2915 > URL: https://issues.apache.org/jira/browse/AIRFLOW-2915 > Project: Apache Airflow > Issue Type: Improvement > Components: Documentation >Reporter: Tim Swast >Assignee: Tim Swast >Priority: Minor > Fix For: 2.0.0 > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (AIRFLOW-3810) Duplication of autoclass in `code.rst` and `integration.rst`
[ https://issues.apache.org/jira/browse/AIRFLOW-3810?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16801970#comment-16801970 ] ASF subversion and git services commented on AIRFLOW-3810: -- Commit a4000f30251278945025326a72c653ba3e8d53aa in airflow's branch refs/heads/v1-10-stable from Kamil Breguła [ https://gitbox.apache.org/repos/asf?p=airflow.git;h=a4000f3 ] [AIRFLOW-3810] Remove duplicate autoclass directive (#4656) > Duplication of autoclass in `code.rst` and `integration.rst` > > > Key: AIRFLOW-3810 > URL: https://issues.apache.org/jira/browse/AIRFLOW-3810 > Project: Apache Airflow > Issue Type: Sub-task > Components: docs >Reporter: Kamil Bregula >Assignee: Kamil Bregula >Priority: Minor > Fix For: 2.0.0 > > > Hello community, > We want to rewrite the `integration.rst` file so that it does not contain > duplicates from `code.rst ' (API Reference). > > Greetings -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (AIRFLOW-4160) Redirecting of "Trigger Dag" Button in DAG Page is not working
[ https://issues.apache.org/jira/browse/AIRFLOW-4160?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16801964#comment-16801964 ] ASF subversion and git services commented on AIRFLOW-4160: -- Commit 363f41c7746b412cf01ec956dd156f6a5d638015 in airflow's branch refs/heads/master from Xiaodong [ https://gitbox.apache.org/repos/asf?p=airflow.git;h=363f41c ] [AIRFLOW-4160] Fix redirecting of 'Trigger Dag' Button in DAG Page (#4981) > Redirecting of "Trigger Dag" Button in DAG Page is not working > -- > > Key: AIRFLOW-4160 > URL: https://issues.apache.org/jira/browse/AIRFLOW-4160 > Project: Apache Airflow > Issue Type: Bug > Components: webapp >Reporter: Xiaodong DENG >Assignee: Xiaodong DENG >Priority: Minor > > "Trigger Dag" Button in DAG Page was added in > [https://github.com/apache/airflow/pull/4373] > but it was broken by [https://github.com/apache/airflow/pull/4643] > > This issue is found in the testing process of 1.10.3b1 -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (AIRFLOW-4160) Redirecting of "Trigger Dag" Button in DAG Page is not working
[ https://issues.apache.org/jira/browse/AIRFLOW-4160?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16801963#comment-16801963 ] ASF GitHub Bot commented on AIRFLOW-4160: - feng-tao commented on pull request #4981: [AIRFLOW-4160] Fix redirecting of 'Trigger Dag' Button in DAG Page URL: https://github.com/apache/airflow/pull/4981 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org > Redirecting of "Trigger Dag" Button in DAG Page is not working > -- > > Key: AIRFLOW-4160 > URL: https://issues.apache.org/jira/browse/AIRFLOW-4160 > Project: Apache Airflow > Issue Type: Bug > Components: webapp >Reporter: Xiaodong DENG >Assignee: Xiaodong DENG >Priority: Minor > > "Trigger Dag" Button in DAG Page was added in > [https://github.com/apache/airflow/pull/4373] > but it was broken by [https://github.com/apache/airflow/pull/4643] > > This issue is found in the testing process of 1.10.3b1 -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[GitHub] [airflow] feng-tao merged pull request #4981: [AIRFLOW-4160] Fix redirecting of 'Trigger Dag' Button in DAG Page
feng-tao merged pull request #4981: [AIRFLOW-4160] Fix redirecting of 'Trigger Dag' Button in DAG Page URL: https://github.com/apache/airflow/pull/4981 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Updated] (AIRFLOW-4162) Support layout blocks in SlackAPIPostOperator & SlackWebhookOperator
[ https://issues.apache.org/jira/browse/AIRFLOW-4162?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bumsoo Kim updated AIRFLOW-4162: Labels: features (was: ) > Support layout blocks in SlackAPIPostOperator & SlackWebhookOperator > > > Key: AIRFLOW-4162 > URL: https://issues.apache.org/jira/browse/AIRFLOW-4162 > Project: Apache Airflow > Issue Type: Improvement > Components: operators >Reporter: Bumsoo Kim >Assignee: Bumsoo Kim >Priority: Minor > Labels: features > > Slack introduced a new API called "*layout blocks"* to create rich message > layouts. > [https://api.slack.com/messaging/composing/layouts] > > We can utilize this new feature by simply adding `blocks` array field to the > payload. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (AIRFLOW-4162) Support layout blocks in SlackAPIPostOperator & SlackWebhookOperator
[ https://issues.apache.org/jira/browse/AIRFLOW-4162?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bumsoo Kim updated AIRFLOW-4162: Affects Version/s: 1.10.2 > Support layout blocks in SlackAPIPostOperator & SlackWebhookOperator > > > Key: AIRFLOW-4162 > URL: https://issues.apache.org/jira/browse/AIRFLOW-4162 > Project: Apache Airflow > Issue Type: Improvement > Components: operators >Affects Versions: 1.10.2 >Reporter: Bumsoo Kim >Assignee: Bumsoo Kim >Priority: Minor > Labels: features > > Slack introduced a new API called "*layout blocks"* to create rich message > layouts. > [https://api.slack.com/messaging/composing/layouts] > > We can utilize this new feature by simply adding `blocks` array field to the > payload. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (AIRFLOW-4162) Support layout blocks in SlackAPIPostOperator & SlackWebhookOperator
Bumsoo Kim created AIRFLOW-4162: --- Summary: Support layout blocks in SlackAPIPostOperator & SlackWebhookOperator Key: AIRFLOW-4162 URL: https://issues.apache.org/jira/browse/AIRFLOW-4162 Project: Apache Airflow Issue Type: Improvement Components: operators Reporter: Bumsoo Kim Assignee: Bumsoo Kim Slack introduced a new API called "*layout blocks"* to create rich message layouts. [https://api.slack.com/messaging/composing/layouts] We can utilize this new feature by simply adding `blocks` array field to the payload. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[GitHub] [airflow] zhongjiajie commented on issue #4896: [AIRFLOW-4061] Remove incubator in CI process
zhongjiajie commented on issue #4896: [AIRFLOW-4061] Remove incubator in CI process URL: https://github.com/apache/airflow/pull/4896#issuecomment-476718428 @Fokko I remove "skip" and start test. but I could not find any release in https://hub.docker.com/r/airflowci/airflow-ci This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [airflow] zhongjiajie edited a comment on issue #4896: [AIRFLOW-4061] Remove incubator in CI process
zhongjiajie edited a comment on issue #4896: [AIRFLOW-4061] Remove incubator in CI process URL: https://github.com/apache/airflow/pull/4896#issuecomment-476718428 @Fokko I remove "skip" and start test. but I could not find any release in https://hub.docker.com/r/airflowci/airflow-ci and I don't think CI will pass This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [airflow] codecov-io commented on issue #4788: [AIRFLOW-3811][3/3] Add automatic generation of API Reference
codecov-io commented on issue #4788: [AIRFLOW-3811][3/3] Add automatic generation of API Reference URL: https://github.com/apache/airflow/pull/4788#issuecomment-476712851 # [Codecov](https://codecov.io/gh/apache/airflow/pull/4788?src=pr=h1) Report > Merging [#4788](https://codecov.io/gh/apache/airflow/pull/4788?src=pr=desc) into [master](https://codecov.io/gh/apache/airflow/commit/5c49ae3af1086fe1bf4df374a3b2bd6bdde37b87?src=pr=desc) will **increase** coverage by `0.07%`. > The diff coverage is `100%`. [](https://codecov.io/gh/apache/airflow/pull/4788?src=pr=tree) ```diff @@Coverage Diff @@ ## master#4788 +/- ## == + Coverage 75.67% 75.75% +0.07% == Files 458 458 Lines 2985631072+1216 == + Hits2259423539 +945 - Misses 7262 7533 +271 ``` | [Impacted Files](https://codecov.io/gh/apache/airflow/pull/4788?src=pr=tree) | Coverage Δ | | |---|---|---| | [airflow/operators/bash\_operator.py](https://codecov.io/gh/apache/airflow/pull/4788/diff?src=pr=tree#diff-YWlyZmxvdy9vcGVyYXRvcnMvYmFzaF9vcGVyYXRvci5weQ==) | `91.22% <ø> (ø)` | :arrow_up: | | [airflow/contrib/hooks/gcp\_transfer\_hook.py](https://codecov.io/gh/apache/airflow/pull/4788/diff?src=pr=tree#diff-YWlyZmxvdy9jb250cmliL2hvb2tzL2djcF90cmFuc2Zlcl9ob29rLnB5) | `95.55% <ø> (ø)` | :arrow_up: | | [airflow/operators/python\_operator.py](https://codecov.io/gh/apache/airflow/pull/4788/diff?src=pr=tree#diff-YWlyZmxvdy9vcGVyYXRvcnMvcHl0aG9uX29wZXJhdG9yLnB5) | `95.8% <ø> (ø)` | :arrow_up: | | [...rflow/contrib/operators/kubernetes\_pod\_operator.py](https://codecov.io/gh/apache/airflow/pull/4788/diff?src=pr=tree#diff-YWlyZmxvdy9jb250cmliL29wZXJhdG9ycy9rdWJlcm5ldGVzX3BvZF9vcGVyYXRvci5weQ==) | `98.55% <ø> (-0.07%)` | :arrow_down: | | [airflow/hooks/hdfs\_hook.py](https://codecov.io/gh/apache/airflow/pull/4788/diff?src=pr=tree#diff-YWlyZmxvdy9ob29rcy9oZGZzX2hvb2sucHk=) | `92.5% <100%> (ø)` | :arrow_up: | | [airflow/models/\_\_init\_\_.py](https://codecov.io/gh/apache/airflow/pull/4788/diff?src=pr=tree#diff-YWlyZmxvdy9tb2RlbHMvX19pbml0X18ucHk=) | `89.95% <0%> (-3%)` | :arrow_down: | | [airflow/contrib/operators/aws\_athena\_operator.py](https://codecov.io/gh/apache/airflow/pull/4788/diff?src=pr=tree#diff-YWlyZmxvdy9jb250cmliL29wZXJhdG9ycy9hd3NfYXRoZW5hX29wZXJhdG9yLnB5) | `67.69% <0%> (-2.77%)` | :arrow_down: | | [airflow/utils/db.py](https://codecov.io/gh/apache/airflow/pull/4788/diff?src=pr=tree#diff-YWlyZmxvdy91dGlscy9kYi5weQ==) | `88.28% <0%> (-2.02%)` | :arrow_down: | | [airflow/logging\_config.py](https://codecov.io/gh/apache/airflow/pull/4788/diff?src=pr=tree#diff-YWlyZmxvdy9sb2dnaW5nX2NvbmZpZy5weQ==) | `96.55% <0%> (-1.01%)` | :arrow_down: | | [airflow/configuration.py](https://codecov.io/gh/apache/airflow/pull/4788/diff?src=pr=tree#diff-YWlyZmxvdy9jb25maWd1cmF0aW9uLnB5) | `94.07% <0%> (-0.33%)` | :arrow_down: | | ... and [16 more](https://codecov.io/gh/apache/airflow/pull/4788/diff?src=pr=tree-more) | | -- [Continue to review full report at Codecov](https://codecov.io/gh/apache/airflow/pull/4788?src=pr=continue). > **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta) > `Δ = absolute (impact)`, `ø = not affected`, `? = missing data` > Powered by [Codecov](https://codecov.io/gh/apache/airflow/pull/4788?src=pr=footer). Last update [5c49ae3...eaecc3b](https://codecov.io/gh/apache/airflow/pull/4788?src=pr=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments). This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [airflow] ashb edited a comment on issue #4982: [AIRFLOW-4160] Fix redirecting of 'Trigger Dag' Button in DAG Page (.vs v1-10-stable)
ashb edited a comment on issue #4982: [AIRFLOW-4160] Fix redirecting of 'Trigger Dag' Button in DAG Page (.vs v1-10-stable) URL: https://github.com/apache/airflow/pull/4982#issuecomment-476699697 Do we have a similar one for master too? Ah yes 4981 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [airflow] ashb commented on issue #4982: [AIRFLOW-4160] Fix redirecting of 'Trigger Dag' Button in DAG Page (.vs v1-10-stable)
ashb commented on issue #4982: [AIRFLOW-4160] Fix redirecting of 'Trigger Dag' Button in DAG Page (.vs v1-10-stable) URL: https://github.com/apache/airflow/pull/4982#issuecomment-476699697 Do we have a similar one for master too? This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [airflow] ashb commented on issue #4926: [AIRFLOW-4104] Add type annotations to common classes.
ashb commented on issue #4926: [AIRFLOW-4104] Add type annotations to common classes. URL: https://github.com/apache/airflow/pull/4926#issuecomment-476698790 Curious - why isn't Travis running the tests? This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [airflow] ashb commented on a change in pull request #4926: [AIRFLOW-4104] Add type annotations to common classes.
ashb commented on a change in pull request #4926: [AIRFLOW-4104] Add type annotations to common classes. URL: https://github.com/apache/airflow/pull/4926#discussion_r269159765 ## File path: airflow/models/__init__.py ## @@ -4290,7 +4291,13 @@ def setdefault(cls, key, default, deserialize_json=False): @classmethod @provide_session -def get(cls, key, default_var=__NO_DEFAULT_SENTINEL, deserialize_json=False, session=None): +def get( +cls, +key, # type: str +default_var=__NO_DEFAULT_SENTINEL, # type: Any +deserialize_json=False, # type: bool +session=None Review comment: Yeah, computer enforce style guide ftw. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [airflow] ashb commented on a change in pull request #4926: [AIRFLOW-4104] Add type annotations to common classes.
ashb commented on a change in pull request #4926: [AIRFLOW-4104] Add type
annotations to common classes.
URL: https://github.com/apache/airflow/pull/4926#discussion_r269159444
##
File path: airflow/models/__init__.py
##
@@ -2065,44 +2061,45 @@ class derived from this one results in the creation of
a task object,
@apply_defaults
def __init__(
-self,
-task_id,
-owner=configuration.conf.get('operators', 'DEFAULT_OWNER'),
-email=None,
-email_on_retry=True,
-email_on_failure=True,
-retries=0,
-retry_delay=timedelta(seconds=300),
-retry_exponential_backoff=False,
-max_retry_delay=None,
-start_date=None,
-end_date=None,
-schedule_interval=None, # not hooked as of now
-depends_on_past=False,
-wait_for_downstream=False,
-dag=None,
-params=None,
-default_args=None,
-adhoc=False,
-priority_weight=1,
-weight_rule=WeightRule.DOWNSTREAM,
-queue=configuration.conf.get('celery', 'default_queue'),
-pool=None,
-sla=None,
-execution_timeout=None,
-on_failure_callback=None,
-on_success_callback=None,
-on_retry_callback=None,
-trigger_rule=TriggerRule.ALL_SUCCESS,
-resources=None,
-run_as_user=None,
-task_concurrency=None,
-executor_config=None,
-do_xcom_push=True,
-inlets=None,
-outlets=None,
-*args,
-**kwargs):
+self,
+task_id, # type: str
+owner=configuration.conf.get('operators', 'DEFAULT_OWNER'), # type:
str
+email=None, # type: Optional[str]
+email_on_retry=True, # type: bool
+email_on_failure=True, # type: bool
+retries=0, # type: int
+retry_delay=timedelta(seconds=300), # type: timedelta
+retry_exponential_backoff=False, # type: bool
+max_retry_delay=None, # type: Optional[datetime]
+start_date=None, # type: Optional[datetime]
+end_date=None, # type: Optional[datetime]
+schedule_interval=None, # not hooked as of now
Review comment:
So it was.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [airflow] jmcarp commented on issue #4926: [AIRFLOW-4104] Add type annotations to common classes.
jmcarp commented on issue #4926: [AIRFLOW-4104] Add type annotations to common classes. URL: https://github.com/apache/airflow/pull/4926#issuecomment-476696372 Conflicts resolved again This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [airflow] ashb edited a comment on issue #4827: [AIRFLOW-XXX] Add contribute guide, updating to docs
ashb edited a comment on issue #4827: [AIRFLOW-XXX] Add contribute guide, updating to docs URL: https://github.com/apache/airflow/pull/4827#issuecomment-476692985 Someone suggested https://docs.openstack.org/reno/latest/ might be worth looking at. Oh - that is entirely unrelated to the contributing doc though. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [airflow] ashb commented on issue #4827: [AIRFLOW-XXX] Add contribute guide, updating to docs
ashb commented on issue #4827: [AIRFLOW-XXX] Add contribute guide, updating to docs URL: https://github.com/apache/airflow/pull/4827#issuecomment-476692985 Someone suggested https://docs.openstack.org/reno/latest/ might be worth looking at This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [airflow] mik-laj commented on issue #4788: [AIRFLOW-3811][3/3] Add automatic generation of API Reference
mik-laj commented on issue #4788: [AIRFLOW-3811][3/3] Add automatic generation
of API Reference
URL: https://github.com/apache/airflow/pull/4788#issuecomment-476683063
@Fokko Travis have unreleased errors on one job. Can you run it again?
```
--
2) ERROR: test_integration_run_dag_with_scheduler_failure
(tests.contrib.minikube.test_kubernetes_executor.KubernetesExecutorTest)
==
- >> end captured logging << -
urllib3.connectionpool: DEBUG: Starting new HTTP connection (1):
10.20.5.63:30809
airflow.logging_config: DEBUG: Unable to load custom logging, using
default config instead
airflow.utils.log.logging_mixin.LoggingMixin: DEBUG: Cannot import due
to doesn't look like a module path
root: INFO: Generating grammar tables from
/usr/lib/python3.5/lib2to3/PatternGrammar.txt
root: INFO: Generating grammar tables from
/usr/lib/python3.5/lib2to3/Grammar.txt
airflow.settings: INFO: Configured default timezone
airflow.utils.log.logging_mixin.LoggingMixin: INFO: Creating new FAB
webserver config file in: /home/airflow/webserver_config.py
airflow.utils.log.logging_mixin.LoggingMixin: INFO: Reading the config
from /home/airflow/airflow.cfg
airflow.utils.log.logging_mixin.LoggingMixin: INFO: Creating new Airflow
config file in: /home/airflow/airflow.cfg
>> begin captured logging <<
- >> end captured stdout << --
[2019-03-26 13:38:20,174] {connectionpool.py:205} DEBUG - Starting new
HTTP connection (1): 10.20.5.63:30809
>> begin captured stdout << -
ConnectionError: HTTPConnectionPool(host='10.20.5.63', port=30809): Max
retries exceeded with url:
/api/experimental/dags/example_kubernetes_executor_config/paused/false
(Caused by NewConnectionError(': Failed to establish a new connection: [Errno 111] Connection
refused',))
raise ConnectionError(e, request=request)
.tox/py35-backend_postgres-env_kubernetes/lib/python3.5/site-packages/requests/adapters.py
line 516 in send
r = adapter.send(request, **kwargs)
.tox/py35-backend_postgres-env_kubernetes/lib/python3.5/site-packages/requests/sessions.py
line 646 in send
resp = self.send(prep, **send_kwargs)
.tox/py35-backend_postgres-env_kubernetes/lib/python3.5/site-packages/requests/sessions.py
line 533 in request
return session.request(method=method, url=url, **kwargs)
.tox/py35-backend_postgres-env_kubernetes/lib/python3.5/site-packages/requests/api.py
line 60 in request
return request('get', url, params=params, **kwargs)
.tox/py35-backend_postgres-env_kubernetes/lib/python3.5/site-packages/requests/api.py
line 75 in get
'dags/{dag_id}/paused/false'.format(host=host, dag_id=dag_id)
tests/contrib/minikube/test_kubernetes_executor.py line 134 in start_dag
result_json = self.start_dag(dag_id=dag_id, host=host)
tests/contrib/minikube/test_kubernetes_executor.py line 173 in
test_integration_run_dag
Traceback (most recent call last):
--
1) ERROR: test_integration_run_dag
(tests.contrib.minikube.test_kubernetes_executor.KubernetesExecutorTest)
==
```
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[jira] [Work started] (AIRFLOW-4158) Prefix and delimiter support for S3CopyObjectOperator & S3DeleteObjectsOperator
[ https://issues.apache.org/jira/browse/AIRFLOW-4158?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Work on AIRFLOW-4158 started by Mohammed Sherif KK. --- > Prefix and delimiter support for S3CopyObjectOperator & > S3DeleteObjectsOperator > --- > > Key: AIRFLOW-4158 > URL: https://issues.apache.org/jira/browse/AIRFLOW-4158 > Project: Apache Airflow > Issue Type: Improvement > Components: aws, operators >Reporter: Mohammed Sherif KK >Assignee: Mohammed Sherif KK >Priority: Major > Labels: feature > > In S3DeleteObjectsOperator and S3CopyObjectOperator, it only supports the > exact key of the file, there is no feature to copy or delete using prefix and > delimiter. > By this change, we can either use key or prefix and/or delimiter with > S3DeleteObjectsOperator and S3CopyObjectOperator, if we use prefix and/or > delimiter we can delete/copy all files with the same prefix. It is equivalent > to --recursive option of aws command. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (AIRFLOW-4161) Big query to mysql Operator
Antoine Dussarps created AIRFLOW-4161: - Summary: Big query to mysql Operator Key: AIRFLOW-4161 URL: https://issues.apache.org/jira/browse/AIRFLOW-4161 Project: Apache Airflow Issue Type: New Feature Components: contrib Reporter: Antoine Dussarps Assignee: Antoine Dussarps [Contrib] We developed an operator that safely copy a bigquery table onto MySql. It can be used for speed improvement on some pre-agregated requests. Does not work on views. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[GitHub] [airflow] GuzikJakub commented on a change in pull request #4715: [AIRFLOW-3894] unstructured connection to webhdfs
GuzikJakub commented on a change in pull request #4715: [AIRFLOW-3894]
unstructured connection to webhdfs
URL: https://github.com/apache/airflow/pull/4715#discussion_r269122466
##
File path: airflow/hooks/webhdfs_hook.py
##
@@ -55,14 +55,22 @@ def get_conn(self):
for nn in nn_connections:
try:
self.log.debug('Trying namenode %s', nn.host)
-connection_str = 'http://{nn.host}:{nn.port}'.format(nn=nn)
+connection_str = self.connection_url(
+'http://',
Review comment:
I changed the format
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [airflow] GuzikJakub commented on a change in pull request #4715: [AIRFLOW-3894] unstructured connection to webhdfs
GuzikJakub commented on a change in pull request #4715: [AIRFLOW-3894]
unstructured connection to webhdfs
URL: https://github.com/apache/airflow/pull/4715#discussion_r269122466
##
File path: airflow/hooks/webhdfs_hook.py
##
@@ -55,14 +55,22 @@ def get_conn(self):
for nn in nn_connections:
try:
self.log.debug('Trying namenode %s', nn.host)
-connection_str = 'http://{nn.host}:{nn.port}'.format(nn=nn)
+connection_str = self.connection_url(
+'http://',
Review comment:
I changed the format
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [airflow] GuzikJakub commented on a change in pull request #4715: [AIRFLOW-3894] unstructured connection to webhdfs
GuzikJakub commented on a change in pull request #4715: [AIRFLOW-3894]
unstructured connection to webhdfs
URL: https://github.com/apache/airflow/pull/4715#discussion_r269122044
##
File path: airflow/hooks/webhdfs_hook.py
##
@@ -55,14 +55,22 @@ def get_conn(self):
for nn in nn_connections:
try:
self.log.debug('Trying namenode %s', nn.host)
-connection_str = 'http://{nn.host}:{nn.port}'.format(nn=nn)
+connection_str = self.connection_url(
+'http://',
Review comment:
Finally, we get a connection like we should, for example:
url = 'http://host:port;http://host:port'
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [airflow] codecov-io edited a comment on issue #4887: [AIRFLOW-4055] Add AWS SQS Sensor
codecov-io edited a comment on issue #4887: [AIRFLOW-4055] Add AWS SQS Sensor URL: https://github.com/apache/airflow/pull/4887#issuecomment-471307289 # [Codecov](https://codecov.io/gh/apache/airflow/pull/4887?src=pr=h1) Report > Merging [#4887](https://codecov.io/gh/apache/airflow/pull/4887?src=pr=desc) into [master](https://codecov.io/gh/apache/airflow/commit/e220ac5bcfd56f048f552f47e7e1c5813f7b928f?src=pr=desc) will **increase** coverage by `0.89%`. > The diff coverage is `100%`. [](https://codecov.io/gh/apache/airflow/pull/4887?src=pr=tree) ```diff @@Coverage Diff@@ ## master #4887 +/- ## = + Coverage75.3% 76.2% +0.89% = Files 450 460 +10 Lines 29023 30961+1938 = + Hits21855 23593+1738 - Misses 71687368 +200 ``` | [Impacted Files](https://codecov.io/gh/apache/airflow/pull/4887?src=pr=tree) | Coverage Δ | | |---|---|---| | [airflow/contrib/sensors/aws\_sqs\_sensor.py](https://codecov.io/gh/apache/airflow/pull/4887/diff?src=pr=tree#diff-YWlyZmxvdy9jb250cmliL3NlbnNvcnMvYXdzX3Nxc19zZW5zb3IucHk=) | `100% <100%> (ø)` | | | [airflow/contrib/hooks/aws\_sqs\_hook.py](https://codecov.io/gh/apache/airflow/pull/4887/diff?src=pr=tree#diff-YWlyZmxvdy9jb250cmliL2hvb2tzL2F3c19zcXNfaG9vay5weQ==) | `100% <100%> (ø)` | | | [...w/contrib/operators/s3\_to\_gcs\_transfer\_operator.py](https://codecov.io/gh/apache/airflow/pull/4887/diff?src=pr=tree#diff-YWlyZmxvdy9jb250cmliL29wZXJhdG9ycy9zM190b19nY3NfdHJhbnNmZXJfb3BlcmF0b3IucHk=) | `0% <0%> (-100%)` | :arrow_down: | | [.../contrib/operators/gcs\_to\_gcs\_transfer\_operator.py](https://codecov.io/gh/apache/airflow/pull/4887/diff?src=pr=tree#diff-YWlyZmxvdy9jb250cmliL29wZXJhdG9ycy9nY3NfdG9fZ2NzX3RyYW5zZmVyX29wZXJhdG9yLnB5) | `0% <0%> (-100%)` | :arrow_down: | | [airflow/www/static\_config.py](https://codecov.io/gh/apache/airflow/pull/4887/diff?src=pr=tree#diff-YWlyZmxvdy93d3cvc3RhdGljX2NvbmZpZy5weQ==) | `83.33% <0%> (-6.67%)` | :arrow_down: | | [airflow/task/task\_runner/base\_task\_runner.py](https://codecov.io/gh/apache/airflow/pull/4887/diff?src=pr=tree#diff-YWlyZmxvdy90YXNrL3Rhc2tfcnVubmVyL2Jhc2VfdGFza19ydW5uZXIucHk=) | `76.66% <0%> (-2.65%)` | :arrow_down: | | [airflow/contrib/executors/kubernetes\_executor.py](https://codecov.io/gh/apache/airflow/pull/4887/diff?src=pr=tree#diff-YWlyZmxvdy9jb250cmliL2V4ZWN1dG9ycy9rdWJlcm5ldGVzX2V4ZWN1dG9yLnB5) | `62.96% <0%> (-2.27%)` | :arrow_down: | | [airflow/contrib/operators/aws\_athena\_operator.py](https://codecov.io/gh/apache/airflow/pull/4887/diff?src=pr=tree#diff-YWlyZmxvdy9jb250cmliL29wZXJhdG9ycy9hd3NfYXRoZW5hX29wZXJhdG9yLnB5) | `67.69% <0%> (-2.08%)` | :arrow_down: | | [airflow/logging\_config.py](https://codecov.io/gh/apache/airflow/pull/4887/diff?src=pr=tree#diff-YWlyZmxvdy9sb2dnaW5nX2NvbmZpZy5weQ==) | `96.55% <0%> (-1.01%)` | :arrow_down: | | [airflow/configuration.py](https://codecov.io/gh/apache/airflow/pull/4887/diff?src=pr=tree#diff-YWlyZmxvdy9jb25maWd1cmF0aW9uLnB5) | `93.42% <0%> (-0.99%)` | :arrow_down: | | ... and [72 more](https://codecov.io/gh/apache/airflow/pull/4887/diff?src=pr=tree-more) | | -- [Continue to review full report at Codecov](https://codecov.io/gh/apache/airflow/pull/4887?src=pr=continue). > **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta) > `Δ = absolute (impact)`, `ø = not affected`, `? = missing data` > Powered by [Codecov](https://codecov.io/gh/apache/airflow/pull/4887?src=pr=footer). Last update [e220ac5...7ee203f](https://codecov.io/gh/apache/airflow/pull/4887?src=pr=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments). This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [airflow] codecov-io commented on issue #4923: [AIRFLOW-4092] Add gRPCOperator, unit test and added to auto doc
codecov-io commented on issue #4923: [AIRFLOW-4092] Add gRPCOperator, unit test and added to auto doc URL: https://github.com/apache/airflow/pull/4923#issuecomment-476660336 # [Codecov](https://codecov.io/gh/apache/airflow/pull/4923?src=pr=h1) Report > Merging [#4923](https://codecov.io/gh/apache/airflow/pull/4923?src=pr=desc) into [master](https://codecov.io/gh/apache/airflow/commit/5d2c740e2d8e2ebaa7358e839fa03b981d358f88?src=pr=desc) will **decrease** coverage by `0.62%`. > The diff coverage is `100%`. [](https://codecov.io/gh/apache/airflow/pull/4923?src=pr=tree) ```diff @@Coverage Diff @@ ## master#4923 +/- ## == - Coverage 75.75% 75.12% -0.63% == Files 458 459 +1 Lines 2986129886 +25 == - Hits2262022453 -167 - Misses 7241 7433 +192 ``` | [Impacted Files](https://codecov.io/gh/apache/airflow/pull/4923?src=pr=tree) | Coverage Δ | | |---|---|---| | [airflow/contrib/operators/grpc\_operator.py](https://codecov.io/gh/apache/airflow/pull/4923/diff?src=pr=tree#diff-YWlyZmxvdy9jb250cmliL29wZXJhdG9ycy9ncnBjX29wZXJhdG9yLnB5) | `100% <100%> (ø)` | | | [airflow/operators/postgres\_operator.py](https://codecov.io/gh/apache/airflow/pull/4923/diff?src=pr=tree#diff-YWlyZmxvdy9vcGVyYXRvcnMvcG9zdGdyZXNfb3BlcmF0b3IucHk=) | `0% <0%> (-100%)` | :arrow_down: | | [airflow/operators/mysql\_operator.py](https://codecov.io/gh/apache/airflow/pull/4923/diff?src=pr=tree#diff-YWlyZmxvdy9vcGVyYXRvcnMvbXlzcWxfb3BlcmF0b3IucHk=) | `0% <0%> (-100%)` | :arrow_down: | | [airflow/operators/mysql\_to\_hive.py](https://codecov.io/gh/apache/airflow/pull/4923/diff?src=pr=tree#diff-YWlyZmxvdy9vcGVyYXRvcnMvbXlzcWxfdG9faGl2ZS5weQ==) | `0% <0%> (-100%)` | :arrow_down: | | [airflow/operators/generic\_transfer.py](https://codecov.io/gh/apache/airflow/pull/4923/diff?src=pr=tree#diff-YWlyZmxvdy9vcGVyYXRvcnMvZ2VuZXJpY190cmFuc2Zlci5weQ==) | `0% <0%> (-100%)` | :arrow_down: | | [airflow/executors/celery\_executor.py](https://codecov.io/gh/apache/airflow/pull/4923/diff?src=pr=tree#diff-YWlyZmxvdy9leGVjdXRvcnMvY2VsZXJ5X2V4ZWN1dG9yLnB5) | `38.46% <0%> (-38.47%)` | :arrow_down: | | [airflow/utils/sqlalchemy.py](https://codecov.io/gh/apache/airflow/pull/4923/diff?src=pr=tree#diff-YWlyZmxvdy91dGlscy9zcWxhbGNoZW15LnB5) | `76.13% <0%> (-5.69%)` | :arrow_down: | | [airflow/contrib/hooks/aws\_athena\_hook.py](https://codecov.io/gh/apache/airflow/pull/4923/diff?src=pr=tree#diff-YWlyZmxvdy9jb250cmliL2hvb2tzL2F3c19hdGhlbmFfaG9vay5weQ==) | `60% <0%> (-5.46%)` | :arrow_down: | | [airflow/hooks/dbapi\_hook.py](https://codecov.io/gh/apache/airflow/pull/4923/diff?src=pr=tree#diff-YWlyZmxvdy9ob29rcy9kYmFwaV9ob29rLnB5) | `84.8% <0%> (-3.21%)` | :arrow_down: | | [airflow/contrib/operators/aws\_athena\_operator.py](https://codecov.io/gh/apache/airflow/pull/4923/diff?src=pr=tree#diff-YWlyZmxvdy9jb250cmliL29wZXJhdG9ycy9hd3NfYXRoZW5hX29wZXJhdG9yLnB5) | `70.45% <0%> (-3.02%)` | :arrow_down: | | ... and [17 more](https://codecov.io/gh/apache/airflow/pull/4923/diff?src=pr=tree-more) | | -- [Continue to review full report at Codecov](https://codecov.io/gh/apache/airflow/pull/4923?src=pr=continue). > **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta) > `Δ = absolute (impact)`, `ø = not affected`, `? = missing data` > Powered by [Codecov](https://codecov.io/gh/apache/airflow/pull/4923?src=pr=footer). Last update [5d2c740...ffe2920](https://codecov.io/gh/apache/airflow/pull/4923?src=pr=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments). This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Commented] (AIRFLOW-3810) Duplication of autoclass in `code.rst` and `integration.rst`
[ https://issues.apache.org/jira/browse/AIRFLOW-3810?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16801726#comment-16801726 ] ASF subversion and git services commented on AIRFLOW-3810: -- Commit a4000f30251278945025326a72c653ba3e8d53aa in airflow's branch refs/heads/v1-10-test from Kamil Breguła [ https://gitbox.apache.org/repos/asf?p=airflow.git;h=a4000f3 ] [AIRFLOW-3810] Remove duplicate autoclass directive (#4656) > Duplication of autoclass in `code.rst` and `integration.rst` > > > Key: AIRFLOW-3810 > URL: https://issues.apache.org/jira/browse/AIRFLOW-3810 > Project: Apache Airflow > Issue Type: Sub-task > Components: docs >Reporter: Kamil Bregula >Assignee: Kamil Bregula >Priority: Minor > Fix For: 2.0.0 > > > Hello community, > We want to rewrite the `integration.rst` file so that it does not contain > duplicates from `code.rst ' (API Reference). > > Greetings -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[GitHub] [airflow] codecov-io edited a comment on issue #4772: [AIRFLOW-3937] KubernetesPodOperator support for envFrom configMapRef…
codecov-io edited a comment on issue #4772: [AIRFLOW-3937] KubernetesPodOperator support for envFrom configMapRef… URL: https://github.com/apache/airflow/pull/4772#issuecomment-471665712 # [Codecov](https://codecov.io/gh/apache/airflow/pull/4772?src=pr=h1) Report > Merging [#4772](https://codecov.io/gh/apache/airflow/pull/4772?src=pr=desc) into [master](https://codecov.io/gh/apache/airflow/commit/5eccaf642e3686f8562121d448b0800c0315c8fa?src=pr=desc) will **decrease** coverage by `0.69%`. > The diff coverage is `73.91%`. [](https://codecov.io/gh/apache/airflow/pull/4772?src=pr=tree) ```diff @@Coverage Diff@@ ## master#4772 +/- ## = - Coverage 75.75% 75.06% -0.7% = Files 458 458 Lines 2986130524+663 = + Hits2262222913+291 - Misses 7239 7611+372 ``` | [Impacted Files](https://codecov.io/gh/apache/airflow/pull/4772?src=pr=tree) | Coverage Δ | | |---|---|---| | [...rflow/contrib/operators/kubernetes\_pod\_operator.py](https://codecov.io/gh/apache/airflow/pull/4772/diff?src=pr=tree#diff-YWlyZmxvdy9jb250cmliL29wZXJhdG9ycy9rdWJlcm5ldGVzX3BvZF9vcGVyYXRvci5weQ==) | `98.64% <100%> (+0.03%)` | :arrow_up: | | [airflow/contrib/kubernetes/pod.py](https://codecov.io/gh/apache/airflow/pull/4772/diff?src=pr=tree#diff-YWlyZmxvdy9jb250cmliL2t1YmVybmV0ZXMvcG9kLnB5) | `100% <100%> (ø)` | :arrow_up: | | [airflow/contrib/kubernetes/secret.py](https://codecov.io/gh/apache/airflow/pull/4772/diff?src=pr=tree#diff-YWlyZmxvdy9jb250cmliL2t1YmVybmV0ZXMvc2VjcmV0LnB5) | `69.23% <62.5%> (+44.23%)` | :arrow_up: | | [...etes\_request\_factory/kubernetes\_request\_factory.py](https://codecov.io/gh/apache/airflow/pull/4772/diff?src=pr=tree#diff-YWlyZmxvdy9jb250cmliL2t1YmVybmV0ZXMva3ViZXJuZXRlc19yZXF1ZXN0X2ZhY3Rvcnkva3ViZXJuZXRlc19yZXF1ZXN0X2ZhY3RvcnkucHk=) | `72.11% <75%> (-0.52%)` | :arrow_down: | | [airflow/bin/cli.py](https://codecov.io/gh/apache/airflow/pull/4772/diff?src=pr=tree#diff-YWlyZmxvdy9iaW4vY2xpLnB5) | `54.96% <0%> (-12.05%)` | :arrow_down: | | [airflow/models/\_\_init\_\_.py](https://codecov.io/gh/apache/airflow/pull/4772/diff?src=pr=tree#diff-YWlyZmxvdy9tb2RlbHMvX19pbml0X18ucHk=) | `90% <0%> (-3%)` | :arrow_down: | | [airflow/contrib/operators/gcs\_to\_bq.py](https://codecov.io/gh/apache/airflow/pull/4772/diff?src=pr=tree#diff-YWlyZmxvdy9jb250cmliL29wZXJhdG9ycy9nY3NfdG9fYnEucHk=) | `0% <0%> (ø)` | :arrow_up: | | [airflow/jobs.py](https://codecov.io/gh/apache/airflow/pull/4772/diff?src=pr=tree#diff-YWlyZmxvdy9qb2JzLnB5) | `79.03% <0%> (+1.53%)` | :arrow_up: | -- [Continue to review full report at Codecov](https://codecov.io/gh/apache/airflow/pull/4772?src=pr=continue). > **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta) > `Δ = absolute (impact)`, `ø = not affected`, `? = missing data` > Powered by [Codecov](https://codecov.io/gh/apache/airflow/pull/4772?src=pr=footer). Last update [5eccaf6...3c26b94](https://codecov.io/gh/apache/airflow/pull/4772?src=pr=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments). This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Commented] (AIRFLOW-4160) Redirecting of "Trigger Dag" Button in DAG Page is not working
[ https://issues.apache.org/jira/browse/AIRFLOW-4160?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16801744#comment-16801744 ] ASF GitHub Bot commented on AIRFLOW-4160: - XD-DENG commented on pull request #4982: [AIRFLOW-4160] Fix redirecting of 'Trigger Dag' Button in DAG Page (.vs v1-10-stable) URL: https://github.com/apache/airflow/pull/4982 **This PR is for branch 1-10-stable only**. ### Jira - https://issues.apache.org/jira/browse/AIRFLOW-4160 ### Description "Trigger Dag" Button in DAG Page was added in https://github.com/apache/airflow/pull/4373 but it was broken by https://github.com/apache/airflow/pull/4643 This issue is found in the testing process of 1.10.3b1 . ### Tests Tested locally and the fix is working This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org > Redirecting of "Trigger Dag" Button in DAG Page is not working > -- > > Key: AIRFLOW-4160 > URL: https://issues.apache.org/jira/browse/AIRFLOW-4160 > Project: Apache Airflow > Issue Type: Bug > Components: webapp >Reporter: Xiaodong DENG >Assignee: Xiaodong DENG >Priority: Minor > > "Trigger Dag" Button in DAG Page was added in > [https://github.com/apache/airflow/pull/4373] > but it was broken by [https://github.com/apache/airflow/pull/4643] > > This issue is found in the testing process of 1.10.3b1 -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[GitHub] [airflow] XD-DENG commented on issue #4982: [AIRFLOW-4160] Fix redirecting of 'Trigger Dag' Button in DAG Page (.vs v1-10-stable)
XD-DENG commented on issue #4982: [AIRFLOW-4160] Fix redirecting of 'Trigger Dag' Button in DAG Page (.vs v1-10-stable) URL: https://github.com/apache/airflow/pull/4982#issuecomment-476654938 @ashb PTAL. This PR is for branch 1-10-stable. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [airflow] XD-DENG opened a new pull request #4982: [AIRFLOW-4160] Fix redirecting of 'Trigger Dag' Button in DAG Page (.vs v1-10-stable)
XD-DENG opened a new pull request #4982: [AIRFLOW-4160] Fix redirecting of 'Trigger Dag' Button in DAG Page (.vs v1-10-stable) URL: https://github.com/apache/airflow/pull/4982 **This PR is for branch 1-10-stable only**. ### Jira - https://issues.apache.org/jira/browse/AIRFLOW-4160 ### Description "Trigger Dag" Button in DAG Page was added in https://github.com/apache/airflow/pull/4373 but it was broken by https://github.com/apache/airflow/pull/4643 This issue is found in the testing process of 1.10.3b1 . ### Tests Tested locally and the fix is working This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Commented] (AIRFLOW-4160) Redirecting of "Trigger Dag" Button in DAG Page is not working
[ https://issues.apache.org/jira/browse/AIRFLOW-4160?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16801732#comment-16801732 ] ASF GitHub Bot commented on AIRFLOW-4160: - XD-DENG commented on pull request #4981: [AIRFLOW-4160] Fix redirecting of 'Trigger Dag' Button in DAG Page URL: https://github.com/apache/airflow/pull/4981 ### Jira - https://issues.apache.org/jira/browse/AIRFLOW-4160 ### Description "Trigger Dag" Button in DAG Page was added in https://github.com/apache/airflow/pull/4373 but it was broken by https://github.com/apache/airflow/pull/4643 This issue is found in the testing process of 1.10.3b1 . ### Tests Tested locally and the fix is working This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org > Redirecting of "Trigger Dag" Button in DAG Page is not working > -- > > Key: AIRFLOW-4160 > URL: https://issues.apache.org/jira/browse/AIRFLOW-4160 > Project: Apache Airflow > Issue Type: Bug > Components: webapp >Reporter: Xiaodong DENG >Assignee: Xiaodong DENG >Priority: Minor > > "Trigger Dag" Button in DAG Page was added in > [https://github.com/apache/airflow/pull/4373] > but it was broken by [https://github.com/apache/airflow/pull/4643] > > This issue is found in the testing process of 1.10.3b1 -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[GitHub] [airflow] XD-DENG commented on issue #4981: [AIRFLOW-4160] Fix redirecting of 'Trigger Dag' Button in DAG Page
XD-DENG commented on issue #4981: [AIRFLOW-4160] Fix redirecting of 'Trigger Dag' Button in DAG Page URL: https://github.com/apache/airflow/pull/4981#issuecomment-476647615 @ashb PTAL. Using the solution you proposed in our earlier discussion. Tested locally and it's working well. I will prepare another PR against branch 1-10-stable as well This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [airflow] XD-DENG opened a new pull request #4981: [AIRFLOW-4160] Fix redirecting of 'Trigger Dag' Button in DAG Page
XD-DENG opened a new pull request #4981: [AIRFLOW-4160] Fix redirecting of 'Trigger Dag' Button in DAG Page URL: https://github.com/apache/airflow/pull/4981 ### Jira - https://issues.apache.org/jira/browse/AIRFLOW-4160 ### Description "Trigger Dag" Button in DAG Page was added in https://github.com/apache/airflow/pull/4373 but it was broken by https://github.com/apache/airflow/pull/4643 This issue is found in the testing process of 1.10.3b1 . ### Tests Tested locally and the fix is working This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Commented] (AIRFLOW-3659) Create Google Cloud Transfer Service Operators
[ https://issues.apache.org/jira/browse/AIRFLOW-3659?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16801727#comment-16801727 ] ASF subversion and git services commented on AIRFLOW-3659: -- Commit 2cf72f5f44cc890acbe1a6ec5bdb89a925dd59d6 in airflow's branch refs/heads/v1-10-test from Kamil Breguła [ https://gitbox.apache.org/repos/asf?p=airflow.git;h=2cf72f5 ] [AIRFLOW-3659] Create Google Cloud Transfer Service Operators (#4792) Co-authored-by: Antoni Smolinski > Create Google Cloud Transfer Service Operators > -- > > Key: AIRFLOW-3659 > URL: https://issues.apache.org/jira/browse/AIRFLOW-3659 > Project: Apache Airflow > Issue Type: New Feature >Reporter: Jarek Potiuk >Assignee: Kamil Bregula >Priority: Major > Fix For: 2.0.0, 1.10.3 > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (AIRFLOW-3202) Update Airflow Docs with missing AWS hooks/operators
[ https://issues.apache.org/jira/browse/AIRFLOW-3202?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16801725#comment-16801725 ] ASF subversion and git services commented on AIRFLOW-3202: -- Commit ffa63b34c0c614a0ac1d7a61cfec780b5ba47ecd in airflow's branch refs/heads/v1-10-test from Siddharth Gupta [ https://gitbox.apache.org/repos/asf?p=airflow.git;h=ffa63b3 ] [AIRFLOW-3202] add missing documentation for AWS hooks/operator (#4048) Add missing documentation for AWS hooks/operator > Update Airflow Docs with missing AWS hooks/operators > > > Key: AIRFLOW-3202 > URL: https://issues.apache.org/jira/browse/AIRFLOW-3202 > Project: Apache Airflow > Issue Type: Improvement >Reporter: Siddharth >Priority: Major > Fix For: 2.0.0 > > > Adding missing documentation for: > # AWS Kinesis > # AWS DynamoDB > # AWS Lambda -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (AIRFLOW-2915) Add example DAG for GoogleCloudStorageToBigQueryOperator
[ https://issues.apache.org/jira/browse/AIRFLOW-2915?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16801723#comment-16801723 ] ASF subversion and git services commented on AIRFLOW-2915: -- Commit d8cb43df6dde4d2c668934f070961ce6e0c2cd0e in airflow's branch refs/heads/v1-10-test from Tim Swast [ https://gitbox.apache.org/repos/asf?p=airflow.git;h=d8cb43d ] [AIRFLOW-2915] Add example DAG for GoogleCloudStorageToBigQueryOperator (#3763) * Don't add DAG if GCP operators aren't installed. * Remove extra blank lines for flake8. > Add example DAG for GoogleCloudStorageToBigQueryOperator > > > Key: AIRFLOW-2915 > URL: https://issues.apache.org/jira/browse/AIRFLOW-2915 > Project: Apache Airflow > Issue Type: Improvement > Components: Documentation >Reporter: Tim Swast >Assignee: Tim Swast >Priority: Minor > Fix For: 2.0.0 > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (AIRFLOW-3908) Cloud Vision Operators
[ https://issues.apache.org/jira/browse/AIRFLOW-3908?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16801724#comment-16801724 ] ASF subversion and git services commented on AIRFLOW-3908: -- Commit 7d20912d68de07a8bdaeeb2ff529b2823b85993a in airflow's branch refs/heads/v1-10-test from Kamil Breguła [ https://gitbox.apache.org/repos/asf?p=airflow.git;h=7d20912 ] [AIRFLOW-3908] Add more Google Cloud Vision operators (#4791) > Cloud Vision Operators > -- > > Key: AIRFLOW-3908 > URL: https://issues.apache.org/jira/browse/AIRFLOW-3908 > Project: Apache Airflow > Issue Type: Improvement >Reporter: Kamil Bregula >Assignee: Kamil Bregula >Priority: Major > Fix For: 2.0.0, 1.10.3 > > > Hello > I want to create operators for Cloud Vision: > |*Operator name*|*API URL /description*| > |CloudVisionAnnotateImage, > |[Docs|https://googleapis.github.io/google-cloud-python/latest/vision/gapic/v1/api.html#google.cloud.vision_v1.ImageAnnotatorClient.annotate_image]| > |CloudVisionAddProductToProductSetOperator, > |[Docs|https://googleapis.github.io/google-cloud-python/latest/vision/gapic/v1/api.html#google.cloud.vision_v1.ProductSearchClient.add_product_to_product_set]| > |CloudVisionRemoveProductFromProductSet, > |[Docs|https://googleapis.github.io/google-cloud-python/latest/vision/gapic/v1/api.html#google.cloud.vision_v1.ProductSearchClient.remove_product_from_product_set]| > |CloudVisionReferenceImageCreate|[Docs|https://googleapis.github.io/google-cloud-python/latest/vision/gapic/v1/api.html#google.cloud.vision_v1.ProductSearchClient.create_reference_image]| > Greets -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Resolved] (AIRFLOW-4072) enable GKEPodOperator xcom
[
https://issues.apache.org/jira/browse/AIRFLOW-4072?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Fokko Driesprong resolved AIRFLOW-4072.
---
Resolution: Fixed
Fix Version/s: 2.0.0
> enable GKEPodOperator xcom
> --
>
> Key: AIRFLOW-4072
> URL: https://issues.apache.org/jira/browse/AIRFLOW-4072
> Project: Apache Airflow
> Issue Type: Bug
>Reporter: Ben Marengo
>Assignee: Ben Marengo
>Priority: Trivial
> Fix For: 2.0.0
>
>
> this provides consistent functionality with {{KubernetesPodOperator}}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[GitHub] [airflow] andriisoldatenko commented on a change in pull request #4459: [AIRFLOW-3649] Feature to add extra labels to kubernetes worker pods
andriisoldatenko commented on a change in pull request #4459: [AIRFLOW-3649] Feature to add extra labels to kubernetes worker pods URL: https://github.com/apache/airflow/pull/4459#discussion_r269106802 ## File path: airflow/contrib/executors/kubernetes_executor.py ## @@ -252,6 +265,15 @@ def _validate(self): 'or `dags_in_image` ' 'or `git_repo and git_branch and git_dags_folder_mount_point`') +# cannot allow certain labels, as airflow needs them +if self.kube_extra_labels: +for k in self.protected_labels: +if k in self.kube_extra_labels: +raise AirflowConfigException( +'In kubernetes mode `extra_labels` cannot include ' +'`airflow-worker`, `dag_id`, `task_id`, `try_number`, or `execution_date`' Review comment: I prefer to use `self.protected_labels` instead of hardcode it here ^ This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Commented] (AIRFLOW-4072) enable GKEPodOperator xcom
[
https://issues.apache.org/jira/browse/AIRFLOW-4072?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16801704#comment-16801704
]
ASF GitHub Bot commented on AIRFLOW-4072:
-
Fokko commented on pull request #4905: [AIRFLOW-4072] enable GKEPodOperator xcom
URL: https://github.com/apache/airflow/pull/4905
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
> enable GKEPodOperator xcom
> --
>
> Key: AIRFLOW-4072
> URL: https://issues.apache.org/jira/browse/AIRFLOW-4072
> Project: Apache Airflow
> Issue Type: Bug
>Reporter: Ben Marengo
>Assignee: Ben Marengo
>Priority: Trivial
>
> this provides consistent functionality with {{KubernetesPodOperator}}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (AIRFLOW-4072) enable GKEPodOperator xcom
[
https://issues.apache.org/jira/browse/AIRFLOW-4072?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16801705#comment-16801705
]
ASF subversion and git services commented on AIRFLOW-4072:
--
Commit f51b426ab18c33de16a6fc6a261e6d3b87b79510 in airflow's branch
refs/heads/master from marengaz
[ https://gitbox.apache.org/repos/asf?p=airflow.git;h=f51b426 ]
[AIRFLOW-4072] enable GKEPodOperator xcom (#4905)
this provides consistent functionality with KubernetesPodOperator
> enable GKEPodOperator xcom
> --
>
> Key: AIRFLOW-4072
> URL: https://issues.apache.org/jira/browse/AIRFLOW-4072
> Project: Apache Airflow
> Issue Type: Bug
>Reporter: Ben Marengo
>Assignee: Ben Marengo
>Priority: Trivial
>
> this provides consistent functionality with {{KubernetesPodOperator}}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[GitHub] [airflow] Fokko merged pull request #4905: [AIRFLOW-4072] enable GKEPodOperator xcom
Fokko merged pull request #4905: [AIRFLOW-4072] enable GKEPodOperator xcom URL: https://github.com/apache/airflow/pull/4905 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Created] (AIRFLOW-4160) Redirecting of "Trigger Dag" Button in DAG Page is not working
Xiaodong DENG created AIRFLOW-4160: -- Summary: Redirecting of "Trigger Dag" Button in DAG Page is not working Key: AIRFLOW-4160 URL: https://issues.apache.org/jira/browse/AIRFLOW-4160 Project: Apache Airflow Issue Type: Bug Components: webapp Reporter: Xiaodong DENG Assignee: Xiaodong DENG "Trigger Dag" Button in DAG Page was added in [https://github.com/apache/airflow/pull/4373] but it was broken by [https://github.com/apache/airflow/pull/4643] This issue is found in the testing process of 1.10.3b1 -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[GitHub] [airflow] kaxil commented on issue #4749: [AIRFLOW-3934] Increase standard Dataproc PD size
kaxil commented on issue #4749: [AIRFLOW-3934] Increase standard Dataproc PD size URL: https://github.com/apache/airflow/pull/4749#issuecomment-476636341 I think you forgot to push the commit @Fokko - I don't see the update This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Created] (AIRFLOW-4159) Add support for additional static labels for kubernetes executor
Andrii Soldatenko created AIRFLOW-4159: -- Summary: Add support for additional static labels for kubernetes executor Key: AIRFLOW-4159 URL: https://issues.apache.org/jira/browse/AIRFLOW-4159 Project: Apache Airflow Issue Type: Improvement Components: kubernetes Reporter: Andrii Soldatenko Assignee: Andrii Soldatenko -- This message was sent by Atlassian JIRA (v7.6.3#76005)
