[GitHub] [airflow] KKcorps commented on issue #6515: [AIRFLOW-XXX] GSoD: How to make DAGs production ready
KKcorps commented on issue #6515: [AIRFLOW-XXX] GSoD: How to make DAGs production ready URL: https://github.com/apache/airflow/pull/6515#issuecomment-551902906 I was also thinking to actually place this doc in the main section and name is **Best Practices** or something similar. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [airflow] codecov-io edited a comment on issue #5297: AIRFLOW-2143 - Fix TaskTries graph counts off-by-1
codecov-io edited a comment on issue #5297: AIRFLOW-2143 - Fix TaskTries graph counts off-by-1 URL: https://github.com/apache/airflow/pull/5297#issuecomment-493643996 # [Codecov](https://codecov.io/gh/apache/airflow/pull/5297?src=pr=h1) Report > Merging [#5297](https://codecov.io/gh/apache/airflow/pull/5297?src=pr=desc) into [master](https://codecov.io/gh/apache/airflow/commit/33ddcd9ef7b0c7c1e888e4aad93a641022571a9f?src=pr=desc) will **increase** coverage by `0.23%`. > The diff coverage is `80%`. [](https://codecov.io/gh/apache/airflow/pull/5297?src=pr=tree) ```diff @@Coverage Diff @@ ## master#5297 +/- ## == + Coverage83.8% 84.04% +0.23% == Files 635 635 Lines 3675036750 == + Hits3080030887 +87 + Misses 5950 5863 -87 ``` | [Impacted Files](https://codecov.io/gh/apache/airflow/pull/5297?src=pr=tree) | Coverage Δ | | |---|---|---| | [airflow/models/taskinstance.py](https://codecov.io/gh/apache/airflow/pull/5297/diff?src=pr=tree#diff-YWlyZmxvdy9tb2RlbHMvdGFza2luc3RhbmNlLnB5) | `93.81% <100%> (+0.02%)` | :arrow_up: | | [airflow/www/views.py](https://codecov.io/gh/apache/airflow/pull/5297/diff?src=pr=tree#diff-YWlyZmxvdy93d3cvdmlld3MucHk=) | `76.58% <50%> (+0.1%)` | :arrow_up: | | [airflow/hooks/dbapi\_hook.py](https://codecov.io/gh/apache/airflow/pull/5297/diff?src=pr=tree#diff-YWlyZmxvdy9ob29rcy9kYmFwaV9ob29rLnB5) | `91.52% <0%> (+0.84%)` | :arrow_up: | | [airflow/models/connection.py](https://codecov.io/gh/apache/airflow/pull/5297/diff?src=pr=tree#diff-YWlyZmxvdy9tb2RlbHMvY29ubmVjdGlvbi5weQ==) | `65% <0%> (+1.11%)` | :arrow_up: | | [airflow/hooks/hive\_hooks.py](https://codecov.io/gh/apache/airflow/pull/5297/diff?src=pr=tree#diff-YWlyZmxvdy9ob29rcy9oaXZlX2hvb2tzLnB5) | `77.6% <0%> (+1.78%)` | :arrow_up: | | [airflow/utils/sqlalchemy.py](https://codecov.io/gh/apache/airflow/pull/5297/diff?src=pr=tree#diff-YWlyZmxvdy91dGlscy9zcWxhbGNoZW15LnB5) | `93.22% <0%> (+6.77%)` | :arrow_up: | | [airflow/operators/mysql\_operator.py](https://codecov.io/gh/apache/airflow/pull/5297/diff?src=pr=tree#diff-YWlyZmxvdy9vcGVyYXRvcnMvbXlzcWxfb3BlcmF0b3IucHk=) | `100% <0%> (+100%)` | :arrow_up: | | [airflow/operators/mysql\_to\_hive.py](https://codecov.io/gh/apache/airflow/pull/5297/diff?src=pr=tree#diff-YWlyZmxvdy9vcGVyYXRvcnMvbXlzcWxfdG9faGl2ZS5weQ==) | `100% <0%> (+100%)` | :arrow_up: | -- [Continue to review full report at Codecov](https://codecov.io/gh/apache/airflow/pull/5297?src=pr=continue). > **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta) > `Δ = absolute (impact)`, `ø = not affected`, `? = missing data` > Powered by [Codecov](https://codecov.io/gh/apache/airflow/pull/5297?src=pr=footer). Last update [33ddcd9...182fa38](https://codecov.io/gh/apache/airflow/pull/5297?src=pr=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments). This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [airflow] codecov-io edited a comment on issue #5297: AIRFLOW-2143 - Fix TaskTries graph counts off-by-1
codecov-io edited a comment on issue #5297: AIRFLOW-2143 - Fix TaskTries graph counts off-by-1 URL: https://github.com/apache/airflow/pull/5297#issuecomment-493643996 # [Codecov](https://codecov.io/gh/apache/airflow/pull/5297?src=pr=h1) Report > Merging [#5297](https://codecov.io/gh/apache/airflow/pull/5297?src=pr=desc) into [master](https://codecov.io/gh/apache/airflow/commit/33ddcd9ef7b0c7c1e888e4aad93a641022571a9f?src=pr=desc) will **increase** coverage by `<.01%`. > The diff coverage is `80%`. [](https://codecov.io/gh/apache/airflow/pull/5297?src=pr=tree) ```diff @@Coverage Diff @@ ## master#5297 +/- ## == + Coverage83.8% 83.81% +<.01% == Files 635 635 Lines 3675036750 == + Hits3080030802 +2 + Misses 5950 5948 -2 ``` | [Impacted Files](https://codecov.io/gh/apache/airflow/pull/5297?src=pr=tree) | Coverage Δ | | |---|---|---| | [airflow/models/taskinstance.py](https://codecov.io/gh/apache/airflow/pull/5297/diff?src=pr=tree#diff-YWlyZmxvdy9tb2RlbHMvdGFza2luc3RhbmNlLnB5) | `93.81% <100%> (+0.02%)` | :arrow_up: | | [airflow/www/views.py](https://codecov.io/gh/apache/airflow/pull/5297/diff?src=pr=tree#diff-YWlyZmxvdy93d3cvdmlld3MucHk=) | `76.58% <50%> (+0.1%)` | :arrow_up: | | [airflow/hooks/dbapi\_hook.py](https://codecov.io/gh/apache/airflow/pull/5297/diff?src=pr=tree#diff-YWlyZmxvdy9ob29rcy9kYmFwaV9ob29rLnB5) | `90.67% <0%> (ø)` | :arrow_up: | | [airflow/models/connection.py](https://codecov.io/gh/apache/airflow/pull/5297/diff?src=pr=tree#diff-YWlyZmxvdy9tb2RlbHMvY29ubmVjdGlvbi5weQ==) | `63.88% <0%> (ø)` | :arrow_up: | | [airflow/hooks/hive\_hooks.py](https://codecov.io/gh/apache/airflow/pull/5297/diff?src=pr=tree#diff-YWlyZmxvdy9ob29rcy9oaXZlX2hvb2tzLnB5) | `75.82% <0%> (ø)` | :arrow_up: | | [airflow/utils/sqlalchemy.py](https://codecov.io/gh/apache/airflow/pull/5297/diff?src=pr=tree#diff-YWlyZmxvdy91dGlscy9zcWxhbGNoZW15LnB5) | `86.44% <0%> (ø)` | :arrow_up: | | [airflow/operators/mysql\_operator.py](https://codecov.io/gh/apache/airflow/pull/5297/diff?src=pr=tree#diff-YWlyZmxvdy9vcGVyYXRvcnMvbXlzcWxfb3BlcmF0b3IucHk=) | `0% <0%> (ø)` | :arrow_up: | | [airflow/operators/mysql\_to\_hive.py](https://codecov.io/gh/apache/airflow/pull/5297/diff?src=pr=tree#diff-YWlyZmxvdy9vcGVyYXRvcnMvbXlzcWxfdG9faGl2ZS5weQ==) | `0% <0%> (ø)` | :arrow_up: | -- [Continue to review full report at Codecov](https://codecov.io/gh/apache/airflow/pull/5297?src=pr=continue). > **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta) > `Δ = absolute (impact)`, `ø = not affected`, `? = missing data` > Powered by [Codecov](https://codecov.io/gh/apache/airflow/pull/5297?src=pr=footer). Last update [33ddcd9...182fa38](https://codecov.io/gh/apache/airflow/pull/5297?src=pr=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments). This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [airflow] codecov-io edited a comment on issue #5297: AIRFLOW-2143 - Fix TaskTries graph counts off-by-1
codecov-io edited a comment on issue #5297: AIRFLOW-2143 - Fix TaskTries graph counts off-by-1 URL: https://github.com/apache/airflow/pull/5297#issuecomment-493643996 # [Codecov](https://codecov.io/gh/apache/airflow/pull/5297?src=pr=h1) Report > Merging [#5297](https://codecov.io/gh/apache/airflow/pull/5297?src=pr=desc) into [master](https://codecov.io/gh/apache/airflow/commit/33ddcd9ef7b0c7c1e888e4aad93a641022571a9f?src=pr=desc) will **increase** coverage by `0.23%`. > The diff coverage is `80%`. [](https://codecov.io/gh/apache/airflow/pull/5297?src=pr=tree) ```diff @@Coverage Diff @@ ## master#5297 +/- ## == + Coverage83.8% 84.04% +0.23% == Files 635 635 Lines 3675036750 == + Hits3080030887 +87 + Misses 5950 5863 -87 ``` | [Impacted Files](https://codecov.io/gh/apache/airflow/pull/5297?src=pr=tree) | Coverage Δ | | |---|---|---| | [airflow/models/taskinstance.py](https://codecov.io/gh/apache/airflow/pull/5297/diff?src=pr=tree#diff-YWlyZmxvdy9tb2RlbHMvdGFza2luc3RhbmNlLnB5) | `93.81% <100%> (+0.02%)` | :arrow_up: | | [airflow/www/views.py](https://codecov.io/gh/apache/airflow/pull/5297/diff?src=pr=tree#diff-YWlyZmxvdy93d3cvdmlld3MucHk=) | `76.58% <50%> (+0.1%)` | :arrow_up: | | [airflow/hooks/dbapi\_hook.py](https://codecov.io/gh/apache/airflow/pull/5297/diff?src=pr=tree#diff-YWlyZmxvdy9ob29rcy9kYmFwaV9ob29rLnB5) | `91.52% <0%> (+0.84%)` | :arrow_up: | | [airflow/models/connection.py](https://codecov.io/gh/apache/airflow/pull/5297/diff?src=pr=tree#diff-YWlyZmxvdy9tb2RlbHMvY29ubmVjdGlvbi5weQ==) | `65% <0%> (+1.11%)` | :arrow_up: | | [airflow/hooks/hive\_hooks.py](https://codecov.io/gh/apache/airflow/pull/5297/diff?src=pr=tree#diff-YWlyZmxvdy9ob29rcy9oaXZlX2hvb2tzLnB5) | `77.6% <0%> (+1.78%)` | :arrow_up: | | [airflow/utils/sqlalchemy.py](https://codecov.io/gh/apache/airflow/pull/5297/diff?src=pr=tree#diff-YWlyZmxvdy91dGlscy9zcWxhbGNoZW15LnB5) | `93.22% <0%> (+6.77%)` | :arrow_up: | | [airflow/operators/mysql\_operator.py](https://codecov.io/gh/apache/airflow/pull/5297/diff?src=pr=tree#diff-YWlyZmxvdy9vcGVyYXRvcnMvbXlzcWxfb3BlcmF0b3IucHk=) | `100% <0%> (+100%)` | :arrow_up: | | [airflow/operators/mysql\_to\_hive.py](https://codecov.io/gh/apache/airflow/pull/5297/diff?src=pr=tree#diff-YWlyZmxvdy9vcGVyYXRvcnMvbXlzcWxfdG9faGl2ZS5weQ==) | `100% <0%> (+100%)` | :arrow_up: | -- [Continue to review full report at Codecov](https://codecov.io/gh/apache/airflow/pull/5297?src=pr=continue). > **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta) > `Δ = absolute (impact)`, `ø = not affected`, `? = missing data` > Powered by [Codecov](https://codecov.io/gh/apache/airflow/pull/5297?src=pr=footer). Last update [33ddcd9...182fa38](https://codecov.io/gh/apache/airflow/pull/5297?src=pr=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments). This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [airflow] codecov-io edited a comment on issue #5297: AIRFLOW-2143 - Fix TaskTries graph counts off-by-1
codecov-io edited a comment on issue #5297: AIRFLOW-2143 - Fix TaskTries graph counts off-by-1 URL: https://github.com/apache/airflow/pull/5297#issuecomment-493643996 # [Codecov](https://codecov.io/gh/apache/airflow/pull/5297?src=pr=h1) Report > Merging [#5297](https://codecov.io/gh/apache/airflow/pull/5297?src=pr=desc) into [master](https://codecov.io/gh/apache/airflow/commit/33ddcd9ef7b0c7c1e888e4aad93a641022571a9f?src=pr=desc) will **increase** coverage by `0.23%`. > The diff coverage is `80%`. [](https://codecov.io/gh/apache/airflow/pull/5297?src=pr=tree) ```diff @@Coverage Diff @@ ## master#5297 +/- ## == + Coverage83.8% 84.04% +0.23% == Files 635 635 Lines 3675036750 == + Hits3080030887 +87 + Misses 5950 5863 -87 ``` | [Impacted Files](https://codecov.io/gh/apache/airflow/pull/5297?src=pr=tree) | Coverage Δ | | |---|---|---| | [airflow/models/taskinstance.py](https://codecov.io/gh/apache/airflow/pull/5297/diff?src=pr=tree#diff-YWlyZmxvdy9tb2RlbHMvdGFza2luc3RhbmNlLnB5) | `93.81% <100%> (+0.02%)` | :arrow_up: | | [airflow/www/views.py](https://codecov.io/gh/apache/airflow/pull/5297/diff?src=pr=tree#diff-YWlyZmxvdy93d3cvdmlld3MucHk=) | `76.58% <50%> (+0.1%)` | :arrow_up: | | [airflow/hooks/dbapi\_hook.py](https://codecov.io/gh/apache/airflow/pull/5297/diff?src=pr=tree#diff-YWlyZmxvdy9ob29rcy9kYmFwaV9ob29rLnB5) | `91.52% <0%> (+0.84%)` | :arrow_up: | | [airflow/models/connection.py](https://codecov.io/gh/apache/airflow/pull/5297/diff?src=pr=tree#diff-YWlyZmxvdy9tb2RlbHMvY29ubmVjdGlvbi5weQ==) | `65% <0%> (+1.11%)` | :arrow_up: | | [airflow/hooks/hive\_hooks.py](https://codecov.io/gh/apache/airflow/pull/5297/diff?src=pr=tree#diff-YWlyZmxvdy9ob29rcy9oaXZlX2hvb2tzLnB5) | `77.6% <0%> (+1.78%)` | :arrow_up: | | [airflow/utils/sqlalchemy.py](https://codecov.io/gh/apache/airflow/pull/5297/diff?src=pr=tree#diff-YWlyZmxvdy91dGlscy9zcWxhbGNoZW15LnB5) | `93.22% <0%> (+6.77%)` | :arrow_up: | | [airflow/operators/mysql\_operator.py](https://codecov.io/gh/apache/airflow/pull/5297/diff?src=pr=tree#diff-YWlyZmxvdy9vcGVyYXRvcnMvbXlzcWxfb3BlcmF0b3IucHk=) | `100% <0%> (+100%)` | :arrow_up: | | [airflow/operators/mysql\_to\_hive.py](https://codecov.io/gh/apache/airflow/pull/5297/diff?src=pr=tree#diff-YWlyZmxvdy9vcGVyYXRvcnMvbXlzcWxfdG9faGl2ZS5weQ==) | `100% <0%> (+100%)` | :arrow_up: | -- [Continue to review full report at Codecov](https://codecov.io/gh/apache/airflow/pull/5297?src=pr=continue). > **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta) > `Δ = absolute (impact)`, `ø = not affected`, `? = missing data` > Powered by [Codecov](https://codecov.io/gh/apache/airflow/pull/5297?src=pr=footer). Last update [33ddcd9...182fa38](https://codecov.io/gh/apache/airflow/pull/5297?src=pr=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments). This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [airflow] ratb3rt commented on a change in pull request #6502: [AIRFLOW-5786] Migrate AWS SNS to /providers/amazon/aws
ratb3rt commented on a change in pull request #6502: [AIRFLOW-5786] Migrate AWS SNS to /providers/amazon/aws URL: https://github.com/apache/airflow/pull/6502#discussion_r344326320 ## File path: UPDATING.md ## @@ -53,8 +53,10 @@ Migrated are: | airflow.hooks.S3_hook.S3Hook | airflow.providers.aws.hooks.s3.S3Hook| Review comment: the s3hook isn't part of this PR/ticket, that was #6443 - i don't mind also moving all of the providers/aws/* into providers/amazon/aws/* as part of this PR, if wanted... This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [airflow] potiuk commented on a change in pull request #6502: [AIRFLOW-5786] Migrate AWS SNS to /providers/amazon/aws
potiuk commented on a change in pull request #6502: [AIRFLOW-5786] Migrate AWS SNS to /providers/amazon/aws URL: https://github.com/apache/airflow/pull/6502#discussion_r344271200 ## File path: UPDATING.md ## @@ -53,8 +53,10 @@ Migrated are: | airflow.hooks.S3_hook.S3Hook | airflow.providers.aws.hooks.s3.S3Hook| Review comment: Those need to be updated as well. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [airflow-site] mik-laj closed issue #44: Create a style for code/admonition/other inline-components
mik-laj closed issue #44: Create a style for code/admonition/other inline-components URL: https://github.com/apache/airflow-site/issues/44 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [airflow-site] mik-laj merged pull request #138: Add roadmap styles
mik-laj merged pull request #138: Add roadmap styles URL: https://github.com/apache/airflow-site/pull/138 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [airflow-site] mik-laj merged pull request #123: Add sphinx theme
mik-laj merged pull request #123: Add sphinx theme URL: https://github.com/apache/airflow-site/pull/123 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [airflow] potiuk commented on issue #6515: [AIRFLOW-XXX] GSoD: How to make DAGs production ready
potiuk commented on issue #6515: [AIRFLOW-XXX] GSoD: How to make DAGs production ready URL: https://github.com/apache/airflow/pull/6515#issuecomment-551906055 > I was also thinking to actually place this doc in the main section and name is **Best Practices** or something similar. I like it too :) This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[airflow-site] branch aip-11 updated: Add sphinx theme (#123)
This is an automated email from the ASF dual-hosted git repository.
kamilbregula pushed a commit to branch aip-11
in repository https://gitbox.apache.org/repos/asf/airflow-site.git
The following commit(s) were added to refs/heads/aip-11 by this push:
new 91f760c Add sphinx theme (#123)
91f760c is described below
commit 91f760cd62cb4b3aa5b40d455bed9ae694f7083d
Author: Kamil Breguła
AuthorDate: Fri Nov 8 18:17:31 2019 +0100
Add sphinx theme (#123)
---
.gitignore | 4 +
.pre-commit-config.yaml| 77 ++-
landing-pages/site/assets/scss/_roadmap.scss | 7 +-
landing-pages/src/docs-index.js| 26 +
landing-pages/src/js/tocTree.js| 27 +
landing-pages/webpack.common.js| 3 +-
license-templates/LICENSE.rst | 16 +
.../LICENSE.txt| 0
site.sh| 18 +-
sphinx_airflow_theme/README.md | 20 +
sphinx_airflow_theme/demo/awesome-cat.jpg | Bin 0 -> 5873700 bytes
sphinx_airflow_theme/demo/conf.py | 77 +++
sphinx_airflow_theme/demo/docs.sh | 80 +++
sphinx_airflow_theme/demo/index.rst| 36 ++
sphinx_airflow_theme/demo/playground.rst | 600 +
sphinx_airflow_theme/setup.py | 86 +++
.../sphinx_airflow_theme/__init__.py | 20 +-
.../sphinx_airflow_theme/breadcrumbs.html | 41 ++
.../sphinx_airflow_theme/footer.html | 148 +
.../sphinx_airflow_theme/globaltoc.html| 90
.../sphinx_airflow_theme/header.html | 171 ++
.../sphinx_airflow_theme/layout.html | 365 +
.../sphinx_airflow_theme/searchbox.html| 47 ++
.../sphinx_airflow_theme/searchresults.html| 58 ++
.../suggest_change_button.html | 74 +++
.../sphinx_airflow_theme/theme.conf| 8 +
.../sphinx_airflow_theme/version-selector.html | 33 ++
27 files changed, 2098 insertions(+), 34 deletions(-)
diff --git a/.gitignore b/.gitignore
index 4a0474a..67169bd 100644
--- a/.gitignore
+++ b/.gitignore
@@ -18,3 +18,7 @@
.idea/
.DS_Store
dist/
+*.egg-info/
+__pycache__/
+_build/
+sphinx_airflow_theme/sphinx_airflow_theme/static/_gen/
diff --git a/.pre-commit-config.yaml b/.pre-commit-config.yaml
index 1599c5b..91653d6 100644
--- a/.pre-commit-config.yaml
+++ b/.pre-commit-config.yaml
@@ -36,7 +36,7 @@ repos:
- --comment-style
- "|#|"
- --license-filepath
- - license-templates.txt
+ - license-templates/LICENSE.txt
- --fuzzy-match-generates-todo
files: ^\.gitmodules$|^\.gitignore$|^\.gitignore$|^Dockerfile.*$
- id: insert-license
@@ -46,7 +46,7 @@ repos:
- --comment-style
- "/**| *| */"
- --license-filepath
- - license-templates.txt
+ - license-templates/LICENSE.txt
- --fuzzy-match-generates-todo
- id: insert-license
name: Add licence for all shell files
@@ -56,7 +56,7 @@ repos:
- --comment-style
- "|#|"
- --license-filepath
- - license-templates.txt
+ - license-templates/LICENSE.txt
- --fuzzy-match-generates-todo
- id: insert-license
name: Add licence for all yaml files
@@ -65,7 +65,7 @@ repos:
- --comment-style
- "|#|"
- --license-filepath
- - license-templates.txt
+ - license-templates/LICENSE.txt
- --fuzzy-match-generates-todo
- id: insert-license
name: Add licence for all TOML files
@@ -74,7 +74,7 @@ repos:
- --comment-style
- "|#|"
- --license-filepath
- - license-templates.txt
+ - license-templates/LICENSE.txt
- --fuzzy-match-generates-todo
- id: insert-license
name: Add licence for all HTML files
@@ -83,9 +83,36 @@ repos:
- --comment-style
- "{{/*||*/}}"
- --license-filepath
- - license-templates.txt
+ - license-templates/LICENSE.txt
- --fuzzy-match-generates-todo
files: ^landing-pages/site/layouts/.+\.html$
+ - id: insert-license
+name: Add licence for all Python files
+types: [python]
+args:
+ - --comment-style
+ - "|#|"
+ - --license-filepath
+ - license-templates/LICENSE.txt
+ - --fuzzy-match-generates-todo
+ - id: insert-license
+name: Add license for all rst files
+args:
+ - --comment-style
+ - "||"
+ - --license-filepath
+ - license-templates/LICENSE.rst
+ - --fuzzy-match-generates-todo
+files: \.rst$
+ -
[GitHub] [airflow-site] mik-laj opened a new pull request #146: [DONT-MERGE] Add fake docs with thmee
mik-laj opened a new pull request #146: [DONT-MERGE] Add fake docs with thmee URL: https://github.com/apache/airflow-site/pull/146 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [airflow] codecov-io edited a comment on issue #6527: [AIRFLOW-5876][part of AIRFLOW-5697][depends on AIRFLOW-5716] Fetch all Dataflow jobs during searching
codecov-io edited a comment on issue #6527: [AIRFLOW-5876][part of AIRFLOW-5697][depends on AIRFLOW-5716] Fetch all Dataflow jobs during searching URL: https://github.com/apache/airflow/pull/6527#issuecomment-551934161 # [Codecov](https://codecov.io/gh/apache/airflow/pull/6527?src=pr=h1) Report > Merging [#6527](https://codecov.io/gh/apache/airflow/pull/6527?src=pr=desc) into [master](https://codecov.io/gh/apache/airflow/commit/33ddcd9ef7b0c7c1e888e4aad93a641022571a9f?src=pr=desc) will **decrease** coverage by `0.05%`. > The diff coverage is `85.71%`. [](https://codecov.io/gh/apache/airflow/pull/6527?src=pr=tree) ```diff @@Coverage Diff @@ ## master#6527 +/- ## == - Coverage83.8% 83.75% -0.06% == Files 635 635 Lines 3675036751 +1 == - Hits3080030779 -21 - Misses 5950 5972 +22 ``` | [Impacted Files](https://codecov.io/gh/apache/airflow/pull/6527?src=pr=tree) | Coverage Δ | | |---|---|---| | [airflow/gcp/hooks/dataflow.py](https://codecov.io/gh/apache/airflow/pull/6527/diff?src=pr=tree#diff-YWlyZmxvdy9nY3AvaG9va3MvZGF0YWZsb3cucHk=) | `91.46% <85.71%> (-0.79%)` | :arrow_down: | | [airflow/kubernetes/volume\_mount.py](https://codecov.io/gh/apache/airflow/pull/6527/diff?src=pr=tree#diff-YWlyZmxvdy9rdWJlcm5ldGVzL3ZvbHVtZV9tb3VudC5weQ==) | `44.44% <0%> (-55.56%)` | :arrow_down: | | [airflow/kubernetes/volume.py](https://codecov.io/gh/apache/airflow/pull/6527/diff?src=pr=tree#diff-YWlyZmxvdy9rdWJlcm5ldGVzL3ZvbHVtZS5weQ==) | `52.94% <0%> (-47.06%)` | :arrow_down: | | [airflow/kubernetes/pod\_launcher.py](https://codecov.io/gh/apache/airflow/pull/6527/diff?src=pr=tree#diff-YWlyZmxvdy9rdWJlcm5ldGVzL3BvZF9sYXVuY2hlci5weQ==) | `45.25% <0%> (-46.72%)` | :arrow_down: | | [airflow/kubernetes/kube\_client.py](https://codecov.io/gh/apache/airflow/pull/6527/diff?src=pr=tree#diff-YWlyZmxvdy9rdWJlcm5ldGVzL2t1YmVfY2xpZW50LnB5) | `33.33% <0%> (-41.67%)` | :arrow_down: | | [...rflow/contrib/operators/kubernetes\_pod\_operator.py](https://codecov.io/gh/apache/airflow/pull/6527/diff?src=pr=tree#diff-YWlyZmxvdy9jb250cmliL29wZXJhdG9ycy9rdWJlcm5ldGVzX3BvZF9vcGVyYXRvci5weQ==) | `70.14% <0%> (-28.36%)` | :arrow_down: | | [airflow/utils/dag\_processing.py](https://codecov.io/gh/apache/airflow/pull/6527/diff?src=pr=tree#diff-YWlyZmxvdy91dGlscy9kYWdfcHJvY2Vzc2luZy5weQ==) | `58.48% <0%> (+0.32%)` | :arrow_up: | | [airflow/hooks/dbapi\_hook.py](https://codecov.io/gh/apache/airflow/pull/6527/diff?src=pr=tree#diff-YWlyZmxvdy9ob29rcy9kYmFwaV9ob29rLnB5) | `91.52% <0%> (+0.84%)` | :arrow_up: | | [airflow/models/connection.py](https://codecov.io/gh/apache/airflow/pull/6527/diff?src=pr=tree#diff-YWlyZmxvdy9tb2RlbHMvY29ubmVjdGlvbi5weQ==) | `65% <0%> (+1.11%)` | :arrow_up: | | [airflow/hooks/hive\_hooks.py](https://codecov.io/gh/apache/airflow/pull/6527/diff?src=pr=tree#diff-YWlyZmxvdy9ob29rcy9oaXZlX2hvb2tzLnB5) | `77.6% <0%> (+1.78%)` | :arrow_up: | | ... and [4 more](https://codecov.io/gh/apache/airflow/pull/6527/diff?src=pr=tree-more) | | -- [Continue to review full report at Codecov](https://codecov.io/gh/apache/airflow/pull/6527?src=pr=continue). > **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta) > `Δ = absolute (impact)`, `ø = not affected`, `? = missing data` > Powered by [Codecov](https://codecov.io/gh/apache/airflow/pull/6527?src=pr=footer). Last update [33ddcd9...072034e](https://codecov.io/gh/apache/airflow/pull/6527?src=pr=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments). This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Work started] (AIRFLOW-5873) KubernetesPodOperator fixes and test
[ https://issues.apache.org/jira/browse/AIRFLOW-5873?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Work on AIRFLOW-5873 started by David de L.. > KubernetesPodOperator fixes and test > > > Key: AIRFLOW-5873 > URL: https://issues.apache.org/jira/browse/AIRFLOW-5873 > Project: Apache Airflow > Issue Type: Bug > Components: operators >Affects Versions: 1.10.6 >Reporter: David de L. >Assignee: David de L. >Priority: Major > Fix For: 1.10.7 > > > * `KubernetesPodOperator` kwarg `resources` is erroneously passed to > `base_operator`, instead should only go to `PodGenerator`. The two have > different syntax. (both on `master` and `v1-10-test` branches) > * `KubernetesPodOperator` kwarg `in_cluster` erroneously defaults to False > in comparison to `default_args.py` > * `kubernetes/pod.py`: `Resources` does not have `__slots__` so accepts > arbitrary values in `setattr` (present on `v1-10-test` branch > [https://github.com/apache/airflow/blame/v1-10-test/airflow/contrib/operators/kubernetes_pod_operator.py#L166]) > * `v1-10-test` is behind `master` with KubernetesPodOperator fixes and > refactors > ** e.g. move kubernetes folder one level up from `/contrib` > [https://github.com/apache/airflow/blame/4dd24a2c595d4042ffe745aed947eaaea6abb652/airflow/contrib/operators/kubernetes_pod_operator.py#L21] > ** fix `xcom_push` to `do_xcom_push` > [https://github.com/apache/airflow/blame/4dd24a2c595d4042ffe745aed947eaaea6abb652/airflow/contrib/operators/kubernetes_pod_operator.py#L90] -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (AIRFLOW-5881) Dag gets stuck in "Scheduled" State when scheduling a large number of tasks
David Hartig created AIRFLOW-5881: - Summary: Dag gets stuck in "Scheduled" State when scheduling a large number of tasks Key: AIRFLOW-5881 URL: https://issues.apache.org/jira/browse/AIRFLOW-5881 Project: Apache Airflow Issue Type: Bug Components: scheduler Affects Versions: 1.10.6 Reporter: David Hartig Attachments: 2 (1).log, airflow.cnf Running with the KubernetesExecutor in and AKS cluster, when we upgraded to version 1.10.6 we noticed that the all the Dags stop making progress but start running and immediate exiting with the following message: "Instance State' FAILED: Task is in the 'scheduled' state which is not a valid state for execution. The task must be cleared in order to be run." See attached log file for the worker. Nothing seems out of the ordinary in the Scheduler log. Reverting to 1.10.5 clears the problem. Note that at the time of the failure maybe 100 or so tasks are in this state, with 70 coming from one highly parallelized dag. Attached is the also a redacted airflow config flag. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[GitHub] [airflow] codecov-io edited a comment on issue #6527: [AIRFLOW-5876][part of AIRFLOW-5697][depends on AIRFLOW-5716] Fetch all Dataflow jobs during searching
codecov-io edited a comment on issue #6527: [AIRFLOW-5876][part of AIRFLOW-5697][depends on AIRFLOW-5716] Fetch all Dataflow jobs during searching URL: https://github.com/apache/airflow/pull/6527#issuecomment-551934161 # [Codecov](https://codecov.io/gh/apache/airflow/pull/6527?src=pr=h1) Report > Merging [#6527](https://codecov.io/gh/apache/airflow/pull/6527?src=pr=desc) into [master](https://codecov.io/gh/apache/airflow/commit/33ddcd9ef7b0c7c1e888e4aad93a641022571a9f?src=pr=desc) will **decrease** coverage by `0.05%`. > The diff coverage is `85.71%`. [](https://codecov.io/gh/apache/airflow/pull/6527?src=pr=tree) ```diff @@Coverage Diff @@ ## master#6527 +/- ## == - Coverage83.8% 83.75% -0.06% == Files 635 635 Lines 3675036751 +1 == - Hits3080030779 -21 - Misses 5950 5972 +22 ``` | [Impacted Files](https://codecov.io/gh/apache/airflow/pull/6527?src=pr=tree) | Coverage Δ | | |---|---|---| | [airflow/gcp/hooks/dataflow.py](https://codecov.io/gh/apache/airflow/pull/6527/diff?src=pr=tree#diff-YWlyZmxvdy9nY3AvaG9va3MvZGF0YWZsb3cucHk=) | `91.46% <85.71%> (-0.79%)` | :arrow_down: | | [airflow/kubernetes/volume\_mount.py](https://codecov.io/gh/apache/airflow/pull/6527/diff?src=pr=tree#diff-YWlyZmxvdy9rdWJlcm5ldGVzL3ZvbHVtZV9tb3VudC5weQ==) | `44.44% <0%> (-55.56%)` | :arrow_down: | | [airflow/kubernetes/volume.py](https://codecov.io/gh/apache/airflow/pull/6527/diff?src=pr=tree#diff-YWlyZmxvdy9rdWJlcm5ldGVzL3ZvbHVtZS5weQ==) | `52.94% <0%> (-47.06%)` | :arrow_down: | | [airflow/kubernetes/pod\_launcher.py](https://codecov.io/gh/apache/airflow/pull/6527/diff?src=pr=tree#diff-YWlyZmxvdy9rdWJlcm5ldGVzL3BvZF9sYXVuY2hlci5weQ==) | `45.25% <0%> (-46.72%)` | :arrow_down: | | [airflow/kubernetes/kube\_client.py](https://codecov.io/gh/apache/airflow/pull/6527/diff?src=pr=tree#diff-YWlyZmxvdy9rdWJlcm5ldGVzL2t1YmVfY2xpZW50LnB5) | `33.33% <0%> (-41.67%)` | :arrow_down: | | [...rflow/contrib/operators/kubernetes\_pod\_operator.py](https://codecov.io/gh/apache/airflow/pull/6527/diff?src=pr=tree#diff-YWlyZmxvdy9jb250cmliL29wZXJhdG9ycy9rdWJlcm5ldGVzX3BvZF9vcGVyYXRvci5weQ==) | `70.14% <0%> (-28.36%)` | :arrow_down: | | [airflow/utils/dag\_processing.py](https://codecov.io/gh/apache/airflow/pull/6527/diff?src=pr=tree#diff-YWlyZmxvdy91dGlscy9kYWdfcHJvY2Vzc2luZy5weQ==) | `58.48% <0%> (+0.32%)` | :arrow_up: | | [airflow/hooks/dbapi\_hook.py](https://codecov.io/gh/apache/airflow/pull/6527/diff?src=pr=tree#diff-YWlyZmxvdy9ob29rcy9kYmFwaV9ob29rLnB5) | `91.52% <0%> (+0.84%)` | :arrow_up: | | [airflow/models/connection.py](https://codecov.io/gh/apache/airflow/pull/6527/diff?src=pr=tree#diff-YWlyZmxvdy9tb2RlbHMvY29ubmVjdGlvbi5weQ==) | `65% <0%> (+1.11%)` | :arrow_up: | | [airflow/hooks/hive\_hooks.py](https://codecov.io/gh/apache/airflow/pull/6527/diff?src=pr=tree#diff-YWlyZmxvdy9ob29rcy9oaXZlX2hvb2tzLnB5) | `77.6% <0%> (+1.78%)` | :arrow_up: | | ... and [4 more](https://codecov.io/gh/apache/airflow/pull/6527/diff?src=pr=tree-more) | | -- [Continue to review full report at Codecov](https://codecov.io/gh/apache/airflow/pull/6527?src=pr=continue). > **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta) > `Δ = absolute (impact)`, `ø = not affected`, `? = missing data` > Powered by [Codecov](https://codecov.io/gh/apache/airflow/pull/6527?src=pr=footer). Last update [33ddcd9...072034e](https://codecov.io/gh/apache/airflow/pull/6527?src=pr=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments). This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [airflow] codecov-io commented on issue #6527: [AIRFLOW-5876][part of AIRFLOW-5697][depends on AIRFLOW-5716] Fetch all Dataflow jobs during searching
codecov-io commented on issue #6527: [AIRFLOW-5876][part of AIRFLOW-5697][depends on AIRFLOW-5716] Fetch all Dataflow jobs during searching URL: https://github.com/apache/airflow/pull/6527#issuecomment-551934161 # [Codecov](https://codecov.io/gh/apache/airflow/pull/6527?src=pr=h1) Report > Merging [#6527](https://codecov.io/gh/apache/airflow/pull/6527?src=pr=desc) into [master](https://codecov.io/gh/apache/airflow/commit/33ddcd9ef7b0c7c1e888e4aad93a641022571a9f?src=pr=desc) will **decrease** coverage by `0.05%`. > The diff coverage is `85.71%`. [](https://codecov.io/gh/apache/airflow/pull/6527?src=pr=tree) ```diff @@Coverage Diff @@ ## master#6527 +/- ## == - Coverage83.8% 83.75% -0.06% == Files 635 635 Lines 3675036751 +1 == - Hits3080030779 -21 - Misses 5950 5972 +22 ``` | [Impacted Files](https://codecov.io/gh/apache/airflow/pull/6527?src=pr=tree) | Coverage Δ | | |---|---|---| | [airflow/gcp/hooks/dataflow.py](https://codecov.io/gh/apache/airflow/pull/6527/diff?src=pr=tree#diff-YWlyZmxvdy9nY3AvaG9va3MvZGF0YWZsb3cucHk=) | `91.46% <85.71%> (-0.79%)` | :arrow_down: | | [airflow/kubernetes/volume\_mount.py](https://codecov.io/gh/apache/airflow/pull/6527/diff?src=pr=tree#diff-YWlyZmxvdy9rdWJlcm5ldGVzL3ZvbHVtZV9tb3VudC5weQ==) | `44.44% <0%> (-55.56%)` | :arrow_down: | | [airflow/kubernetes/volume.py](https://codecov.io/gh/apache/airflow/pull/6527/diff?src=pr=tree#diff-YWlyZmxvdy9rdWJlcm5ldGVzL3ZvbHVtZS5weQ==) | `52.94% <0%> (-47.06%)` | :arrow_down: | | [airflow/kubernetes/pod\_launcher.py](https://codecov.io/gh/apache/airflow/pull/6527/diff?src=pr=tree#diff-YWlyZmxvdy9rdWJlcm5ldGVzL3BvZF9sYXVuY2hlci5weQ==) | `45.25% <0%> (-46.72%)` | :arrow_down: | | [airflow/kubernetes/kube\_client.py](https://codecov.io/gh/apache/airflow/pull/6527/diff?src=pr=tree#diff-YWlyZmxvdy9rdWJlcm5ldGVzL2t1YmVfY2xpZW50LnB5) | `33.33% <0%> (-41.67%)` | :arrow_down: | | [...rflow/contrib/operators/kubernetes\_pod\_operator.py](https://codecov.io/gh/apache/airflow/pull/6527/diff?src=pr=tree#diff-YWlyZmxvdy9jb250cmliL29wZXJhdG9ycy9rdWJlcm5ldGVzX3BvZF9vcGVyYXRvci5weQ==) | `70.14% <0%> (-28.36%)` | :arrow_down: | | [airflow/utils/dag\_processing.py](https://codecov.io/gh/apache/airflow/pull/6527/diff?src=pr=tree#diff-YWlyZmxvdy91dGlscy9kYWdfcHJvY2Vzc2luZy5weQ==) | `58.48% <0%> (+0.32%)` | :arrow_up: | | [airflow/hooks/dbapi\_hook.py](https://codecov.io/gh/apache/airflow/pull/6527/diff?src=pr=tree#diff-YWlyZmxvdy9ob29rcy9kYmFwaV9ob29rLnB5) | `91.52% <0%> (+0.84%)` | :arrow_up: | | [airflow/models/connection.py](https://codecov.io/gh/apache/airflow/pull/6527/diff?src=pr=tree#diff-YWlyZmxvdy9tb2RlbHMvY29ubmVjdGlvbi5weQ==) | `65% <0%> (+1.11%)` | :arrow_up: | | [airflow/hooks/hive\_hooks.py](https://codecov.io/gh/apache/airflow/pull/6527/diff?src=pr=tree#diff-YWlyZmxvdy9ob29rcy9oaXZlX2hvb2tzLnB5) | `77.6% <0%> (+1.78%)` | :arrow_up: | | ... and [4 more](https://codecov.io/gh/apache/airflow/pull/6527/diff?src=pr=tree-more) | | -- [Continue to review full report at Codecov](https://codecov.io/gh/apache/airflow/pull/6527?src=pr=continue). > **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta) > `Δ = absolute (impact)`, `ø = not affected`, `? = missing data` > Powered by [Codecov](https://codecov.io/gh/apache/airflow/pull/6527?src=pr=footer). Last update [33ddcd9...072034e](https://codecov.io/gh/apache/airflow/pull/6527?src=pr=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments). This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Commented] (AIRFLOW-5823) Add a new hook for ingesting data into Apache Pinot
[ https://issues.apache.org/jira/browse/AIRFLOW-5823?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16970547#comment-16970547 ] ASF subversion and git services commented on AIRFLOW-5823: -- Commit 813cd431204b71f4e7c27f2bee9fafcc910da4d0 in airflow's branch refs/heads/master from Kengo Seki [ https://gitbox.apache.org/repos/asf?p=airflow.git;h=813cd43 ] [AIRFLOW-5823] Add a new hook for ingesting data into Apache Pinot (#6482) > Add a new hook for ingesting data into Apache Pinot > --- > > Key: AIRFLOW-5823 > URL: https://issues.apache.org/jira/browse/AIRFLOW-5823 > Project: Apache Airflow > Issue Type: New Feature > Components: hooks >Affects Versions: 1.10.5 >Reporter: Kengo Seki >Assignee: Kengo Seki >Priority: Major > Fix For: 1.10.7 > > > Currently, Airflow has PinotDbApiHook for querying Pinot, but there's no > specific hook for ingesting data into it. I'd like to add a new hook which > provides a minimum set of pinot-admin.sh subcommands required to ingest > offline data. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Resolved] (AIRFLOW-5823) Add a new hook for ingesting data into Apache Pinot
[ https://issues.apache.org/jira/browse/AIRFLOW-5823?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kaxil Naik resolved AIRFLOW-5823. - Fix Version/s: 1.10.7 Resolution: Fixed > Add a new hook for ingesting data into Apache Pinot > --- > > Key: AIRFLOW-5823 > URL: https://issues.apache.org/jira/browse/AIRFLOW-5823 > Project: Apache Airflow > Issue Type: New Feature > Components: hooks >Affects Versions: 1.10.5 >Reporter: Kengo Seki >Assignee: Kengo Seki >Priority: Major > Fix For: 1.10.7 > > > Currently, Airflow has PinotDbApiHook for querying Pinot, but there's no > specific hook for ingesting data into it. I'd like to add a new hook which > provides a minimum set of pinot-admin.sh subcommands required to ingest > offline data. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (AIRFLOW-5823) Add a new hook for ingesting data into Apache Pinot
[ https://issues.apache.org/jira/browse/AIRFLOW-5823?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16970546#comment-16970546 ] ASF GitHub Bot commented on AIRFLOW-5823: - kaxil commented on pull request #6482: [AIRFLOW-5823] Add a new hook for ingesting data into Apache Pinot URL: https://github.com/apache/airflow/pull/6482 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org > Add a new hook for ingesting data into Apache Pinot > --- > > Key: AIRFLOW-5823 > URL: https://issues.apache.org/jira/browse/AIRFLOW-5823 > Project: Apache Airflow > Issue Type: New Feature > Components: hooks >Affects Versions: 1.10.5 >Reporter: Kengo Seki >Assignee: Kengo Seki >Priority: Major > > Currently, Airflow has PinotDbApiHook for querying Pinot, but there's no > specific hook for ingesting data into it. I'd like to add a new hook which > provides a minimum set of pinot-admin.sh subcommands required to ingest > offline data. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[GitHub] [airflow] kaxil merged pull request #6482: [AIRFLOW-5823] Add a new hook for ingesting data into Apache Pinot
kaxil merged pull request #6482: [AIRFLOW-5823] Add a new hook for ingesting data into Apache Pinot URL: https://github.com/apache/airflow/pull/6482 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [airflow] codecov-io edited a comment on issue #6386: [AIRFLOW-5716][part of AIRFLOW-5697] Simplify DataflowJobsController logic
codecov-io edited a comment on issue #6386: [AIRFLOW-5716][part of AIRFLOW-5697] Simplify DataflowJobsController logic URL: https://github.com/apache/airflow/pull/6386#issuecomment-546581408 # [Codecov](https://codecov.io/gh/apache/airflow/pull/6386?src=pr=h1) Report > Merging [#6386](https://codecov.io/gh/apache/airflow/pull/6386?src=pr=desc) into [master](https://codecov.io/gh/apache/airflow/commit/33ddcd9ef7b0c7c1e888e4aad93a641022571a9f?src=pr=desc) will **decrease** coverage by `0.06%`. > The diff coverage is `85.29%`. [](https://codecov.io/gh/apache/airflow/pull/6386?src=pr=tree) ```diff @@Coverage Diff @@ ## master#6386 +/- ## == - Coverage83.8% 83.74% -0.07% == Files 635 635 Lines 3675036743 -7 == - Hits3080030769 -31 - Misses 5950 5974 +24 ``` | [Impacted Files](https://codecov.io/gh/apache/airflow/pull/6386?src=pr=tree) | Coverage Δ | | |---|---|---| | [airflow/gcp/hooks/dataflow.py](https://codecov.io/gh/apache/airflow/pull/6386/diff?src=pr=tree#diff-YWlyZmxvdy9nY3AvaG9va3MvZGF0YWZsb3cucHk=) | `91.59% <85.29%> (-0.65%)` | :arrow_down: | | [airflow/kubernetes/volume\_mount.py](https://codecov.io/gh/apache/airflow/pull/6386/diff?src=pr=tree#diff-YWlyZmxvdy9rdWJlcm5ldGVzL3ZvbHVtZV9tb3VudC5weQ==) | `44.44% <0%> (-55.56%)` | :arrow_down: | | [airflow/kubernetes/volume.py](https://codecov.io/gh/apache/airflow/pull/6386/diff?src=pr=tree#diff-YWlyZmxvdy9rdWJlcm5ldGVzL3ZvbHVtZS5weQ==) | `52.94% <0%> (-47.06%)` | :arrow_down: | | [airflow/kubernetes/pod\_launcher.py](https://codecov.io/gh/apache/airflow/pull/6386/diff?src=pr=tree#diff-YWlyZmxvdy9rdWJlcm5ldGVzL3BvZF9sYXVuY2hlci5weQ==) | `45.25% <0%> (-46.72%)` | :arrow_down: | | [airflow/kubernetes/kube\_client.py](https://codecov.io/gh/apache/airflow/pull/6386/diff?src=pr=tree#diff-YWlyZmxvdy9rdWJlcm5ldGVzL2t1YmVfY2xpZW50LnB5) | `33.33% <0%> (-41.67%)` | :arrow_down: | | [...rflow/contrib/operators/kubernetes\_pod\_operator.py](https://codecov.io/gh/apache/airflow/pull/6386/diff?src=pr=tree#diff-YWlyZmxvdy9jb250cmliL29wZXJhdG9ycy9rdWJlcm5ldGVzX3BvZF9vcGVyYXRvci5weQ==) | `70.14% <0%> (-28.36%)` | :arrow_down: | | [airflow/models/taskinstance.py](https://codecov.io/gh/apache/airflow/pull/6386/diff?src=pr=tree#diff-YWlyZmxvdy9tb2RlbHMvdGFza2luc3RhbmNlLnB5) | `93.28% <0%> (-0.51%)` | :arrow_down: | | [airflow/utils/dag\_processing.py](https://codecov.io/gh/apache/airflow/pull/6386/diff?src=pr=tree#diff-YWlyZmxvdy91dGlscy9kYWdfcHJvY2Vzc2luZy5weQ==) | `58.48% <0%> (+0.32%)` | :arrow_up: | | [airflow/hooks/dbapi\_hook.py](https://codecov.io/gh/apache/airflow/pull/6386/diff?src=pr=tree#diff-YWlyZmxvdy9ob29rcy9kYmFwaV9ob29rLnB5) | `91.52% <0%> (+0.84%)` | :arrow_up: | | [airflow/models/connection.py](https://codecov.io/gh/apache/airflow/pull/6386/diff?src=pr=tree#diff-YWlyZmxvdy9tb2RlbHMvY29ubmVjdGlvbi5weQ==) | `65% <0%> (+1.11%)` | :arrow_up: | | ... and [5 more](https://codecov.io/gh/apache/airflow/pull/6386/diff?src=pr=tree-more) | | -- [Continue to review full report at Codecov](https://codecov.io/gh/apache/airflow/pull/6386?src=pr=continue). > **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta) > `Δ = absolute (impact)`, `ø = not affected`, `? = missing data` > Powered by [Codecov](https://codecov.io/gh/apache/airflow/pull/6386?src=pr=footer). Last update [33ddcd9...679ae91](https://codecov.io/gh/apache/airflow/pull/6386?src=pr=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments). This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[airflow-site] branch aip-11 updated: Add version selector (#143)
This is an automated email from the ASF dual-hosted git repository.
kamilbregula pushed a commit to branch aip-11
in repository https://gitbox.apache.org/repos/asf/airflow-site.git
The following commit(s) were added to refs/heads/aip-11 by this push:
new faad101 Add version selector (#143)
faad101 is described below
commit faad101b593adab46556d031169ca9d91df7f667
Author: Kamil Breguła
AuthorDate: Fri Nov 8 18:26:44 2019 +0100
Add version selector (#143)
---
Dockerfile | 3 ++
.../show_docs_index_json.sh| 24 --
landing-pages/.gitignore | 2 +-
landing-pages/create-index.js | 2 +-
.../assets/scss/_dropdown.scss}| 24 +++---
landing-pages/site/assets/scss/main-custom.scss| 1 +
.../site/layouts/partials}/version-selector.html | 24 +-
landing-pages/site/static/{ => _gen}/indexes/.keep | 0
landing-pages/src/docs-index.js| 1 +
landing-pages/src/index.js | 1 +
landing-pages/src/js/searchBlogPosts.js| 4 +-
landing-pages/src/js/versionSelector.js| 56 ++
site.sh| 6 +++
sphinx_airflow_theme/demo/conf.py | 3 +-
.../sphinx_airflow_theme/__init__.py | 1 -
.../sphinx_airflow_theme/version-selector.html | 20
16 files changed, 121 insertions(+), 51 deletions(-)
diff --git a/Dockerfile b/Dockerfile
index 585cbc7..d338ebc 100644
--- a/Dockerfile
+++ b/Dockerfile
@@ -54,6 +54,9 @@ RUN curl -sS https://dl.yarnpkg.com/debian/pubkey.gpg |
apt-key add - \
&& apt-get clean \
&& rm -rf /var/lib/apt/lists/*
+RUN curl -sL
"https://github.com/stedolan/jq/releases/download/jq-1.6/jq-linux64; >
/usr/local/bin/jq \
+&& chmod +x /usr/local/bin/jq
+
RUN HUGOHOME="$(mktemp -d)" \
&& export HUGOHOME \
&& curl -sL
https://github.com/gohugoio/hugo/releases/download/v0.58.3/hugo_extended_0.58.3_Linux-64bit.tar.gz
> "${HUGOHOME}/hugo.tar.gz" \
diff --git a/sphinx_airflow_theme/sphinx_airflow_theme/__init__.py
b/docs-archive/show_docs_index_json.sh
old mode 100644
new mode 100755
similarity index 60%
copy from sphinx_airflow_theme/sphinx_airflow_theme/__init__.py
copy to docs-archive/show_docs_index_json.sh
index 7165840..297d737
--- a/sphinx_airflow_theme/sphinx_airflow_theme/__init__.py
+++ b/docs-archive/show_docs_index_json.sh
@@ -1,3 +1,5 @@
+#!/usr/bin/env bash
+
# Licensed to the Apache Software Foundation (ASF) under one
# or more contributor license agreements. See the NOTICE file
# distributed with this work for additional information
@@ -15,20 +17,14 @@
# specific language governing permissions and limitations
# under the License.
-from os import path
-
-__version__ = '0.0.1'
-__version_full__ = __version__
-
+set -euo pipefail
-def get_html_theme_path():
-"""Return list of HTML theme paths."""
-cur_dir = path.abspath(path.dirname(path.dirname(__file__)))
-return cur_dir
+MY_DIR="$(cd "$(dirname "$0")" && pwd)"
+pushd "${MY_DIR}" &>/dev/null || exit 1
+jq -n '{stable: $stable, versions: $versions[0:-1] | split("\n") }' \
+-M \
+--rawfile stable <(cat stable.txt | tr -d '[:space:]') \
+--rawfile versions <(ls -d ./*/ | cut -d "/" -f 2)
-# See
http://www.sphinx-doc.org/en/stable/theming.html#distribute-your-theme-as-a-python-package
-def setup(app):
-app.add_html_theme('sphinx_airflow_theme',
path.abspath(path.dirname(__file__)))
-app.add_stylesheet('_gen/css//main-custom.min.css')
-app.add_js_file('_gen/js/docs.js')
+popd &>/dev/null || exit 1
diff --git a/landing-pages/.gitignore b/landing-pages/.gitignore
index c8b191f..4aa6c9e 100644
--- a/landing-pages/.gitignore
+++ b/landing-pages/.gitignore
@@ -2,4 +2,4 @@ node_modules/
dist/
site/data/webpack.json
resources/
-site/static/indexes/
+site/static/_gen/
diff --git a/landing-pages/create-index.js b/landing-pages/create-index.js
index 69ec28f..495b5fc 100644
--- a/landing-pages/create-index.js
+++ b/landing-pages/create-index.js
@@ -27,7 +27,7 @@ const parse =
promisify(frontMatterParser.parse.bind(frontMatterParser));
const lunrjs = require("lunr");
const contentDirectory = `${__dirname}/site/content`;
-const outputtDirectory = `${__dirname}/site/static/indexes`;
+const outputtDirectory = `${__dirname}/site/static/_gen/indexes`;
async function isDirectoryExists(dirPath) {
diff --git a/landing-pages/src/docs-index.js
b/landing-pages/site/assets/scss/_dropdown.scss
similarity index 75%
copy from landing-pages/src/docs-index.js
copy to landing-pages/site/assets/scss/_dropdown.scss
index 884ea65..888d67c 100644
--- a/landing-pages/src/docs-index.js
+++ b/landing-pages/site/assets/scss/_dropdown.scss
@@ -16,11 +16,21 @@
* specific language governing permissions and limitations
* under the License.
*/
+@import "fonts";
[GitHub] [airflow] codecov-io edited a comment on issue #6484: [AIRFLOW-5832] add pagerduty hook
codecov-io edited a comment on issue #6484: [AIRFLOW-5832] add pagerduty hook URL: https://github.com/apache/airflow/pull/6484#issuecomment-548906525 # [Codecov](https://codecov.io/gh/apache/airflow/pull/6484?src=pr=h1) Report > Merging [#6484](https://codecov.io/gh/apache/airflow/pull/6484?src=pr=desc) into [master](https://codecov.io/gh/apache/airflow/commit/33ddcd9ef7b0c7c1e888e4aad93a641022571a9f?src=pr=desc) will **decrease** coverage by `0.03%`. > The diff coverage is `81.57%`. [](https://codecov.io/gh/apache/airflow/pull/6484?src=pr=tree) ```diff @@Coverage Diff @@ ## master#6484 +/- ## == - Coverage83.8% 83.77% -0.04% == Files 635 636 +1 Lines 3675036887 +137 == + Hits3080030901 +101 - Misses 5950 5986 +36 ``` | [Impacted Files](https://codecov.io/gh/apache/airflow/pull/6484?src=pr=tree) | Coverage Δ | | |---|---|---| | [airflow/contrib/hooks/pagerduty\_hook.py](https://codecov.io/gh/apache/airflow/pull/6484/diff?src=pr=tree#diff-YWlyZmxvdy9jb250cmliL2hvb2tzL3BhZ2VyZHV0eV9ob29rLnB5) | `81.57% <81.57%> (ø)` | | | [airflow/kubernetes/volume\_mount.py](https://codecov.io/gh/apache/airflow/pull/6484/diff?src=pr=tree#diff-YWlyZmxvdy9rdWJlcm5ldGVzL3ZvbHVtZV9tb3VudC5weQ==) | `44.44% <0%> (-55.56%)` | :arrow_down: | | [airflow/kubernetes/volume.py](https://codecov.io/gh/apache/airflow/pull/6484/diff?src=pr=tree#diff-YWlyZmxvdy9rdWJlcm5ldGVzL3ZvbHVtZS5weQ==) | `52.94% <0%> (-47.06%)` | :arrow_down: | | [airflow/kubernetes/pod\_launcher.py](https://codecov.io/gh/apache/airflow/pull/6484/diff?src=pr=tree#diff-YWlyZmxvdy9rdWJlcm5ldGVzL3BvZF9sYXVuY2hlci5weQ==) | `45.25% <0%> (-46.72%)` | :arrow_down: | | [airflow/kubernetes/kube\_client.py](https://codecov.io/gh/apache/airflow/pull/6484/diff?src=pr=tree#diff-YWlyZmxvdy9rdWJlcm5ldGVzL2t1YmVfY2xpZW50LnB5) | `33.33% <0%> (-41.67%)` | :arrow_down: | | [...rflow/contrib/operators/kubernetes\_pod\_operator.py](https://codecov.io/gh/apache/airflow/pull/6484/diff?src=pr=tree#diff-YWlyZmxvdy9jb250cmliL29wZXJhdG9ycy9rdWJlcm5ldGVzX3BvZF9vcGVyYXRvci5weQ==) | `70.14% <0%> (-28.36%)` | :arrow_down: | | [airflow/utils/db.py](https://codecov.io/gh/apache/airflow/pull/6484/diff?src=pr=tree#diff-YWlyZmxvdy91dGlscy9kYi5weQ==) | `90.17% <0%> (+0.08%)` | :arrow_up: | | [airflow/utils/dag\_processing.py](https://codecov.io/gh/apache/airflow/pull/6484/diff?src=pr=tree#diff-YWlyZmxvdy91dGlscy9kYWdfcHJvY2Vzc2luZy5weQ==) | `58.48% <0%> (+0.32%)` | :arrow_up: | | [airflow/hooks/dbapi\_hook.py](https://codecov.io/gh/apache/airflow/pull/6484/diff?src=pr=tree#diff-YWlyZmxvdy9ob29rcy9kYmFwaV9ob29rLnB5) | `91.52% <0%> (+0.84%)` | :arrow_up: | | ... and [6 more](https://codecov.io/gh/apache/airflow/pull/6484/diff?src=pr=tree-more) | | -- [Continue to review full report at Codecov](https://codecov.io/gh/apache/airflow/pull/6484?src=pr=continue). > **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta) > `Δ = absolute (impact)`, `ø = not affected`, `? = missing data` > Powered by [Codecov](https://codecov.io/gh/apache/airflow/pull/6484?src=pr=footer). Last update [33ddcd9...b278f58](https://codecov.io/gh/apache/airflow/pull/6484?src=pr=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments). This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [airflow] codecov-io edited a comment on issue #6484: [AIRFLOW-5832] add pagerduty hook
codecov-io edited a comment on issue #6484: [AIRFLOW-5832] add pagerduty hook URL: https://github.com/apache/airflow/pull/6484#issuecomment-548906525 # [Codecov](https://codecov.io/gh/apache/airflow/pull/6484?src=pr=h1) Report > Merging [#6484](https://codecov.io/gh/apache/airflow/pull/6484?src=pr=desc) into [master](https://codecov.io/gh/apache/airflow/commit/33ddcd9ef7b0c7c1e888e4aad93a641022571a9f?src=pr=desc) will **decrease** coverage by `0.03%`. > The diff coverage is `81.57%`. [](https://codecov.io/gh/apache/airflow/pull/6484?src=pr=tree) ```diff @@Coverage Diff @@ ## master#6484 +/- ## == - Coverage83.8% 83.77% -0.04% == Files 635 636 +1 Lines 3675036887 +137 == + Hits3080030901 +101 - Misses 5950 5986 +36 ``` | [Impacted Files](https://codecov.io/gh/apache/airflow/pull/6484?src=pr=tree) | Coverage Δ | | |---|---|---| | [airflow/contrib/hooks/pagerduty\_hook.py](https://codecov.io/gh/apache/airflow/pull/6484/diff?src=pr=tree#diff-YWlyZmxvdy9jb250cmliL2hvb2tzL3BhZ2VyZHV0eV9ob29rLnB5) | `81.57% <81.57%> (ø)` | | | [airflow/kubernetes/volume\_mount.py](https://codecov.io/gh/apache/airflow/pull/6484/diff?src=pr=tree#diff-YWlyZmxvdy9rdWJlcm5ldGVzL3ZvbHVtZV9tb3VudC5weQ==) | `44.44% <0%> (-55.56%)` | :arrow_down: | | [airflow/kubernetes/volume.py](https://codecov.io/gh/apache/airflow/pull/6484/diff?src=pr=tree#diff-YWlyZmxvdy9rdWJlcm5ldGVzL3ZvbHVtZS5weQ==) | `52.94% <0%> (-47.06%)` | :arrow_down: | | [airflow/kubernetes/pod\_launcher.py](https://codecov.io/gh/apache/airflow/pull/6484/diff?src=pr=tree#diff-YWlyZmxvdy9rdWJlcm5ldGVzL3BvZF9sYXVuY2hlci5weQ==) | `45.25% <0%> (-46.72%)` | :arrow_down: | | [airflow/kubernetes/kube\_client.py](https://codecov.io/gh/apache/airflow/pull/6484/diff?src=pr=tree#diff-YWlyZmxvdy9rdWJlcm5ldGVzL2t1YmVfY2xpZW50LnB5) | `33.33% <0%> (-41.67%)` | :arrow_down: | | [...rflow/contrib/operators/kubernetes\_pod\_operator.py](https://codecov.io/gh/apache/airflow/pull/6484/diff?src=pr=tree#diff-YWlyZmxvdy9jb250cmliL29wZXJhdG9ycy9rdWJlcm5ldGVzX3BvZF9vcGVyYXRvci5weQ==) | `70.14% <0%> (-28.36%)` | :arrow_down: | | [airflow/utils/db.py](https://codecov.io/gh/apache/airflow/pull/6484/diff?src=pr=tree#diff-YWlyZmxvdy91dGlscy9kYi5weQ==) | `90.17% <0%> (+0.08%)` | :arrow_up: | | [airflow/utils/dag\_processing.py](https://codecov.io/gh/apache/airflow/pull/6484/diff?src=pr=tree#diff-YWlyZmxvdy91dGlscy9kYWdfcHJvY2Vzc2luZy5weQ==) | `58.48% <0%> (+0.32%)` | :arrow_up: | | [airflow/hooks/dbapi\_hook.py](https://codecov.io/gh/apache/airflow/pull/6484/diff?src=pr=tree#diff-YWlyZmxvdy9ob29rcy9kYmFwaV9ob29rLnB5) | `91.52% <0%> (+0.84%)` | :arrow_up: | | ... and [6 more](https://codecov.io/gh/apache/airflow/pull/6484/diff?src=pr=tree-more) | | -- [Continue to review full report at Codecov](https://codecov.io/gh/apache/airflow/pull/6484?src=pr=continue). > **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta) > `Δ = absolute (impact)`, `ø = not affected`, `? = missing data` > Powered by [Codecov](https://codecov.io/gh/apache/airflow/pull/6484?src=pr=footer). Last update [33ddcd9...b278f58](https://codecov.io/gh/apache/airflow/pull/6484?src=pr=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments). This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [airflow] codecov-io edited a comment on issue #6484: [AIRFLOW-5832] add pagerduty hook
codecov-io edited a comment on issue #6484: [AIRFLOW-5832] add pagerduty hook URL: https://github.com/apache/airflow/pull/6484#issuecomment-548906525 # [Codecov](https://codecov.io/gh/apache/airflow/pull/6484?src=pr=h1) Report > Merging [#6484](https://codecov.io/gh/apache/airflow/pull/6484?src=pr=desc) into [master](https://codecov.io/gh/apache/airflow/commit/33ddcd9ef7b0c7c1e888e4aad93a641022571a9f?src=pr=desc) will **decrease** coverage by `0.03%`. > The diff coverage is `81.57%`. [](https://codecov.io/gh/apache/airflow/pull/6484?src=pr=tree) ```diff @@Coverage Diff @@ ## master#6484 +/- ## == - Coverage83.8% 83.77% -0.04% == Files 635 636 +1 Lines 3675036887 +137 == + Hits3080030901 +101 - Misses 5950 5986 +36 ``` | [Impacted Files](https://codecov.io/gh/apache/airflow/pull/6484?src=pr=tree) | Coverage Δ | | |---|---|---| | [airflow/contrib/hooks/pagerduty\_hook.py](https://codecov.io/gh/apache/airflow/pull/6484/diff?src=pr=tree#diff-YWlyZmxvdy9jb250cmliL2hvb2tzL3BhZ2VyZHV0eV9ob29rLnB5) | `81.57% <81.57%> (ø)` | | | [airflow/kubernetes/volume\_mount.py](https://codecov.io/gh/apache/airflow/pull/6484/diff?src=pr=tree#diff-YWlyZmxvdy9rdWJlcm5ldGVzL3ZvbHVtZV9tb3VudC5weQ==) | `44.44% <0%> (-55.56%)` | :arrow_down: | | [airflow/kubernetes/volume.py](https://codecov.io/gh/apache/airflow/pull/6484/diff?src=pr=tree#diff-YWlyZmxvdy9rdWJlcm5ldGVzL3ZvbHVtZS5weQ==) | `52.94% <0%> (-47.06%)` | :arrow_down: | | [airflow/kubernetes/pod\_launcher.py](https://codecov.io/gh/apache/airflow/pull/6484/diff?src=pr=tree#diff-YWlyZmxvdy9rdWJlcm5ldGVzL3BvZF9sYXVuY2hlci5weQ==) | `45.25% <0%> (-46.72%)` | :arrow_down: | | [airflow/kubernetes/kube\_client.py](https://codecov.io/gh/apache/airflow/pull/6484/diff?src=pr=tree#diff-YWlyZmxvdy9rdWJlcm5ldGVzL2t1YmVfY2xpZW50LnB5) | `33.33% <0%> (-41.67%)` | :arrow_down: | | [...rflow/contrib/operators/kubernetes\_pod\_operator.py](https://codecov.io/gh/apache/airflow/pull/6484/diff?src=pr=tree#diff-YWlyZmxvdy9jb250cmliL29wZXJhdG9ycy9rdWJlcm5ldGVzX3BvZF9vcGVyYXRvci5weQ==) | `70.14% <0%> (-28.36%)` | :arrow_down: | | [airflow/utils/db.py](https://codecov.io/gh/apache/airflow/pull/6484/diff?src=pr=tree#diff-YWlyZmxvdy91dGlscy9kYi5weQ==) | `90.17% <0%> (+0.08%)` | :arrow_up: | | [airflow/utils/dag\_processing.py](https://codecov.io/gh/apache/airflow/pull/6484/diff?src=pr=tree#diff-YWlyZmxvdy91dGlscy9kYWdfcHJvY2Vzc2luZy5weQ==) | `58.48% <0%> (+0.32%)` | :arrow_up: | | [airflow/hooks/dbapi\_hook.py](https://codecov.io/gh/apache/airflow/pull/6484/diff?src=pr=tree#diff-YWlyZmxvdy9ob29rcy9kYmFwaV9ob29rLnB5) | `91.52% <0%> (+0.84%)` | :arrow_up: | | ... and [6 more](https://codecov.io/gh/apache/airflow/pull/6484/diff?src=pr=tree-more) | | -- [Continue to review full report at Codecov](https://codecov.io/gh/apache/airflow/pull/6484?src=pr=continue). > **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta) > `Δ = absolute (impact)`, `ø = not affected`, `? = missing data` > Powered by [Codecov](https://codecov.io/gh/apache/airflow/pull/6484?src=pr=footer). Last update [33ddcd9...b278f58](https://codecov.io/gh/apache/airflow/pull/6484?src=pr=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments). This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [airflow-site] mik-laj closed issue #37: Create a documentation page
mik-laj closed issue #37: Create a documentation page URL: https://github.com/apache/airflow-site/issues/37 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [airflow-site] mik-laj closed issue #41: Integrate the rating with the Google Analytics
mik-laj closed issue #41: Integrate the rating with the Google Analytics URL: https://github.com/apache/airflow-site/issues/41 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [airflow-site] mik-laj closed issue #43: Add a next/prev navigation in docs
mik-laj closed issue #43: Add a next/prev navigation in docs URL: https://github.com/apache/airflow-site/issues/43 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [airflow-site] mik-laj closed issue #51: Create a "Suggest a change on this page" button on landing pages
mik-laj closed issue #51: Create a "Suggest a change on this page" button on landing pages URL: https://github.com/apache/airflow-site/issues/51 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [airflow-site] mik-laj closed issue #54: Create a "Suggest a change on this page" button in sphinx
mik-laj closed issue #54: Create a "Suggest a change on this page" button in sphinx URL: https://github.com/apache/airflow-site/issues/54 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [airflow-site] mik-laj closed issue #55: Implement right navigation in Sphinx
mik-laj closed issue #55: Implement right navigation in Sphinx URL: https://github.com/apache/airflow-site/issues/55 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [airflow-site] mik-laj closed issue #40: Implement tracking page progress in right navigation
mik-laj closed issue #40: Implement tracking page progress in right navigation URL: https://github.com/apache/airflow-site/issues/40 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [airflow-site] mik-laj closed issue #39: Add left-right navigation on documentation page
mik-laj closed issue #39: Add left-right navigation on documentation page URL: https://github.com/apache/airflow-site/issues/39 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [airflow-site] mik-laj closed issue #38: Add a version/language selector on documentation page
mik-laj closed issue #38: Add a version/language selector on documentation page URL: https://github.com/apache/airflow-site/issues/38 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [airflow-site] mik-laj closed issue #23: Create a rating component
mik-laj closed issue #23: Create a rating component URL: https://github.com/apache/airflow-site/issues/23 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [airflow-site] mik-laj merged pull request #143: Add version selector
mik-laj merged pull request #143: Add version selector URL: https://github.com/apache/airflow-site/pull/143 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[airflow-site] branch aip-11 updated: Fix highlights style in docs (#145)
This is an automated email from the ASF dual-hosted git repository.
kamilbregula pushed a commit to branch aip-11
in repository https://gitbox.apache.org/repos/asf/airflow-site.git
The following commit(s) were added to refs/heads/aip-11 by this push:
new c07c4f0 Fix highlights style in docs (#145)
c07c4f0 is described below
commit c07c4f0dd8598b5f31a2142fb6a8891c5355dc12
Author: Kamil Gabryjelski
AuthorDate: Fri Nov 8 18:38:27 2019 +0100
Fix highlights style in docs (#145)
---
landing-pages/site/assets/scss/_highlights.scss| 97 ++
.../site/assets/scss/_markdown-content.scss| 16 +---
landing-pages/site/assets/scss/_rst-content.scss | 50 ++-
landing-pages/site/assets/scss/main-custom.scss| 2 +-
landing-pages/site/config.toml | 1 +
landing-pages/site/content/en/roadmap/_index.md| 6 +-
6 files changed, 111 insertions(+), 61 deletions(-)
diff --git a/landing-pages/site/assets/scss/_highlights.scss
b/landing-pages/site/assets/scss/_highlights.scss
new file mode 100644
index 000..3840a67
--- /dev/null
+++ b/landing-pages/site/assets/scss/_highlights.scss
@@ -0,0 +1,97 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+@import "colors";
+
+.chroma, .highlight {
+ background-color: #ff;
+
+ .lntd {
+vertical-align: top;
+padding: 0;
+margin: 0;
+border: 0;
+ }
+
+ .lntable {
+border-spacing: 0;
+padding: 0;
+margin: 0;
+border: 0;
+width: auto;
+overflow: auto;
+display: block;
+ }
+
+ .hl {
+display: block;
+width: 100%;
+background-color: #cc;
+ }
+
+ .lnt, .ln {
+margin-right: 0.4em;
+padding: 0 0.4em 0 0.4em;
+color: #7f7f7f;
+ }
+
+ .k, .kc, .kd, .kn, .kp, .kr {
+color: #ff;
+ }
+
+ .kt, .nc {
+color: #2b91af;
+ }
+
+ .s, .sa, .sb, .sc, .dl, .sd, .s2, .se, .sh, .si, .sx, .sr, .s1, .ss {
+color: #a31515;
+ }
+
+ .ow {
+color: #ff;
+ }
+
+ .c, .ch, .cm, .c1, .cs {
+color: #008000;
+ }
+
+ .cp, .cpf {
+color: #ff;
+ }
+
+ .ge {
+font-style: italic;
+ }
+
+ .gh, .gp, .gs, .gu {
+font-weight: bold;
+ }
+}
+
+pre {
+ margin: 40px 0;
+ padding: 16px 20px;
+ border: solid 1px map-get($colors, very-light-pink);
+ border-radius: 5px;
+ width: fit-content;
+ max-width: 100%;
+
+ span {
+@extend .monotext--brownish-grey;
+ }
+}
diff --git a/landing-pages/site/assets/scss/_markdown-content.scss
b/landing-pages/site/assets/scss/_markdown-content.scss
index 5d9bd32..6eda9f9 100644
--- a/landing-pages/site/assets/scss/_markdown-content.scss
+++ b/landing-pages/site/assets/scss/_markdown-content.scss
@@ -17,8 +17,6 @@
* under the License.
*/
-@import "colors";
-
.markdown-content {
h1, h2, h3, h4, h5 {
@extend .subtitle__large--greyish-brown;
@@ -31,17 +29,7 @@
margin-bottom: 30px;
}
- pre {
-margin: 40px 0;
-padding: 16px 20px;
-background-color: transparent !important;
-border: solid 1px map-get($colors, very-light-pink);
-border-radius: 5px;
-width: fit-content;
-max-width: 100%;
-
-span {
- @extend .monotext--brownish-grey;
-}
+ pre span {
+@extend .monotext--brownish-grey;
}
}
diff --git a/landing-pages/site/assets/scss/_rst-content.scss
b/landing-pages/site/assets/scss/_rst-content.scss
index 8b31684..98ef829 100644
--- a/landing-pages/site/assets/scss/_rst-content.scss
+++ b/landing-pages/site/assets/scss/_rst-content.scss
@@ -68,15 +68,11 @@
}
code {
-white-space: nowrap;
max-width: 100%;
-background: #fff;
-border: solid 1px #e1e4e5;
-font-size: 75%;
+color: #51504f;
padding: 0 5px;
font-family: 'Roboto Mono', monospace;
-color: #E74C3C;
-overflow-x: auto
+overflow-x: auto;
}
.note, .attention, .caution, .danger, .error, .hint, .important, .tip,
.warning, .admonition-todo, .admonition {
@@ -240,39 +236,11 @@
margin-bottom: 24px
}
- div[class^='highlight'] {
-border: 1px solid #e1e4e5;
-border-radius: 5px;
-overflow-x: auto;
-margin: 1px 0 24px 0;
+ pre {
background-color: #f2f8fe;
}
- div[class^='highlight']
[GitHub] [airflow-site] mik-laj merged pull request #145: Fix highlights style in docs
mik-laj merged pull request #145: Fix highlights style in docs URL: https://github.com/apache/airflow-site/pull/145 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [airflow] feng-tao commented on issue #6482: [AIRFLOW-5823] Add a new hook for ingesting data into Apache Pinot
feng-tao commented on issue #6482: [AIRFLOW-5823] Add a new hook for ingesting data into Apache Pinot URL: https://github.com/apache/airflow/pull/6482#issuecomment-551954027 CI is green now. @kaxil ready to merge? This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Created] (AIRFLOW-5880) Enforce unique task ids
Bas Harenslak created AIRFLOW-5880: -- Summary: Enforce unique task ids Key: AIRFLOW-5880 URL: https://issues.apache.org/jira/browse/AIRFLOW-5880 Project: Apache Airflow Issue Type: Improvement Components: core Affects Versions: 2.0.0 Reporter: Bas Harenslak Currently, task with equal ids in a DAG simply overwrite each other and the last assigned task is saved. This can lead to unexpected behaviour and an exception should be raised. We've had a PendingDeprecationWarning on this for 4 years, time to get rid of it: [https://github.com/apache/airflow/commit/385add2bf3eb7cd6103cfec0e5516234eeb72443] -- This message was sent by Atlassian Jira (v8.3.4#803005)
[GitHub] [airflow] kaxil commented on issue #6515: [AIRFLOW-XXX] GSoD: How to make DAGs production ready
kaxil commented on issue #6515: [AIRFLOW-XXX] GSoD: How to make DAGs production ready URL: https://github.com/apache/airflow/pull/6515#issuecomment-551904317 > I was also thinking to actually place this doc in the main section and name is **Best Practices** or something similar. I like that idea This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [airflow-site] kgabryje opened a new pull request #145: [depends on #123] Fix/highlights style
kgabryje opened a new pull request #145: [depends on #123] Fix/highlights style URL: https://github.com/apache/airflow-site/pull/145 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [airflow] codecov-io edited a comment on issue #6529: [AIRFLOW-5877][depends on AIRFLOW-5876, AIRFLOW-5716][part of AIRFLOW-5697] Improve job_id detection in DataflowRunner
codecov-io edited a comment on issue #6529: [AIRFLOW-5877][depends on AIRFLOW-5876,AIRFLOW-5716][part of AIRFLOW-5697] Improve job_id detection in DataflowRunner URL: https://github.com/apache/airflow/pull/6529#issuecomment-551952578 # [Codecov](https://codecov.io/gh/apache/airflow/pull/6529?src=pr=h1) Report > Merging [#6529](https://codecov.io/gh/apache/airflow/pull/6529?src=pr=desc) into [master](https://codecov.io/gh/apache/airflow/commit/33ddcd9ef7b0c7c1e888e4aad93a641022571a9f?src=pr=desc) will **decrease** coverage by `0.3%`. > The diff coverage is `86.04%`. [](https://codecov.io/gh/apache/airflow/pull/6529?src=pr=tree) ```diff @@Coverage Diff@@ ## master #6529 +/- ## = - Coverage83.8% 83.5% -0.31% = Files 635 635 Lines 36750 36751 +1 = - Hits30800 30690 -110 - Misses 59506061 +111 ``` | [Impacted Files](https://codecov.io/gh/apache/airflow/pull/6529?src=pr=tree) | Coverage Δ | | |---|---|---| | [airflow/gcp/hooks/dataflow.py](https://codecov.io/gh/apache/airflow/pull/6529/diff?src=pr=tree#diff-YWlyZmxvdy9nY3AvaG9va3MvZGF0YWZsb3cucHk=) | `91.46% <86.04%> (-0.79%)` | :arrow_down: | | [airflow/kubernetes/volume\_mount.py](https://codecov.io/gh/apache/airflow/pull/6529/diff?src=pr=tree#diff-YWlyZmxvdy9rdWJlcm5ldGVzL3ZvbHVtZV9tb3VudC5weQ==) | `44.44% <0%> (-55.56%)` | :arrow_down: | | [airflow/kubernetes/volume.py](https://codecov.io/gh/apache/airflow/pull/6529/diff?src=pr=tree#diff-YWlyZmxvdy9rdWJlcm5ldGVzL3ZvbHVtZS5weQ==) | `52.94% <0%> (-47.06%)` | :arrow_down: | | [airflow/kubernetes/pod\_launcher.py](https://codecov.io/gh/apache/airflow/pull/6529/diff?src=pr=tree#diff-YWlyZmxvdy9rdWJlcm5ldGVzL3BvZF9sYXVuY2hlci5weQ==) | `45.25% <0%> (-46.72%)` | :arrow_down: | | [airflow/kubernetes/kube\_client.py](https://codecov.io/gh/apache/airflow/pull/6529/diff?src=pr=tree#diff-YWlyZmxvdy9rdWJlcm5ldGVzL2t1YmVfY2xpZW50LnB5) | `33.33% <0%> (-41.67%)` | :arrow_down: | | [...rflow/contrib/operators/kubernetes\_pod\_operator.py](https://codecov.io/gh/apache/airflow/pull/6529/diff?src=pr=tree#diff-YWlyZmxvdy9jb250cmliL29wZXJhdG9ycy9rdWJlcm5ldGVzX3BvZF9vcGVyYXRvci5weQ==) | `70.14% <0%> (-28.36%)` | :arrow_down: | | [airflow/utils/dag\_processing.py](https://codecov.io/gh/apache/airflow/pull/6529/diff?src=pr=tree#diff-YWlyZmxvdy91dGlscy9kYWdfcHJvY2Vzc2luZy5weQ==) | `58.31% <0%> (+0.16%)` | :arrow_up: | | [airflow/jobs/local\_task\_job.py](https://codecov.io/gh/apache/airflow/pull/6529/diff?src=pr=tree#diff-YWlyZmxvdy9qb2JzL2xvY2FsX3Rhc2tfam9iLnB5) | `86.25% <0%> (+1.25%)` | :arrow_up: | -- [Continue to review full report at Codecov](https://codecov.io/gh/apache/airflow/pull/6529?src=pr=continue). > **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta) > `Δ = absolute (impact)`, `ø = not affected`, `? = missing data` > Powered by [Codecov](https://codecov.io/gh/apache/airflow/pull/6529?src=pr=footer). Last update [33ddcd9...f8512ff](https://codecov.io/gh/apache/airflow/pull/6529?src=pr=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments). This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [airflow] codecov-io commented on issue #6529: [AIRFLOW-5877][depends on AIRFLOW-5876, AIRFLOW-5716][part of AIRFLOW-5697] Improve job_id detection in DataflowRunner
codecov-io commented on issue #6529: [AIRFLOW-5877][depends on AIRFLOW-5876,AIRFLOW-5716][part of AIRFLOW-5697] Improve job_id detection in DataflowRunner URL: https://github.com/apache/airflow/pull/6529#issuecomment-551952578 # [Codecov](https://codecov.io/gh/apache/airflow/pull/6529?src=pr=h1) Report > Merging [#6529](https://codecov.io/gh/apache/airflow/pull/6529?src=pr=desc) into [master](https://codecov.io/gh/apache/airflow/commit/33ddcd9ef7b0c7c1e888e4aad93a641022571a9f?src=pr=desc) will **decrease** coverage by `0.3%`. > The diff coverage is `86.04%`. [](https://codecov.io/gh/apache/airflow/pull/6529?src=pr=tree) ```diff @@Coverage Diff@@ ## master #6529 +/- ## = - Coverage83.8% 83.5% -0.31% = Files 635 635 Lines 36750 36751 +1 = - Hits30800 30690 -110 - Misses 59506061 +111 ``` | [Impacted Files](https://codecov.io/gh/apache/airflow/pull/6529?src=pr=tree) | Coverage Δ | | |---|---|---| | [airflow/gcp/hooks/dataflow.py](https://codecov.io/gh/apache/airflow/pull/6529/diff?src=pr=tree#diff-YWlyZmxvdy9nY3AvaG9va3MvZGF0YWZsb3cucHk=) | `91.46% <86.04%> (-0.79%)` | :arrow_down: | | [airflow/kubernetes/volume\_mount.py](https://codecov.io/gh/apache/airflow/pull/6529/diff?src=pr=tree#diff-YWlyZmxvdy9rdWJlcm5ldGVzL3ZvbHVtZV9tb3VudC5weQ==) | `44.44% <0%> (-55.56%)` | :arrow_down: | | [airflow/kubernetes/volume.py](https://codecov.io/gh/apache/airflow/pull/6529/diff?src=pr=tree#diff-YWlyZmxvdy9rdWJlcm5ldGVzL3ZvbHVtZS5weQ==) | `52.94% <0%> (-47.06%)` | :arrow_down: | | [airflow/kubernetes/pod\_launcher.py](https://codecov.io/gh/apache/airflow/pull/6529/diff?src=pr=tree#diff-YWlyZmxvdy9rdWJlcm5ldGVzL3BvZF9sYXVuY2hlci5weQ==) | `45.25% <0%> (-46.72%)` | :arrow_down: | | [airflow/kubernetes/kube\_client.py](https://codecov.io/gh/apache/airflow/pull/6529/diff?src=pr=tree#diff-YWlyZmxvdy9rdWJlcm5ldGVzL2t1YmVfY2xpZW50LnB5) | `33.33% <0%> (-41.67%)` | :arrow_down: | | [...rflow/contrib/operators/kubernetes\_pod\_operator.py](https://codecov.io/gh/apache/airflow/pull/6529/diff?src=pr=tree#diff-YWlyZmxvdy9jb250cmliL29wZXJhdG9ycy9rdWJlcm5ldGVzX3BvZF9vcGVyYXRvci5weQ==) | `70.14% <0%> (-28.36%)` | :arrow_down: | | [airflow/utils/dag\_processing.py](https://codecov.io/gh/apache/airflow/pull/6529/diff?src=pr=tree#diff-YWlyZmxvdy91dGlscy9kYWdfcHJvY2Vzc2luZy5weQ==) | `58.31% <0%> (+0.16%)` | :arrow_up: | | [airflow/jobs/local\_task\_job.py](https://codecov.io/gh/apache/airflow/pull/6529/diff?src=pr=tree#diff-YWlyZmxvdy9qb2JzL2xvY2FsX3Rhc2tfam9iLnB5) | `86.25% <0%> (+1.25%)` | :arrow_up: | -- [Continue to review full report at Codecov](https://codecov.io/gh/apache/airflow/pull/6529?src=pr=continue). > **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta) > `Δ = absolute (impact)`, `ø = not affected`, `? = missing data` > Powered by [Codecov](https://codecov.io/gh/apache/airflow/pull/6529?src=pr=footer). Last update [33ddcd9...f8512ff](https://codecov.io/gh/apache/airflow/pull/6529?src=pr=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments). This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [airflow-site] mik-laj opened a new issue #147: Words don't break in search results.
mik-laj opened a new issue #147: Words don't break in search results. URL: https://github.com/apache/airflow-site/issues/147  Reported by @aijamalnk This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Updated] (AIRFLOW-5881) Dag gets stuck in "Scheduled" State when scheduling a large number of tasks
[ https://issues.apache.org/jira/browse/AIRFLOW-5881?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] David Hartig updated AIRFLOW-5881: -- Description: Running with the KubernetesExecutor in and AKS cluster, when we upgraded to version 1.10.6 we noticed that the all the Dags stop making progress but start running and immediate exiting with the following message: "Instance State' FAILED: Task is in the 'scheduled' state which is not a valid state for execution. The task must be cleared in order to be run." See attached log file for the worker. Nothing seems out of the ordinary in the Scheduler log. Reverting to 1.10.5 clears the problem. Note that at the time of the failure maybe 100 or so tasks are in this state, with 70 coming from one highly parallelized dag. Clearing the scheduled tasks just makes them reappear shortly thereafter. Marking them "up_for_retry" results in one being executed but then the system is stuck in the original zombie state. Attached is the also a redacted airflow config flag. was: Running with the KubernetesExecutor in and AKS cluster, when we upgraded to version 1.10.6 we noticed that the all the Dags stop making progress but start running and immediate exiting with the following message: "Instance State' FAILED: Task is in the 'scheduled' state which is not a valid state for execution. The task must be cleared in order to be run." See attached log file for the worker. Nothing seems out of the ordinary in the Scheduler log. Reverting to 1.10.5 clears the problem. Note that at the time of the failure maybe 100 or so tasks are in this state, with 70 coming from one highly parallelized dag. Clearing the scheduled tasks just makes them reappear shortly thereafter. Marking them "up_for_retry" results in that one being executed but then the system is stuck in the original zombie state. Attached is the also a redacted airflow config flag. > Dag gets stuck in "Scheduled" State when scheduling a large number of tasks > --- > > Key: AIRFLOW-5881 > URL: https://issues.apache.org/jira/browse/AIRFLOW-5881 > Project: Apache Airflow > Issue Type: Bug > Components: scheduler >Affects Versions: 1.10.6 >Reporter: David Hartig >Priority: Critical > Attachments: 2 (1).log, airflow.cnf > > > Running with the KubernetesExecutor in and AKS cluster, when we upgraded to > version 1.10.6 we noticed that the all the Dags stop making progress but > start running and immediate exiting with the following message: > "Instance State' FAILED: Task is in the 'scheduled' state which is not a > valid state for execution. The task must be cleared in order to be run." > See attached log file for the worker. Nothing seems out of the ordinary in > the Scheduler log. > Reverting to 1.10.5 clears the problem. > Note that at the time of the failure maybe 100 or so tasks are in this state, > with 70 coming from one highly parallelized dag. Clearing the scheduled tasks > just makes them reappear shortly thereafter. Marking them "up_for_retry" > results in one being executed but then the system is stuck in the original > zombie state. > Attached is the also a redacted airflow config flag. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (AIRFLOW-5881) Dag gets stuck in "Scheduled" State when scheduling a large number of tasks
[ https://issues.apache.org/jira/browse/AIRFLOW-5881?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] David Hartig updated AIRFLOW-5881: -- Description: Running with the KubernetesExecutor in and AKS cluster, when we upgraded to version 1.10.6 we noticed that the all the Dags stop making progress but start running and immediate exiting with the following message: "Instance State' FAILED: Task is in the 'scheduled' state which is not a valid state for execution. The task must be cleared in order to be run." See attached log file for the worker. Nothing seems out of the ordinary in the Scheduler log. Reverting to 1.10.5 clears the problem. Note that at the time of the failure maybe 100 or so tasks are in this state, with 70 coming from one highly parallelized dag. Clearing the scheduled tasks just makes them reappear shortly thereafter. Marking them "up_for_retry" results in that one being executed but then the system is stuck in the original zombie state. Attached is the also a redacted airflow config flag. was: Running with the KubernetesExecutor in and AKS cluster, when we upgraded to version 1.10.6 we noticed that the all the Dags stop making progress but start running and immediate exiting with the following message: "Instance State' FAILED: Task is in the 'scheduled' state which is not a valid state for execution. The task must be cleared in order to be run." See attached log file for the worker. Nothing seems out of the ordinary in the Scheduler log. Reverting to 1.10.5 clears the problem. Note that at the time of the failure maybe 100 or so tasks are in this state, with 70 coming from one highly parallelized dag. Attached is the also a redacted airflow config flag. > Dag gets stuck in "Scheduled" State when scheduling a large number of tasks > --- > > Key: AIRFLOW-5881 > URL: https://issues.apache.org/jira/browse/AIRFLOW-5881 > Project: Apache Airflow > Issue Type: Bug > Components: scheduler >Affects Versions: 1.10.6 >Reporter: David Hartig >Priority: Critical > Attachments: 2 (1).log, airflow.cnf > > > Running with the KubernetesExecutor in and AKS cluster, when we upgraded to > version 1.10.6 we noticed that the all the Dags stop making progress but > start running and immediate exiting with the following message: > "Instance State' FAILED: Task is in the 'scheduled' state which is not a > valid state for execution. The task must be cleared in order to be run." > See attached log file for the worker. Nothing seems out of the ordinary in > the Scheduler log. > Reverting to 1.10.5 clears the problem. > Note that at the time of the failure maybe 100 or so tasks are in this state, > with 70 coming from one highly parallelized dag. Clearing the scheduled tasks > just makes them reappear shortly thereafter. Marking them "up_for_retry" > results in that one being executed but then the system is stuck in the > original zombie state. > Attached is the also a redacted airflow config flag. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[GitHub] [airflow] codecov-io edited a comment on issue #6520: [AIRFLOW-5870] Allow -1 for infinite pool size
codecov-io edited a comment on issue #6520: [AIRFLOW-5870] Allow -1 for infinite pool size URL: https://github.com/apache/airflow/pull/6520#issuecomment-552000288 # [Codecov](https://codecov.io/gh/apache/airflow/pull/6520?src=pr=h1) Report > Merging [#6520](https://codecov.io/gh/apache/airflow/pull/6520?src=pr=desc) into [master](https://codecov.io/gh/apache/airflow/commit/813cd431204b71f4e7c27f2bee9fafcc910da4d0?src=pr=desc) will **decrease** coverage by `0.18%`. > The diff coverage is `100%`. [](https://codecov.io/gh/apache/airflow/pull/6520?src=pr=tree) ```diff @@Coverage Diff @@ ## master#6520 +/- ## == - Coverage 83.95% 83.76% -0.19% == Files 635 635 Lines 3684936851 +2 == - Hits3093630868 -68 - Misses 5913 5983 +70 ``` | [Impacted Files](https://codecov.io/gh/apache/airflow/pull/6520?src=pr=tree) | Coverage Δ | | |---|---|---| | [airflow/models/pool.py](https://codecov.io/gh/apache/airflow/pull/6520/diff?src=pr=tree#diff-YWlyZmxvdy9tb2RlbHMvcG9vbC5weQ==) | `97.36% <100%> (+0.14%)` | :arrow_up: | | [airflow/kubernetes/volume\_mount.py](https://codecov.io/gh/apache/airflow/pull/6520/diff?src=pr=tree#diff-YWlyZmxvdy9rdWJlcm5ldGVzL3ZvbHVtZV9tb3VudC5weQ==) | `44.44% <0%> (-55.56%)` | :arrow_down: | | [airflow/kubernetes/volume.py](https://codecov.io/gh/apache/airflow/pull/6520/diff?src=pr=tree#diff-YWlyZmxvdy9rdWJlcm5ldGVzL3ZvbHVtZS5weQ==) | `52.94% <0%> (-47.06%)` | :arrow_down: | | [airflow/kubernetes/pod\_launcher.py](https://codecov.io/gh/apache/airflow/pull/6520/diff?src=pr=tree#diff-YWlyZmxvdy9rdWJlcm5ldGVzL3BvZF9sYXVuY2hlci5weQ==) | `45.25% <0%> (-46.72%)` | :arrow_down: | | [airflow/kubernetes/kube\_client.py](https://codecov.io/gh/apache/airflow/pull/6520/diff?src=pr=tree#diff-YWlyZmxvdy9rdWJlcm5ldGVzL2t1YmVfY2xpZW50LnB5) | `33.33% <0%> (-41.67%)` | :arrow_down: | | [...rflow/contrib/operators/kubernetes\_pod\_operator.py](https://codecov.io/gh/apache/airflow/pull/6520/diff?src=pr=tree#diff-YWlyZmxvdy9jb250cmliL29wZXJhdG9ycy9rdWJlcm5ldGVzX3BvZF9vcGVyYXRvci5weQ==) | `70.14% <0%> (-28.36%)` | :arrow_down: | | [airflow/jobs/backfill\_job.py](https://codecov.io/gh/apache/airflow/pull/6520/diff?src=pr=tree#diff-YWlyZmxvdy9qb2JzL2JhY2tmaWxsX2pvYi5weQ==) | `89.9% <0%> (-1.53%)` | :arrow_down: | | [airflow/models/taskinstance.py](https://codecov.io/gh/apache/airflow/pull/6520/diff?src=pr=tree#diff-YWlyZmxvdy9tb2RlbHMvdGFza2luc3RhbmNlLnB5) | `93.79% <0%> (+0.5%)` | :arrow_up: | | [airflow/jobs/scheduler\_job.py](https://codecov.io/gh/apache/airflow/pull/6520/diff?src=pr=tree#diff-YWlyZmxvdy9qb2JzL3NjaGVkdWxlcl9qb2IucHk=) | `74.96% <0%> (+1.19%)` | :arrow_up: | | [airflow/utils/dag\_processing.py](https://codecov.io/gh/apache/airflow/pull/6520/diff?src=pr=tree#diff-YWlyZmxvdy91dGlscy9kYWdfcHJvY2Vzc2luZy5weQ==) | `57.99% <0%> (+2.14%)` | :arrow_up: | | ... and [4 more](https://codecov.io/gh/apache/airflow/pull/6520/diff?src=pr=tree-more) | | -- [Continue to review full report at Codecov](https://codecov.io/gh/apache/airflow/pull/6520?src=pr=continue). > **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta) > `Δ = absolute (impact)`, `ø = not affected`, `? = missing data` > Powered by [Codecov](https://codecov.io/gh/apache/airflow/pull/6520?src=pr=footer). Last update [813cd43...2b774c7](https://codecov.io/gh/apache/airflow/pull/6520?src=pr=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments). This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [airflow] codecov-io commented on issue #6520: [AIRFLOW-5870] Allow -1 for infinite pool size
codecov-io commented on issue #6520: [AIRFLOW-5870] Allow -1 for infinite pool size URL: https://github.com/apache/airflow/pull/6520#issuecomment-552000288 # [Codecov](https://codecov.io/gh/apache/airflow/pull/6520?src=pr=h1) Report > Merging [#6520](https://codecov.io/gh/apache/airflow/pull/6520?src=pr=desc) into [master](https://codecov.io/gh/apache/airflow/commit/813cd431204b71f4e7c27f2bee9fafcc910da4d0?src=pr=desc) will **decrease** coverage by `0.18%`. > The diff coverage is `100%`. [](https://codecov.io/gh/apache/airflow/pull/6520?src=pr=tree) ```diff @@Coverage Diff @@ ## master#6520 +/- ## == - Coverage 83.95% 83.76% -0.19% == Files 635 635 Lines 3684936851 +2 == - Hits3093630868 -68 - Misses 5913 5983 +70 ``` | [Impacted Files](https://codecov.io/gh/apache/airflow/pull/6520?src=pr=tree) | Coverage Δ | | |---|---|---| | [airflow/models/pool.py](https://codecov.io/gh/apache/airflow/pull/6520/diff?src=pr=tree#diff-YWlyZmxvdy9tb2RlbHMvcG9vbC5weQ==) | `97.36% <100%> (+0.14%)` | :arrow_up: | | [airflow/kubernetes/volume\_mount.py](https://codecov.io/gh/apache/airflow/pull/6520/diff?src=pr=tree#diff-YWlyZmxvdy9rdWJlcm5ldGVzL3ZvbHVtZV9tb3VudC5weQ==) | `44.44% <0%> (-55.56%)` | :arrow_down: | | [airflow/kubernetes/volume.py](https://codecov.io/gh/apache/airflow/pull/6520/diff?src=pr=tree#diff-YWlyZmxvdy9rdWJlcm5ldGVzL3ZvbHVtZS5weQ==) | `52.94% <0%> (-47.06%)` | :arrow_down: | | [airflow/kubernetes/pod\_launcher.py](https://codecov.io/gh/apache/airflow/pull/6520/diff?src=pr=tree#diff-YWlyZmxvdy9rdWJlcm5ldGVzL3BvZF9sYXVuY2hlci5weQ==) | `45.25% <0%> (-46.72%)` | :arrow_down: | | [airflow/kubernetes/kube\_client.py](https://codecov.io/gh/apache/airflow/pull/6520/diff?src=pr=tree#diff-YWlyZmxvdy9rdWJlcm5ldGVzL2t1YmVfY2xpZW50LnB5) | `33.33% <0%> (-41.67%)` | :arrow_down: | | [...rflow/contrib/operators/kubernetes\_pod\_operator.py](https://codecov.io/gh/apache/airflow/pull/6520/diff?src=pr=tree#diff-YWlyZmxvdy9jb250cmliL29wZXJhdG9ycy9rdWJlcm5ldGVzX3BvZF9vcGVyYXRvci5weQ==) | `70.14% <0%> (-28.36%)` | :arrow_down: | | [airflow/jobs/backfill\_job.py](https://codecov.io/gh/apache/airflow/pull/6520/diff?src=pr=tree#diff-YWlyZmxvdy9qb2JzL2JhY2tmaWxsX2pvYi5weQ==) | `89.9% <0%> (-1.53%)` | :arrow_down: | | [airflow/models/taskinstance.py](https://codecov.io/gh/apache/airflow/pull/6520/diff?src=pr=tree#diff-YWlyZmxvdy9tb2RlbHMvdGFza2luc3RhbmNlLnB5) | `93.79% <0%> (+0.5%)` | :arrow_up: | | [airflow/jobs/scheduler\_job.py](https://codecov.io/gh/apache/airflow/pull/6520/diff?src=pr=tree#diff-YWlyZmxvdy9qb2JzL3NjaGVkdWxlcl9qb2IucHk=) | `74.96% <0%> (+1.19%)` | :arrow_up: | | [airflow/utils/dag\_processing.py](https://codecov.io/gh/apache/airflow/pull/6520/diff?src=pr=tree#diff-YWlyZmxvdy91dGlscy9kYWdfcHJvY2Vzc2luZy5weQ==) | `57.99% <0%> (+2.14%)` | :arrow_up: | | ... and [4 more](https://codecov.io/gh/apache/airflow/pull/6520/diff?src=pr=tree-more) | | -- [Continue to review full report at Codecov](https://codecov.io/gh/apache/airflow/pull/6520?src=pr=continue). > **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta) > `Δ = absolute (impact)`, `ø = not affected`, `? = missing data` > Powered by [Codecov](https://codecov.io/gh/apache/airflow/pull/6520?src=pr=footer). Last update [813cd43...2b774c7](https://codecov.io/gh/apache/airflow/pull/6520?src=pr=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments). This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [airflow] codecov-io edited a comment on issue #6520: [AIRFLOW-5870] Allow -1 for infinite pool size
codecov-io edited a comment on issue #6520: [AIRFLOW-5870] Allow -1 for infinite pool size URL: https://github.com/apache/airflow/pull/6520#issuecomment-552000288 # [Codecov](https://codecov.io/gh/apache/airflow/pull/6520?src=pr=h1) Report > Merging [#6520](https://codecov.io/gh/apache/airflow/pull/6520?src=pr=desc) into [master](https://codecov.io/gh/apache/airflow/commit/813cd431204b71f4e7c27f2bee9fafcc910da4d0?src=pr=desc) will **decrease** coverage by `0.18%`. > The diff coverage is `100%`. [](https://codecov.io/gh/apache/airflow/pull/6520?src=pr=tree) ```diff @@Coverage Diff @@ ## master#6520 +/- ## == - Coverage 83.95% 83.76% -0.19% == Files 635 635 Lines 3684936851 +2 == - Hits3093630868 -68 - Misses 5913 5983 +70 ``` | [Impacted Files](https://codecov.io/gh/apache/airflow/pull/6520?src=pr=tree) | Coverage Δ | | |---|---|---| | [airflow/models/pool.py](https://codecov.io/gh/apache/airflow/pull/6520/diff?src=pr=tree#diff-YWlyZmxvdy9tb2RlbHMvcG9vbC5weQ==) | `97.36% <100%> (+0.14%)` | :arrow_up: | | [airflow/kubernetes/volume\_mount.py](https://codecov.io/gh/apache/airflow/pull/6520/diff?src=pr=tree#diff-YWlyZmxvdy9rdWJlcm5ldGVzL3ZvbHVtZV9tb3VudC5weQ==) | `44.44% <0%> (-55.56%)` | :arrow_down: | | [airflow/kubernetes/volume.py](https://codecov.io/gh/apache/airflow/pull/6520/diff?src=pr=tree#diff-YWlyZmxvdy9rdWJlcm5ldGVzL3ZvbHVtZS5weQ==) | `52.94% <0%> (-47.06%)` | :arrow_down: | | [airflow/kubernetes/pod\_launcher.py](https://codecov.io/gh/apache/airflow/pull/6520/diff?src=pr=tree#diff-YWlyZmxvdy9rdWJlcm5ldGVzL3BvZF9sYXVuY2hlci5weQ==) | `45.25% <0%> (-46.72%)` | :arrow_down: | | [airflow/kubernetes/kube\_client.py](https://codecov.io/gh/apache/airflow/pull/6520/diff?src=pr=tree#diff-YWlyZmxvdy9rdWJlcm5ldGVzL2t1YmVfY2xpZW50LnB5) | `33.33% <0%> (-41.67%)` | :arrow_down: | | [...rflow/contrib/operators/kubernetes\_pod\_operator.py](https://codecov.io/gh/apache/airflow/pull/6520/diff?src=pr=tree#diff-YWlyZmxvdy9jb250cmliL29wZXJhdG9ycy9rdWJlcm5ldGVzX3BvZF9vcGVyYXRvci5weQ==) | `70.14% <0%> (-28.36%)` | :arrow_down: | | [airflow/jobs/backfill\_job.py](https://codecov.io/gh/apache/airflow/pull/6520/diff?src=pr=tree#diff-YWlyZmxvdy9qb2JzL2JhY2tmaWxsX2pvYi5weQ==) | `89.9% <0%> (-1.53%)` | :arrow_down: | | [airflow/models/taskinstance.py](https://codecov.io/gh/apache/airflow/pull/6520/diff?src=pr=tree#diff-YWlyZmxvdy9tb2RlbHMvdGFza2luc3RhbmNlLnB5) | `93.79% <0%> (+0.5%)` | :arrow_up: | | [airflow/jobs/scheduler\_job.py](https://codecov.io/gh/apache/airflow/pull/6520/diff?src=pr=tree#diff-YWlyZmxvdy9qb2JzL3NjaGVkdWxlcl9qb2IucHk=) | `74.96% <0%> (+1.19%)` | :arrow_up: | | [airflow/utils/dag\_processing.py](https://codecov.io/gh/apache/airflow/pull/6520/diff?src=pr=tree#diff-YWlyZmxvdy91dGlscy9kYWdfcHJvY2Vzc2luZy5weQ==) | `57.99% <0%> (+2.14%)` | :arrow_up: | | ... and [4 more](https://codecov.io/gh/apache/airflow/pull/6520/diff?src=pr=tree-more) | | -- [Continue to review full report at Codecov](https://codecov.io/gh/apache/airflow/pull/6520?src=pr=continue). > **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta) > `Δ = absolute (impact)`, `ø = not affected`, `? = missing data` > Powered by [Codecov](https://codecov.io/gh/apache/airflow/pull/6520?src=pr=footer). Last update [813cd43...2b774c7](https://codecov.io/gh/apache/airflow/pull/6520?src=pr=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments). This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [airflow] mattbowden-ookla opened a new pull request #6533: [AIRFLOW-XXX] Use psycopg2-binary for postgres operations
mattbowden-ookla opened a new pull request #6533: [AIRFLOW-XXX] Use
psycopg2-binary for postgres operations
URL: https://github.com/apache/airflow/pull/6533
Make sure you have checked _all_ steps below.
### Jira
- [ ] My PR addresses the following [Airflow
Jira](https://issues.apache.org/jira/browse/AIRFLOW/) issues and references
them in the PR title. For example, "\[AIRFLOW-XXX\] My Airflow PR"
- https://issues.apache.org/jira/browse/AIRFLOW-XXX
- In case you are fixing a typo in the documentation you can prepend your
commit with \[AIRFLOW-XXX\], code changes always need a Jira issue.
- In case you are proposing a fundamental code change, you need to create
an Airflow Improvement Proposal
([AIP](https://cwiki.apache.org/confluence/display/AIRFLOW/Airflow+Improvements+Proposals)).
- In case you are adding a dependency, check if the license complies with
the [ASF 3rd Party License
Policy](https://www.apache.org/legal/resolved.html#category-x).
### Description
- [X] Here are some details about my PR, including screenshots of any UI
changes:
This requirement was pinned as of #5036, as related to a breaking change
with respect to the package namespace. This PR modifies which package to refer
to - prior to 2.8 the package was a binary wheel, but with 2.8+ the appropriate
installation package is `psycopg2-binary`
### Explanation:
[Full explanation
here](http://initd.org/psycopg/articles/2018/02/08/psycopg-274-released/)
For most of psycopg2's history, they had distributed pre-compiled binaries
(or "wheels") which would get installed when you did `pip install psycopg2`
(rather than compiling the source code upon installation). At some point
psycopg2 found that there were some segfault issues that were appearing in the
pre-compiled versions. Granted, this could be fixed if you installed by
explicitly saying you wanted to compile from source rather than retrieve the
binary version, but the psycopg2 devs felt that this would be better addressed
by publishing a separate package that was the pre-compiled binary, while
keeping the original package as compile-from-source only. The upshot of this is
that everyone who was used to installing `psycopg2` and getting the
pre-compiled binaries was now having to compile the package on install unless
they moved over to `psycopg2-binary` instead. The packages are effectively the
same, only the pre-compiled versions are usually more optimized, run quicker on
the systems they're build for, and don't require compilation dependencies to
install. So, in accordance with the warning provided by psycopg2 (`UserWarning:
The psycopg2 wheel package will be renamed from release 2.8; in order to keep
installing from binary please use "pip install psycopg2-binary" instead.`),
I've swapping this over to psycopg2-binary.
I've also confirmed that `psycopg2-binary` is available from `2.7.4` onwards:
```bash
$ pip install psycopg2-binary==
Collecting psycopg2-binary==
ERROR: Could not find a version that satisfies the requirement
psycopg2-binary== (from versions: 2.7.4, 2.7.5, 2.7.6, 2.7.6.1, 2.7.7, 2.8,
2.8.1, 2.8.2, 2.8.3, 2.8.4)
ERROR: No matching distribution found for psycopg2-binary==
```
### Tests
- [X] My PR adds the following unit tests __OR__ does not need testing for
this extremely good reason:
No test changes should need to be made for this requirements change
### Commits
- [X] My commits all reference Jira issues in their subject lines, and I
have squashed multiple commits if they address the same issue. In addition, my
commits follow the guidelines from "[How to write a good git commit
message](http://chris.beams.io/posts/git-commit/)":
1. Subject is separated from body by a blank line
1. Subject is limited to 50 characters (not including Jira issue reference)
1. Subject does not end with a period
1. Subject uses the imperative mood ("add", not "adding")
1. Body wraps at 72 characters
1. Body explains "what" and "why", not "how"
### Documentation
- [ ] In case of new functionality, my PR adds documentation that describes
how to use it.
- All the public functions and the classes in the PR contain docstrings
that explain what it does
- If you implement backwards incompatible changes, please leave a note in
the [Updating.md](https://github.com/apache/airflow/blob/master/UPDATING.md) so
we can assign it to a appropriate release
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[jira] [Commented] (AIRFLOW-5882) State dependency should not be ignorable
[
https://issues.apache.org/jira/browse/AIRFLOW-5882?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16970599#comment-16970599
]
ASF GitHub Bot commented on AIRFLOW-5882:
-
saguziel commented on pull request #6531: [AIRFLOW-5882] ti_state_dep should
not be ignoreable
URL: https://github.com/apache/airflow/pull/6531
Make sure you have checked _all_ steps below.
### Jira
- [ ] My PR addresses the following [Airflow
Jira](https://issues.apache.org/jira/browse/AIRFLOW/) issues and references
them in the PR title. For example, "\[AIRFLOW-XXX\] My Airflow PR"
- https://issues.apache.org/jira/browse/AIRFLOW-5882
- In case you are fixing a typo in the documentation you can prepend your
commit with \[AIRFLOW-XXX\], code changes always need a Jira issue.
- In case you are proposing a fundamental code change, you need to create
an Airflow Improvement Proposal
([AIP](https://cwiki.apache.org/confluence/display/AIRFLOW/Airflow+Improvements+Proposals)).
- In case you are adding a dependency, check if the license complies with
the [ASF 3rd Party License
Policy](https://www.apache.org/legal/resolved.html#category-x).
### Description
- [ ] Here are some details about my PR, including screenshots of any UI
changes:
This ti_state dep should not be ignoreable as it would allow double-run in
the "ideal" situation and in the current situation it causes both tasks to die.
Due to celery visibility timeout, the task will always kill itself eventually
### Tests
- [ ] My PR adds the following unit tests __OR__ does not need testing for
this extremely good reason:
will fix existing unit tests if they exist
### Commits
- [ ] My commits all reference Jira issues in their subject lines, and I
have squashed multiple commits if they address the same issue. In addition, my
commits follow the guidelines from "[How to write a good git commit
message](http://chris.beams.io/posts/git-commit/)":
1. Subject is separated from body by a blank line
1. Subject is limited to 50 characters (not including Jira issue reference)
1. Subject does not end with a period
1. Subject uses the imperative mood ("add", not "adding")
1. Body wraps at 72 characters
1. Body explains "what" and "why", not "how"
### Documentation
- [ ] In case of new functionality, my PR adds documentation that describes
how to use it.
- All the public functions and the classes in the PR contain docstrings
that explain what it does
- If you implement backwards incompatible changes, please leave a note in
the [Updating.md](https://github.com/apache/airflow/blob/master/UPDATING.md) so
we can assign it to a appropriate release
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
> State dependency should not be ignorable
>
>
> Key: AIRFLOW-5882
> URL: https://issues.apache.org/jira/browse/AIRFLOW-5882
> Project: Apache Airflow
> Issue Type: Improvement
> Components: worker
>Affects Versions: 1.10.6
>Reporter: Alex Guziel
>Assignee: Alex Guziel
>Priority: Major
>
> This ti_state dep should not be ignoreable as it would allow double-run in
> the "ideal" situation and in the current situation it causes both tasks to
> die. Due to celery visibility timeout, the task will always kill itself
> eventually
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[GitHub] [airflow] saguziel opened a new pull request #6531: [AIRFLOW-5882] ti_state_dep should not be ignoreable
saguziel opened a new pull request #6531: [AIRFLOW-5882] ti_state_dep should
not be ignoreable
URL: https://github.com/apache/airflow/pull/6531
Make sure you have checked _all_ steps below.
### Jira
- [ ] My PR addresses the following [Airflow
Jira](https://issues.apache.org/jira/browse/AIRFLOW/) issues and references
them in the PR title. For example, "\[AIRFLOW-XXX\] My Airflow PR"
- https://issues.apache.org/jira/browse/AIRFLOW-5882
- In case you are fixing a typo in the documentation you can prepend your
commit with \[AIRFLOW-XXX\], code changes always need a Jira issue.
- In case you are proposing a fundamental code change, you need to create
an Airflow Improvement Proposal
([AIP](https://cwiki.apache.org/confluence/display/AIRFLOW/Airflow+Improvements+Proposals)).
- In case you are adding a dependency, check if the license complies with
the [ASF 3rd Party License
Policy](https://www.apache.org/legal/resolved.html#category-x).
### Description
- [ ] Here are some details about my PR, including screenshots of any UI
changes:
This ti_state dep should not be ignoreable as it would allow double-run in
the "ideal" situation and in the current situation it causes both tasks to die.
Due to celery visibility timeout, the task will always kill itself eventually
### Tests
- [ ] My PR adds the following unit tests __OR__ does not need testing for
this extremely good reason:
will fix existing unit tests if they exist
### Commits
- [ ] My commits all reference Jira issues in their subject lines, and I
have squashed multiple commits if they address the same issue. In addition, my
commits follow the guidelines from "[How to write a good git commit
message](http://chris.beams.io/posts/git-commit/)":
1. Subject is separated from body by a blank line
1. Subject is limited to 50 characters (not including Jira issue reference)
1. Subject does not end with a period
1. Subject uses the imperative mood ("add", not "adding")
1. Body wraps at 72 characters
1. Body explains "what" and "why", not "how"
### Documentation
- [ ] In case of new functionality, my PR adds documentation that describes
how to use it.
- All the public functions and the classes in the PR contain docstrings
that explain what it does
- If you implement backwards incompatible changes, please leave a note in
the [Updating.md](https://github.com/apache/airflow/blob/master/UPDATING.md) so
we can assign it to a appropriate release
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[jira] [Updated] (AIRFLOW-5882) Running dependency should not be ignorable
[ https://issues.apache.org/jira/browse/AIRFLOW-5882?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alex Guziel updated AIRFLOW-5882: - Summary: Running dependency should not be ignorable (was: State dependency should not be ignorable) > Running dependency should not be ignorable > -- > > Key: AIRFLOW-5882 > URL: https://issues.apache.org/jira/browse/AIRFLOW-5882 > Project: Apache Airflow > Issue Type: Improvement > Components: worker >Affects Versions: 1.10.6 >Reporter: Alex Guziel >Assignee: Alex Guziel >Priority: Major > > This ti_state dep should not be ignoreable as it would allow double-run in > the "ideal" situation and in the current situation it causes both tasks to > die. Due to celery visibility timeout, the task will always kill itself > eventually -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (AIRFLOW-5882) Running dependency should not be ignorable
[ https://issues.apache.org/jira/browse/AIRFLOW-5882?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alex Guziel updated AIRFLOW-5882: - Description: This ti_state dep should not be ignoreable if the task is in the running state as it would allow double-run in the "ideal" situation and in the current situation it causes both tasks to die. Due to celery visibility timeout, the task will always kill itself eventually (was: This ti_state dep should not be ignoreable as it would allow double-run in the "ideal" situation and in the current situation it causes both tasks to die. Due to celery visibility timeout, the task will always kill itself eventually) > Running dependency should not be ignorable > -- > > Key: AIRFLOW-5882 > URL: https://issues.apache.org/jira/browse/AIRFLOW-5882 > Project: Apache Airflow > Issue Type: Improvement > Components: worker >Affects Versions: 1.10.6 >Reporter: Alex Guziel >Assignee: Alex Guziel >Priority: Major > > This ti_state dep should not be ignoreable if the task is in the running > state as it would allow double-run in the "ideal" situation and in the > current situation it causes both tasks to die. Due to celery visibility > timeout, the task will always kill itself eventually -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (AIRFLOW-5883) Fix inefficient sqlalchemy .count() queries
Alex Guziel created AIRFLOW-5883: Summary: Fix inefficient sqlalchemy .count() queries Key: AIRFLOW-5883 URL: https://issues.apache.org/jira/browse/AIRFLOW-5883 Project: Apache Airflow Issue Type: Improvement Components: database Affects Versions: 1.10.6 Reporter: Alex Guziel Assignee: Alex Guziel SQLAlchemy .count() uses a subquery and is inefficient -- This message was sent by Atlassian Jira (v8.3.4#803005)
[GitHub] [airflow] saguziel opened a new pull request #6532: [Airflow-5883] Don't use .count() from sqlalchemy to count
saguziel opened a new pull request #6532: [Airflow-5883] Don't use .count()
from sqlalchemy to count
URL: https://github.com/apache/airflow/pull/6532
Make sure you have checked _all_ steps below.
### Jira
- [ ] My PR addresses the following [Airflow
Jira](https://issues.apache.org/jira/browse/AIRFLOW/) issues and references
them in the PR title. For example, "\[AIRFLOW-XXX\] My Airflow PR"
- https://issues.apache.org/jira/browse/AIRFLOW-5883
- In case you are fixing a typo in the documentation you can prepend your
commit with \[AIRFLOW-XXX\], code changes always need a Jira issue.
- In case you are proposing a fundamental code change, you need to create
an Airflow Improvement Proposal
([AIP](https://cwiki.apache.org/confluence/display/AIRFLOW/Airflow+Improvements+Proposals)).
- In case you are adding a dependency, check if the license complies with
the [ASF 3rd Party License
Policy](https://www.apache.org/legal/resolved.html#category-x).
### Description
- [ ] Here are some details about my PR, including screenshots of any UI
changes:
### Tests
- [ ] My PR adds the following unit tests __OR__ does not need testing for
this extremely good reason:
### Commits
- [ ] My commits all reference Jira issues in their subject lines, and I
have squashed multiple commits if they address the same issue. In addition, my
commits follow the guidelines from "[How to write a good git commit
message](http://chris.beams.io/posts/git-commit/)":
1. Subject is separated from body by a blank line
1. Subject is limited to 50 characters (not including Jira issue reference)
1. Subject does not end with a period
1. Subject uses the imperative mood ("add", not "adding")
1. Body wraps at 72 characters
1. Body explains "what" and "why", not "how"
### Documentation
- [ ] In case of new functionality, my PR adds documentation that describes
how to use it.
- All the public functions and the classes in the PR contain docstrings
that explain what it does
- If you implement backwards incompatible changes, please leave a note in
the [Updating.md](https://github.com/apache/airflow/blob/master/UPDATING.md) so
we can assign it to a appropriate release
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [airflow] codecov-io commented on issue #6399: [AIRFLOW-5730] Enable get_pandas_df on PinotDbApiHook
codecov-io commented on issue #6399: [AIRFLOW-5730] Enable get_pandas_df on PinotDbApiHook URL: https://github.com/apache/airflow/pull/6399#issuecomment-552018842 # [Codecov](https://codecov.io/gh/apache/airflow/pull/6399?src=pr=h1) Report > Merging [#6399](https://codecov.io/gh/apache/airflow/pull/6399?src=pr=desc) into [master](https://codecov.io/gh/apache/airflow/commit/813cd431204b71f4e7c27f2bee9fafcc910da4d0?src=pr=desc) will **decrease** coverage by `0.53%`. > The diff coverage is `n/a`. [](https://codecov.io/gh/apache/airflow/pull/6399?src=pr=tree) ```diff @@Coverage Diff @@ ## master#6399 +/- ## == - Coverage 84.08% 83.54% -0.54% == Files 635 635 Lines 3684936847 -2 == - Hits3098330783 -200 - Misses 5866 6064 +198 ``` | [Impacted Files](https://codecov.io/gh/apache/airflow/pull/6399?src=pr=tree) | Coverage Δ | | |---|---|---| | [airflow/contrib/hooks/pinot\_hook.py](https://codecov.io/gh/apache/airflow/pull/6399/diff?src=pr=tree#diff-YWlyZmxvdy9jb250cmliL2hvb2tzL3Bpbm90X2hvb2sucHk=) | `91.6% <ø> (+0.62%)` | :arrow_up: | | [airflow/operators/mysql\_operator.py](https://codecov.io/gh/apache/airflow/pull/6399/diff?src=pr=tree#diff-YWlyZmxvdy9vcGVyYXRvcnMvbXlzcWxfb3BlcmF0b3IucHk=) | `0% <0%> (-100%)` | :arrow_down: | | [airflow/operators/mysql\_to\_hive.py](https://codecov.io/gh/apache/airflow/pull/6399/diff?src=pr=tree#diff-YWlyZmxvdy9vcGVyYXRvcnMvbXlzcWxfdG9faGl2ZS5weQ==) | `0% <0%> (-100%)` | :arrow_down: | | [airflow/kubernetes/volume\_mount.py](https://codecov.io/gh/apache/airflow/pull/6399/diff?src=pr=tree#diff-YWlyZmxvdy9rdWJlcm5ldGVzL3ZvbHVtZV9tb3VudC5weQ==) | `44.44% <0%> (-55.56%)` | :arrow_down: | | [airflow/kubernetes/volume.py](https://codecov.io/gh/apache/airflow/pull/6399/diff?src=pr=tree#diff-YWlyZmxvdy9rdWJlcm5ldGVzL3ZvbHVtZS5weQ==) | `52.94% <0%> (-47.06%)` | :arrow_down: | | [airflow/kubernetes/pod\_launcher.py](https://codecov.io/gh/apache/airflow/pull/6399/diff?src=pr=tree#diff-YWlyZmxvdy9rdWJlcm5ldGVzL3BvZF9sYXVuY2hlci5weQ==) | `45.25% <0%> (-46.72%)` | :arrow_down: | | [airflow/kubernetes/kube\_client.py](https://codecov.io/gh/apache/airflow/pull/6399/diff?src=pr=tree#diff-YWlyZmxvdy9rdWJlcm5ldGVzL2t1YmVfY2xpZW50LnB5) | `33.33% <0%> (-41.67%)` | :arrow_down: | | [...rflow/contrib/operators/kubernetes\_pod\_operator.py](https://codecov.io/gh/apache/airflow/pull/6399/diff?src=pr=tree#diff-YWlyZmxvdy9jb250cmliL29wZXJhdG9ycy9rdWJlcm5ldGVzX3BvZF9vcGVyYXRvci5weQ==) | `70.14% <0%> (-28.36%)` | :arrow_down: | | [airflow/utils/sqlalchemy.py](https://codecov.io/gh/apache/airflow/pull/6399/diff?src=pr=tree#diff-YWlyZmxvdy91dGlscy9zcWxhbGNoZW15LnB5) | `86.44% <0%> (-6.78%)` | :arrow_down: | | [airflow/jobs/local\_task\_job.py](https://codecov.io/gh/apache/airflow/pull/6399/diff?src=pr=tree#diff-YWlyZmxvdy9qb2JzL2xvY2FsX3Rhc2tfam9iLnB5) | `85% <0%> (-5%)` | :arrow_down: | | ... and [4 more](https://codecov.io/gh/apache/airflow/pull/6399/diff?src=pr=tree-more) | | -- [Continue to review full report at Codecov](https://codecov.io/gh/apache/airflow/pull/6399?src=pr=continue). > **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta) > `Δ = absolute (impact)`, `ø = not affected`, `? = missing data` > Powered by [Codecov](https://codecov.io/gh/apache/airflow/pull/6399?src=pr=footer). Last update [813cd43...f5973d8](https://codecov.io/gh/apache/airflow/pull/6399?src=pr=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments). This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [airflow] codecov-io edited a comment on issue #6533: [AIRFLOW-XXX] Use psycopg2-binary for postgres operations
codecov-io edited a comment on issue #6533: [AIRFLOW-XXX] Use psycopg2-binary for postgres operations URL: https://github.com/apache/airflow/pull/6533#issuecomment-552062056 # [Codecov](https://codecov.io/gh/apache/airflow/pull/6533?src=pr=h1) Report > Merging [#6533](https://codecov.io/gh/apache/airflow/pull/6533?src=pr=desc) into [master](https://codecov.io/gh/apache/airflow/commit/813cd431204b71f4e7c27f2bee9fafcc910da4d0?src=pr=desc) will **decrease** coverage by `0.3%`. > The diff coverage is `n/a`. [](https://codecov.io/gh/apache/airflow/pull/6533?src=pr=tree) ```diff @@Coverage Diff @@ ## master#6533 +/- ## == - Coverage 84.08% 83.77% -0.31% == Files 635 635 Lines 3684936849 == - Hits3098330871 -112 - Misses 5866 5978 +112 ``` | [Impacted Files](https://codecov.io/gh/apache/airflow/pull/6533?src=pr=tree) | Coverage Δ | | |---|---|---| | [airflow/kubernetes/volume\_mount.py](https://codecov.io/gh/apache/airflow/pull/6533/diff?src=pr=tree#diff-YWlyZmxvdy9rdWJlcm5ldGVzL3ZvbHVtZV9tb3VudC5weQ==) | `44.44% <0%> (-55.56%)` | :arrow_down: | | [airflow/kubernetes/volume.py](https://codecov.io/gh/apache/airflow/pull/6533/diff?src=pr=tree#diff-YWlyZmxvdy9rdWJlcm5ldGVzL3ZvbHVtZS5weQ==) | `52.94% <0%> (-47.06%)` | :arrow_down: | | [airflow/kubernetes/pod\_launcher.py](https://codecov.io/gh/apache/airflow/pull/6533/diff?src=pr=tree#diff-YWlyZmxvdy9rdWJlcm5ldGVzL3BvZF9sYXVuY2hlci5weQ==) | `45.25% <0%> (-46.72%)` | :arrow_down: | | [airflow/kubernetes/kube\_client.py](https://codecov.io/gh/apache/airflow/pull/6533/diff?src=pr=tree#diff-YWlyZmxvdy9rdWJlcm5ldGVzL2t1YmVfY2xpZW50LnB5) | `33.33% <0%> (-41.67%)` | :arrow_down: | | [...rflow/contrib/operators/kubernetes\_pod\_operator.py](https://codecov.io/gh/apache/airflow/pull/6533/diff?src=pr=tree#diff-YWlyZmxvdy9jb250cmliL29wZXJhdG9ycy9rdWJlcm5ldGVzX3BvZF9vcGVyYXRvci5weQ==) | `70.14% <0%> (-28.36%)` | :arrow_down: | | [airflow/utils/dag\_processing.py](https://codecov.io/gh/apache/airflow/pull/6533/diff?src=pr=tree#diff-YWlyZmxvdy91dGlscy9kYWdfcHJvY2Vzc2luZy5weQ==) | `57.99% <0%> (-0.17%)` | :arrow_down: | -- [Continue to review full report at Codecov](https://codecov.io/gh/apache/airflow/pull/6533?src=pr=continue). > **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta) > `Δ = absolute (impact)`, `ø = not affected`, `? = missing data` > Powered by [Codecov](https://codecov.io/gh/apache/airflow/pull/6533?src=pr=footer). Last update [813cd43...5b03903](https://codecov.io/gh/apache/airflow/pull/6533?src=pr=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments). This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [airflow] codecov-io commented on issue #6533: [AIRFLOW-XXX] Use psycopg2-binary for postgres operations
codecov-io commented on issue #6533: [AIRFLOW-XXX] Use psycopg2-binary for postgres operations URL: https://github.com/apache/airflow/pull/6533#issuecomment-552062056 # [Codecov](https://codecov.io/gh/apache/airflow/pull/6533?src=pr=h1) Report > Merging [#6533](https://codecov.io/gh/apache/airflow/pull/6533?src=pr=desc) into [master](https://codecov.io/gh/apache/airflow/commit/813cd431204b71f4e7c27f2bee9fafcc910da4d0?src=pr=desc) will **decrease** coverage by `0.3%`. > The diff coverage is `n/a`. [](https://codecov.io/gh/apache/airflow/pull/6533?src=pr=tree) ```diff @@Coverage Diff @@ ## master#6533 +/- ## == - Coverage 84.08% 83.77% -0.31% == Files 635 635 Lines 3684936849 == - Hits3098330871 -112 - Misses 5866 5978 +112 ``` | [Impacted Files](https://codecov.io/gh/apache/airflow/pull/6533?src=pr=tree) | Coverage Δ | | |---|---|---| | [airflow/kubernetes/volume\_mount.py](https://codecov.io/gh/apache/airflow/pull/6533/diff?src=pr=tree#diff-YWlyZmxvdy9rdWJlcm5ldGVzL3ZvbHVtZV9tb3VudC5weQ==) | `44.44% <0%> (-55.56%)` | :arrow_down: | | [airflow/kubernetes/volume.py](https://codecov.io/gh/apache/airflow/pull/6533/diff?src=pr=tree#diff-YWlyZmxvdy9rdWJlcm5ldGVzL3ZvbHVtZS5weQ==) | `52.94% <0%> (-47.06%)` | :arrow_down: | | [airflow/kubernetes/pod\_launcher.py](https://codecov.io/gh/apache/airflow/pull/6533/diff?src=pr=tree#diff-YWlyZmxvdy9rdWJlcm5ldGVzL3BvZF9sYXVuY2hlci5weQ==) | `45.25% <0%> (-46.72%)` | :arrow_down: | | [airflow/kubernetes/kube\_client.py](https://codecov.io/gh/apache/airflow/pull/6533/diff?src=pr=tree#diff-YWlyZmxvdy9rdWJlcm5ldGVzL2t1YmVfY2xpZW50LnB5) | `33.33% <0%> (-41.67%)` | :arrow_down: | | [...rflow/contrib/operators/kubernetes\_pod\_operator.py](https://codecov.io/gh/apache/airflow/pull/6533/diff?src=pr=tree#diff-YWlyZmxvdy9jb250cmliL29wZXJhdG9ycy9rdWJlcm5ldGVzX3BvZF9vcGVyYXRvci5weQ==) | `70.14% <0%> (-28.36%)` | :arrow_down: | | [airflow/utils/dag\_processing.py](https://codecov.io/gh/apache/airflow/pull/6533/diff?src=pr=tree#diff-YWlyZmxvdy91dGlscy9kYWdfcHJvY2Vzc2luZy5weQ==) | `57.99% <0%> (-0.17%)` | :arrow_down: | -- [Continue to review full report at Codecov](https://codecov.io/gh/apache/airflow/pull/6533?src=pr=continue). > **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta) > `Δ = absolute (impact)`, `ø = not affected`, `? = missing data` > Powered by [Codecov](https://codecov.io/gh/apache/airflow/pull/6533?src=pr=footer). Last update [813cd43...5b03903](https://codecov.io/gh/apache/airflow/pull/6533?src=pr=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments). This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [airflow] codecov-io edited a comment on issue #6533: [AIRFLOW-XXX] Use psycopg2-binary for postgres operations
codecov-io edited a comment on issue #6533: [AIRFLOW-XXX] Use psycopg2-binary for postgres operations URL: https://github.com/apache/airflow/pull/6533#issuecomment-552062056 # [Codecov](https://codecov.io/gh/apache/airflow/pull/6533?src=pr=h1) Report > Merging [#6533](https://codecov.io/gh/apache/airflow/pull/6533?src=pr=desc) into [master](https://codecov.io/gh/apache/airflow/commit/813cd431204b71f4e7c27f2bee9fafcc910da4d0?src=pr=desc) will **decrease** coverage by `0.3%`. > The diff coverage is `n/a`. [](https://codecov.io/gh/apache/airflow/pull/6533?src=pr=tree) ```diff @@Coverage Diff @@ ## master#6533 +/- ## == - Coverage 84.08% 83.77% -0.31% == Files 635 635 Lines 3684936849 == - Hits3098330871 -112 - Misses 5866 5978 +112 ``` | [Impacted Files](https://codecov.io/gh/apache/airflow/pull/6533?src=pr=tree) | Coverage Δ | | |---|---|---| | [airflow/kubernetes/volume\_mount.py](https://codecov.io/gh/apache/airflow/pull/6533/diff?src=pr=tree#diff-YWlyZmxvdy9rdWJlcm5ldGVzL3ZvbHVtZV9tb3VudC5weQ==) | `44.44% <0%> (-55.56%)` | :arrow_down: | | [airflow/kubernetes/volume.py](https://codecov.io/gh/apache/airflow/pull/6533/diff?src=pr=tree#diff-YWlyZmxvdy9rdWJlcm5ldGVzL3ZvbHVtZS5weQ==) | `52.94% <0%> (-47.06%)` | :arrow_down: | | [airflow/kubernetes/pod\_launcher.py](https://codecov.io/gh/apache/airflow/pull/6533/diff?src=pr=tree#diff-YWlyZmxvdy9rdWJlcm5ldGVzL3BvZF9sYXVuY2hlci5weQ==) | `45.25% <0%> (-46.72%)` | :arrow_down: | | [airflow/kubernetes/kube\_client.py](https://codecov.io/gh/apache/airflow/pull/6533/diff?src=pr=tree#diff-YWlyZmxvdy9rdWJlcm5ldGVzL2t1YmVfY2xpZW50LnB5) | `33.33% <0%> (-41.67%)` | :arrow_down: | | [...rflow/contrib/operators/kubernetes\_pod\_operator.py](https://codecov.io/gh/apache/airflow/pull/6533/diff?src=pr=tree#diff-YWlyZmxvdy9jb250cmliL29wZXJhdG9ycy9rdWJlcm5ldGVzX3BvZF9vcGVyYXRvci5weQ==) | `70.14% <0%> (-28.36%)` | :arrow_down: | | [airflow/utils/dag\_processing.py](https://codecov.io/gh/apache/airflow/pull/6533/diff?src=pr=tree#diff-YWlyZmxvdy91dGlscy9kYWdfcHJvY2Vzc2luZy5weQ==) | `57.99% <0%> (-0.17%)` | :arrow_down: | -- [Continue to review full report at Codecov](https://codecov.io/gh/apache/airflow/pull/6533?src=pr=continue). > **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta) > `Δ = absolute (impact)`, `ø = not affected`, `? = missing data` > Powered by [Codecov](https://codecov.io/gh/apache/airflow/pull/6533?src=pr=footer). Last update [813cd43...5b03903](https://codecov.io/gh/apache/airflow/pull/6533?src=pr=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments). This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [airflow] KevinYang21 commented on issue #6533: [AIRFLOW-XXX] Use psycopg2-binary for postgres operations
KevinYang21 commented on issue #6533: [AIRFLOW-XXX] Use psycopg2-binary for postgres operations URL: https://github.com/apache/airflow/pull/6533#issuecomment-552066768 This is not a doc only change, we should create a corresponding JIRA ticket for it. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [airflow] sekikn commented on issue #6399: [AIRFLOW-5730] Enable get_pandas_df on PinotDbApiHook
sekikn commented on issue #6399: [AIRFLOW-5730] Enable get_pandas_df on PinotDbApiHook URL: https://github.com/apache/airflow/pull/6399#issuecomment-551998445 Thanks @OmerJog for taking care of this PR, rebased on master to run the CI again :) This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [airflow] codecov-io commented on issue #6457: [AIRFLOW-5800] Add a default connection entry for PinotDbApiHook
codecov-io commented on issue #6457: [AIRFLOW-5800] Add a default connection entry for PinotDbApiHook URL: https://github.com/apache/airflow/pull/6457#issuecomment-552073991 # [Codecov](https://codecov.io/gh/apache/airflow/pull/6457?src=pr=h1) Report > Merging [#6457](https://codecov.io/gh/apache/airflow/pull/6457?src=pr=desc) into [master](https://codecov.io/gh/apache/airflow/commit/813cd431204b71f4e7c27f2bee9fafcc910da4d0?src=pr=desc) will **decrease** coverage by `0.76%`. > The diff coverage is `100%`. [](https://codecov.io/gh/apache/airflow/pull/6457?src=pr=tree) ```diff @@Coverage Diff @@ ## master#6457 +/- ## == - Coverage 84.08% 83.31% -0.77% == Files 635 631 -4 Lines 3684936639 -210 == - Hits3098330527 -456 - Misses 5866 6112 +246 ``` | [Impacted Files](https://codecov.io/gh/apache/airflow/pull/6457?src=pr=tree) | Coverage Δ | | |---|---|---| | [airflow/utils/db.py](https://codecov.io/gh/apache/airflow/pull/6457/diff?src=pr=tree#diff-YWlyZmxvdy91dGlscy9kYi5weQ==) | `90.17% <100%> (ø)` | :arrow_up: | | [airflow/operators/mysql\_operator.py](https://codecov.io/gh/apache/airflow/pull/6457/diff?src=pr=tree#diff-YWlyZmxvdy9vcGVyYXRvcnMvbXlzcWxfb3BlcmF0b3IucHk=) | `0% <0%> (-100%)` | :arrow_down: | | [airflow/operators/mysql\_to\_hive.py](https://codecov.io/gh/apache/airflow/pull/6457/diff?src=pr=tree#diff-YWlyZmxvdy9vcGVyYXRvcnMvbXlzcWxfdG9faGl2ZS5weQ==) | `0% <0%> (-100%)` | :arrow_down: | | [airflow/kubernetes/volume\_mount.py](https://codecov.io/gh/apache/airflow/pull/6457/diff?src=pr=tree#diff-YWlyZmxvdy9rdWJlcm5ldGVzL3ZvbHVtZV9tb3VudC5weQ==) | `44.44% <0%> (-55.56%)` | :arrow_down: | | [airflow/executors/sequential\_executor.py](https://codecov.io/gh/apache/airflow/pull/6457/diff?src=pr=tree#diff-YWlyZmxvdy9leGVjdXRvcnMvc2VxdWVudGlhbF9leGVjdXRvci5weQ==) | `47.61% <0%> (-52.39%)` | :arrow_down: | | [...ow/contrib/example\_dags/example\_qubole\_operator.py](https://codecov.io/gh/apache/airflow/pull/6457/diff?src=pr=tree#diff-YWlyZmxvdy9jb250cmliL2V4YW1wbGVfZGFncy9leGFtcGxlX3F1Ym9sZV9vcGVyYXRvci5weQ==) | `32.5% <0%> (-47.5%)` | :arrow_down: | | [airflow/kubernetes/volume.py](https://codecov.io/gh/apache/airflow/pull/6457/diff?src=pr=tree#diff-YWlyZmxvdy9rdWJlcm5ldGVzL3ZvbHVtZS5weQ==) | `52.94% <0%> (-47.06%)` | :arrow_down: | | [airflow/kubernetes/pod\_launcher.py](https://codecov.io/gh/apache/airflow/pull/6457/diff?src=pr=tree#diff-YWlyZmxvdy9rdWJlcm5ldGVzL3BvZF9sYXVuY2hlci5weQ==) | `45.25% <0%> (-46.72%)` | :arrow_down: | | [airflow/kubernetes/kube\_client.py](https://codecov.io/gh/apache/airflow/pull/6457/diff?src=pr=tree#diff-YWlyZmxvdy9rdWJlcm5ldGVzL2t1YmVfY2xpZW50LnB5) | `33.33% <0%> (-41.67%)` | :arrow_down: | | [airflow/hooks/presto\_hook.py](https://codecov.io/gh/apache/airflow/pull/6457/diff?src=pr=tree#diff-YWlyZmxvdy9ob29rcy9wcmVzdG9faG9vay5weQ==) | `36.73% <0%> (-40.82%)` | :arrow_down: | | ... and [63 more](https://codecov.io/gh/apache/airflow/pull/6457/diff?src=pr=tree-more) | | -- [Continue to review full report at Codecov](https://codecov.io/gh/apache/airflow/pull/6457?src=pr=continue). > **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta) > `Δ = absolute (impact)`, `ø = not affected`, `? = missing data` > Powered by [Codecov](https://codecov.io/gh/apache/airflow/pull/6457?src=pr=footer). Last update [813cd43...0cb9d2d](https://codecov.io/gh/apache/airflow/pull/6457?src=pr=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments). This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [airflow] codecov-io edited a comment on issue #6399: [AIRFLOW-5730] Enable get_pandas_df on PinotDbApiHook
codecov-io edited a comment on issue #6399: [AIRFLOW-5730] Enable get_pandas_df on PinotDbApiHook URL: https://github.com/apache/airflow/pull/6399#issuecomment-552018842 # [Codecov](https://codecov.io/gh/apache/airflow/pull/6399?src=pr=h1) Report > Merging [#6399](https://codecov.io/gh/apache/airflow/pull/6399?src=pr=desc) into [master](https://codecov.io/gh/apache/airflow/commit/813cd431204b71f4e7c27f2bee9fafcc910da4d0?src=pr=desc) will **decrease** coverage by `0.3%`. > The diff coverage is `n/a`. [](https://codecov.io/gh/apache/airflow/pull/6399?src=pr=tree) ```diff @@Coverage Diff @@ ## master#6399 +/- ## == - Coverage 84.08% 83.77% -0.31% == Files 635 635 Lines 3684936847 -2 == - Hits3098330868 -115 - Misses 5866 5979 +113 ``` | [Impacted Files](https://codecov.io/gh/apache/airflow/pull/6399?src=pr=tree) | Coverage Δ | | |---|---|---| | [airflow/contrib/hooks/pinot\_hook.py](https://codecov.io/gh/apache/airflow/pull/6399/diff?src=pr=tree#diff-YWlyZmxvdy9jb250cmliL2hvb2tzL3Bpbm90X2hvb2sucHk=) | `91.6% <ø> (+0.62%)` | :arrow_up: | | [airflow/operators/mysql\_operator.py](https://codecov.io/gh/apache/airflow/pull/6399/diff?src=pr=tree#diff-YWlyZmxvdy9vcGVyYXRvcnMvbXlzcWxfb3BlcmF0b3IucHk=) | `100% <0%> (ø)` | :arrow_up: | | [airflow/operators/mysql\_to\_hive.py](https://codecov.io/gh/apache/airflow/pull/6399/diff?src=pr=tree#diff-YWlyZmxvdy9vcGVyYXRvcnMvbXlzcWxfdG9faGl2ZS5weQ==) | `100% <0%> (ø)` | :arrow_up: | | [airflow/kubernetes/volume\_mount.py](https://codecov.io/gh/apache/airflow/pull/6399/diff?src=pr=tree#diff-YWlyZmxvdy9rdWJlcm5ldGVzL3ZvbHVtZV9tb3VudC5weQ==) | `44.44% <0%> (-55.56%)` | :arrow_down: | | [airflow/kubernetes/volume.py](https://codecov.io/gh/apache/airflow/pull/6399/diff?src=pr=tree#diff-YWlyZmxvdy9rdWJlcm5ldGVzL3ZvbHVtZS5weQ==) | `52.94% <0%> (-47.06%)` | :arrow_down: | | [airflow/kubernetes/pod\_launcher.py](https://codecov.io/gh/apache/airflow/pull/6399/diff?src=pr=tree#diff-YWlyZmxvdy9rdWJlcm5ldGVzL3BvZF9sYXVuY2hlci5weQ==) | `45.25% <0%> (-46.72%)` | :arrow_down: | | [airflow/kubernetes/kube\_client.py](https://codecov.io/gh/apache/airflow/pull/6399/diff?src=pr=tree#diff-YWlyZmxvdy9rdWJlcm5ldGVzL2t1YmVfY2xpZW50LnB5) | `33.33% <0%> (-41.67%)` | :arrow_down: | | [...rflow/contrib/operators/kubernetes\_pod\_operator.py](https://codecov.io/gh/apache/airflow/pull/6399/diff?src=pr=tree#diff-YWlyZmxvdy9jb250cmliL29wZXJhdG9ycy9rdWJlcm5ldGVzX3BvZF9vcGVyYXRvci5weQ==) | `70.14% <0%> (-28.36%)` | :arrow_down: | | [airflow/utils/sqlalchemy.py](https://codecov.io/gh/apache/airflow/pull/6399/diff?src=pr=tree#diff-YWlyZmxvdy91dGlscy9zcWxhbGNoZW15LnB5) | `93.22% <0%> (ø)` | :arrow_up: | | [airflow/jobs/local\_task\_job.py](https://codecov.io/gh/apache/airflow/pull/6399/diff?src=pr=tree#diff-YWlyZmxvdy9qb2JzL2xvY2FsX3Rhc2tfam9iLnB5) | `85% <0%> (-5%)` | :arrow_down: | | ... and [4 more](https://codecov.io/gh/apache/airflow/pull/6399/diff?src=pr=tree-more) | | -- [Continue to review full report at Codecov](https://codecov.io/gh/apache/airflow/pull/6399?src=pr=continue). > **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta) > `Δ = absolute (impact)`, `ø = not affected`, `? = missing data` > Powered by [Codecov](https://codecov.io/gh/apache/airflow/pull/6399?src=pr=footer). Last update [813cd43...f5973d8](https://codecov.io/gh/apache/airflow/pull/6399?src=pr=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments). This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [airflow] codecov-io edited a comment on issue #6520: [AIRFLOW-5870] Allow -1 for infinite pool size
codecov-io edited a comment on issue #6520: [AIRFLOW-5870] Allow -1 for infinite pool size URL: https://github.com/apache/airflow/pull/6520#issuecomment-552000288 # [Codecov](https://codecov.io/gh/apache/airflow/pull/6520?src=pr=h1) Report > Merging [#6520](https://codecov.io/gh/apache/airflow/pull/6520?src=pr=desc) into [master](https://codecov.io/gh/apache/airflow/commit/813cd431204b71f4e7c27f2bee9fafcc910da4d0?src=pr=desc) will **decrease** coverage by `0.54%`. > The diff coverage is `100%`. [](https://codecov.io/gh/apache/airflow/pull/6520?src=pr=tree) ```diff @@Coverage Diff @@ ## master#6520 +/- ## == - Coverage 84.08% 83.53% -0.56% == Files 635 635 Lines 3684936851 +2 == - Hits3098330782 -201 - Misses 5866 6069 +203 ``` | [Impacted Files](https://codecov.io/gh/apache/airflow/pull/6520?src=pr=tree) | Coverage Δ | | |---|---|---| | [airflow/models/pool.py](https://codecov.io/gh/apache/airflow/pull/6520/diff?src=pr=tree#diff-YWlyZmxvdy9tb2RlbHMvcG9vbC5weQ==) | `97.36% <100%> (+0.14%)` | :arrow_up: | | [airflow/operators/mysql\_operator.py](https://codecov.io/gh/apache/airflow/pull/6520/diff?src=pr=tree#diff-YWlyZmxvdy9vcGVyYXRvcnMvbXlzcWxfb3BlcmF0b3IucHk=) | `0% <0%> (-100%)` | :arrow_down: | | [airflow/operators/mysql\_to\_hive.py](https://codecov.io/gh/apache/airflow/pull/6520/diff?src=pr=tree#diff-YWlyZmxvdy9vcGVyYXRvcnMvbXlzcWxfdG9faGl2ZS5weQ==) | `0% <0%> (-100%)` | :arrow_down: | | [airflow/kubernetes/volume\_mount.py](https://codecov.io/gh/apache/airflow/pull/6520/diff?src=pr=tree#diff-YWlyZmxvdy9rdWJlcm5ldGVzL3ZvbHVtZV9tb3VudC5weQ==) | `44.44% <0%> (-55.56%)` | :arrow_down: | | [airflow/kubernetes/volume.py](https://codecov.io/gh/apache/airflow/pull/6520/diff?src=pr=tree#diff-YWlyZmxvdy9rdWJlcm5ldGVzL3ZvbHVtZS5weQ==) | `52.94% <0%> (-47.06%)` | :arrow_down: | | [airflow/kubernetes/pod\_launcher.py](https://codecov.io/gh/apache/airflow/pull/6520/diff?src=pr=tree#diff-YWlyZmxvdy9rdWJlcm5ldGVzL3BvZF9sYXVuY2hlci5weQ==) | `45.25% <0%> (-46.72%)` | :arrow_down: | | [airflow/kubernetes/kube\_client.py](https://codecov.io/gh/apache/airflow/pull/6520/diff?src=pr=tree#diff-YWlyZmxvdy9rdWJlcm5ldGVzL2t1YmVfY2xpZW50LnB5) | `33.33% <0%> (-41.67%)` | :arrow_down: | | [...rflow/contrib/operators/kubernetes\_pod\_operator.py](https://codecov.io/gh/apache/airflow/pull/6520/diff?src=pr=tree#diff-YWlyZmxvdy9jb250cmliL29wZXJhdG9ycy9rdWJlcm5ldGVzX3BvZF9vcGVyYXRvci5weQ==) | `70.14% <0%> (-28.36%)` | :arrow_down: | | [airflow/utils/sqlalchemy.py](https://codecov.io/gh/apache/airflow/pull/6520/diff?src=pr=tree#diff-YWlyZmxvdy91dGlscy9zcWxhbGNoZW15LnB5) | `86.44% <0%> (-6.78%)` | :arrow_down: | | [airflow/jobs/local\_task\_job.py](https://codecov.io/gh/apache/airflow/pull/6520/diff?src=pr=tree#diff-YWlyZmxvdy9qb2JzL2xvY2FsX3Rhc2tfam9iLnB5) | `85% <0%> (-5%)` | :arrow_down: | | ... and [4 more](https://codecov.io/gh/apache/airflow/pull/6520/diff?src=pr=tree-more) | | -- [Continue to review full report at Codecov](https://codecov.io/gh/apache/airflow/pull/6520?src=pr=continue). > **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta) > `Δ = absolute (impact)`, `ø = not affected`, `? = missing data` > Powered by [Codecov](https://codecov.io/gh/apache/airflow/pull/6520?src=pr=footer). Last update [813cd43...5ceb0ff](https://codecov.io/gh/apache/airflow/pull/6520?src=pr=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments). This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [airflow] codecov-io edited a comment on issue #6520: [AIRFLOW-5870] Allow -1 for infinite pool size
codecov-io edited a comment on issue #6520: [AIRFLOW-5870] Allow -1 for infinite pool size URL: https://github.com/apache/airflow/pull/6520#issuecomment-552000288 # [Codecov](https://codecov.io/gh/apache/airflow/pull/6520?src=pr=h1) Report > Merging [#6520](https://codecov.io/gh/apache/airflow/pull/6520?src=pr=desc) into [master](https://codecov.io/gh/apache/airflow/commit/813cd431204b71f4e7c27f2bee9fafcc910da4d0?src=pr=desc) will **decrease** coverage by `0.31%`. > The diff coverage is `100%`. [](https://codecov.io/gh/apache/airflow/pull/6520?src=pr=tree) ```diff @@Coverage Diff @@ ## master#6520 +/- ## == - Coverage 84.08% 83.76% -0.32% == Files 635 635 Lines 3684936851 +2 == - Hits3098330867 -116 - Misses 5866 5984 +118 ``` | [Impacted Files](https://codecov.io/gh/apache/airflow/pull/6520?src=pr=tree) | Coverage Δ | | |---|---|---| | [airflow/models/pool.py](https://codecov.io/gh/apache/airflow/pull/6520/diff?src=pr=tree#diff-YWlyZmxvdy9tb2RlbHMvcG9vbC5weQ==) | `97.36% <100%> (+0.14%)` | :arrow_up: | | [airflow/kubernetes/volume\_mount.py](https://codecov.io/gh/apache/airflow/pull/6520/diff?src=pr=tree#diff-YWlyZmxvdy9rdWJlcm5ldGVzL3ZvbHVtZV9tb3VudC5weQ==) | `44.44% <0%> (-55.56%)` | :arrow_down: | | [airflow/kubernetes/volume.py](https://codecov.io/gh/apache/airflow/pull/6520/diff?src=pr=tree#diff-YWlyZmxvdy9rdWJlcm5ldGVzL3ZvbHVtZS5weQ==) | `52.94% <0%> (-47.06%)` | :arrow_down: | | [airflow/kubernetes/pod\_launcher.py](https://codecov.io/gh/apache/airflow/pull/6520/diff?src=pr=tree#diff-YWlyZmxvdy9rdWJlcm5ldGVzL3BvZF9sYXVuY2hlci5weQ==) | `45.25% <0%> (-46.72%)` | :arrow_down: | | [airflow/kubernetes/kube\_client.py](https://codecov.io/gh/apache/airflow/pull/6520/diff?src=pr=tree#diff-YWlyZmxvdy9rdWJlcm5ldGVzL2t1YmVfY2xpZW50LnB5) | `33.33% <0%> (-41.67%)` | :arrow_down: | | [...rflow/contrib/operators/kubernetes\_pod\_operator.py](https://codecov.io/gh/apache/airflow/pull/6520/diff?src=pr=tree#diff-YWlyZmxvdy9jb250cmliL29wZXJhdG9ycy9rdWJlcm5ldGVzX3BvZF9vcGVyYXRvci5weQ==) | `70.14% <0%> (-28.36%)` | :arrow_down: | | [airflow/jobs/local\_task\_job.py](https://codecov.io/gh/apache/airflow/pull/6520/diff?src=pr=tree#diff-YWlyZmxvdy9qb2JzL2xvY2FsX3Rhc2tfam9iLnB5) | `85% <0%> (-5%)` | :arrow_down: | | [airflow/models/taskinstance.py](https://codecov.io/gh/apache/airflow/pull/6520/diff?src=pr=tree#diff-YWlyZmxvdy9tb2RlbHMvdGFza2luc3RhbmNlLnB5) | `93.28% <0%> (-0.51%)` | :arrow_down: | -- [Continue to review full report at Codecov](https://codecov.io/gh/apache/airflow/pull/6520?src=pr=continue). > **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta) > `Δ = absolute (impact)`, `ø = not affected`, `? = missing data` > Powered by [Codecov](https://codecov.io/gh/apache/airflow/pull/6520?src=pr=footer). Last update [813cd43...5ceb0ff](https://codecov.io/gh/apache/airflow/pull/6520?src=pr=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments). This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [airflow] codecov-io edited a comment on issue #6520: [AIRFLOW-5870] Allow -1 for infinite pool size
codecov-io edited a comment on issue #6520: [AIRFLOW-5870] Allow -1 for infinite pool size URL: https://github.com/apache/airflow/pull/6520#issuecomment-552000288 # [Codecov](https://codecov.io/gh/apache/airflow/pull/6520?src=pr=h1) Report > Merging [#6520](https://codecov.io/gh/apache/airflow/pull/6520?src=pr=desc) into [master](https://codecov.io/gh/apache/airflow/commit/813cd431204b71f4e7c27f2bee9fafcc910da4d0?src=pr=desc) will **decrease** coverage by `0.31%`. > The diff coverage is `100%`. [](https://codecov.io/gh/apache/airflow/pull/6520?src=pr=tree) ```diff @@Coverage Diff @@ ## master#6520 +/- ## == - Coverage 84.08% 83.76% -0.32% == Files 635 635 Lines 3684936851 +2 == - Hits3098330867 -116 - Misses 5866 5984 +118 ``` | [Impacted Files](https://codecov.io/gh/apache/airflow/pull/6520?src=pr=tree) | Coverage Δ | | |---|---|---| | [airflow/models/pool.py](https://codecov.io/gh/apache/airflow/pull/6520/diff?src=pr=tree#diff-YWlyZmxvdy9tb2RlbHMvcG9vbC5weQ==) | `97.36% <100%> (+0.14%)` | :arrow_up: | | [airflow/kubernetes/volume\_mount.py](https://codecov.io/gh/apache/airflow/pull/6520/diff?src=pr=tree#diff-YWlyZmxvdy9rdWJlcm5ldGVzL3ZvbHVtZV9tb3VudC5weQ==) | `44.44% <0%> (-55.56%)` | :arrow_down: | | [airflow/kubernetes/volume.py](https://codecov.io/gh/apache/airflow/pull/6520/diff?src=pr=tree#diff-YWlyZmxvdy9rdWJlcm5ldGVzL3ZvbHVtZS5weQ==) | `52.94% <0%> (-47.06%)` | :arrow_down: | | [airflow/kubernetes/pod\_launcher.py](https://codecov.io/gh/apache/airflow/pull/6520/diff?src=pr=tree#diff-YWlyZmxvdy9rdWJlcm5ldGVzL3BvZF9sYXVuY2hlci5weQ==) | `45.25% <0%> (-46.72%)` | :arrow_down: | | [airflow/kubernetes/kube\_client.py](https://codecov.io/gh/apache/airflow/pull/6520/diff?src=pr=tree#diff-YWlyZmxvdy9rdWJlcm5ldGVzL2t1YmVfY2xpZW50LnB5) | `33.33% <0%> (-41.67%)` | :arrow_down: | | [...rflow/contrib/operators/kubernetes\_pod\_operator.py](https://codecov.io/gh/apache/airflow/pull/6520/diff?src=pr=tree#diff-YWlyZmxvdy9jb250cmliL29wZXJhdG9ycy9rdWJlcm5ldGVzX3BvZF9vcGVyYXRvci5weQ==) | `70.14% <0%> (-28.36%)` | :arrow_down: | | [airflow/jobs/local\_task\_job.py](https://codecov.io/gh/apache/airflow/pull/6520/diff?src=pr=tree#diff-YWlyZmxvdy9qb2JzL2xvY2FsX3Rhc2tfam9iLnB5) | `85% <0%> (-5%)` | :arrow_down: | | [airflow/models/taskinstance.py](https://codecov.io/gh/apache/airflow/pull/6520/diff?src=pr=tree#diff-YWlyZmxvdy9tb2RlbHMvdGFza2luc3RhbmNlLnB5) | `93.28% <0%> (-0.51%)` | :arrow_down: | -- [Continue to review full report at Codecov](https://codecov.io/gh/apache/airflow/pull/6520?src=pr=continue). > **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta) > `Δ = absolute (impact)`, `ø = not affected`, `? = missing data` > Powered by [Codecov](https://codecov.io/gh/apache/airflow/pull/6520?src=pr=footer). Last update [813cd43...5ceb0ff](https://codecov.io/gh/apache/airflow/pull/6520?src=pr=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments). This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [airflow] sekikn commented on issue #6457: [AIRFLOW-5800] Add a default connection entry for PinotDbApiHook
sekikn commented on issue #6457: [AIRFLOW-5800] Add a default connection entry for PinotDbApiHook URL: https://github.com/apache/airflow/pull/6457#issuecomment-552071978 Rebased on master and resolved the conflict. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Created] (AIRFLOW-5882) State dependency should not be ignorable
Alex Guziel created AIRFLOW-5882: Summary: State dependency should not be ignorable Key: AIRFLOW-5882 URL: https://issues.apache.org/jira/browse/AIRFLOW-5882 Project: Apache Airflow Issue Type: Improvement Components: worker Affects Versions: 1.10.6 Reporter: Alex Guziel Assignee: Alex Guziel This ti_state dep should not be ignoreable as it would allow double-run in the "ideal" situation and in the current situation it causes both tasks to die. Due to celery visibility timeout, the task will always kill itself eventually -- This message was sent by Atlassian Jira (v8.3.4#803005)
[GitHub] [airflow-site] mik-laj opened a new pull request #148: Update the integration list to point to the attached documentation
mik-laj opened a new pull request #148: Update the integration list to point to the attached documentation URL: https://github.com/apache/airflow-site/pull/148 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [airflow] codecov-io commented on issue #6532: [AIRFLOW-5883] Don't use .count() from sqlalchemy to count
codecov-io commented on issue #6532: [AIRFLOW-5883] Don't use .count() from sqlalchemy to count URL: https://github.com/apache/airflow/pull/6532#issuecomment-552059516 # [Codecov](https://codecov.io/gh/apache/airflow/pull/6532?src=pr=h1) Report > Merging [#6532](https://codecov.io/gh/apache/airflow/pull/6532?src=pr=desc) into [master](https://codecov.io/gh/apache/airflow/commit/813cd431204b71f4e7c27f2bee9fafcc910da4d0?src=pr=desc) will **decrease** coverage by `0.36%`. > The diff coverage is `100%`. [](https://codecov.io/gh/apache/airflow/pull/6532?src=pr=tree) ```diff @@Coverage Diff @@ ## master#6532 +/- ## == - Coverage 84.08% 83.71% -0.37% == Files 635 635 Lines 3684936850 +1 == - Hits3098330849 -134 - Misses 5866 6001 +135 ``` | [Impacted Files](https://codecov.io/gh/apache/airflow/pull/6532?src=pr=tree) | Coverage Δ | | |---|---|---| | [airflow/sensors/external\_task\_sensor.py](https://codecov.io/gh/apache/airflow/pull/6532/diff?src=pr=tree#diff-YWlyZmxvdy9zZW5zb3JzL2V4dGVybmFsX3Rhc2tfc2Vuc29yLnB5) | `85.96% <100%> (+0.25%)` | :arrow_up: | | [airflow/models/dag.py](https://codecov.io/gh/apache/airflow/pull/6532/diff?src=pr=tree#diff-YWlyZmxvdy9tb2RlbHMvZGFnLnB5) | `90.87% <100%> (ø)` | :arrow_up: | | [airflow/models/taskinstance.py](https://codecov.io/gh/apache/airflow/pull/6532/diff?src=pr=tree#diff-YWlyZmxvdy9tb2RlbHMvdGFza2luc3RhbmNlLnB5) | `93.79% <100%> (ø)` | :arrow_up: | | [airflow/operators/postgres\_operator.py](https://codecov.io/gh/apache/airflow/pull/6532/diff?src=pr=tree#diff-YWlyZmxvdy9vcGVyYXRvcnMvcG9zdGdyZXNfb3BlcmF0b3IucHk=) | `0% <0%> (-100%)` | :arrow_down: | | [airflow/kubernetes/volume\_mount.py](https://codecov.io/gh/apache/airflow/pull/6532/diff?src=pr=tree#diff-YWlyZmxvdy9rdWJlcm5ldGVzL3ZvbHVtZV9tb3VudC5weQ==) | `44.44% <0%> (-55.56%)` | :arrow_down: | | [airflow/kubernetes/volume.py](https://codecov.io/gh/apache/airflow/pull/6532/diff?src=pr=tree#diff-YWlyZmxvdy9rdWJlcm5ldGVzL3ZvbHVtZS5weQ==) | `52.94% <0%> (-47.06%)` | :arrow_down: | | [airflow/kubernetes/pod\_launcher.py](https://codecov.io/gh/apache/airflow/pull/6532/diff?src=pr=tree#diff-YWlyZmxvdy9rdWJlcm5ldGVzL3BvZF9sYXVuY2hlci5weQ==) | `45.25% <0%> (-46.72%)` | :arrow_down: | | [airflow/kubernetes/kube\_client.py](https://codecov.io/gh/apache/airflow/pull/6532/diff?src=pr=tree#diff-YWlyZmxvdy9rdWJlcm5ldGVzL2t1YmVfY2xpZW50LnB5) | `33.33% <0%> (-41.67%)` | :arrow_down: | | [...rflow/contrib/operators/kubernetes\_pod\_operator.py](https://codecov.io/gh/apache/airflow/pull/6532/diff?src=pr=tree#diff-YWlyZmxvdy9jb250cmliL29wZXJhdG9ycy9rdWJlcm5ldGVzX3BvZF9vcGVyYXRvci5weQ==) | `70.14% <0%> (-28.36%)` | :arrow_down: | | [airflow/utils/sqlalchemy.py](https://codecov.io/gh/apache/airflow/pull/6532/diff?src=pr=tree#diff-YWlyZmxvdy91dGlscy9zcWxhbGNoZW15LnB5) | `91.52% <0%> (-1.7%)` | :arrow_down: | | ... and [3 more](https://codecov.io/gh/apache/airflow/pull/6532/diff?src=pr=tree-more) | | -- [Continue to review full report at Codecov](https://codecov.io/gh/apache/airflow/pull/6532?src=pr=continue). > **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta) > `Δ = absolute (impact)`, `ø = not affected`, `? = missing data` > Powered by [Codecov](https://codecov.io/gh/apache/airflow/pull/6532?src=pr=footer). Last update [813cd43...2a734d5](https://codecov.io/gh/apache/airflow/pull/6532?src=pr=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments). This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [airflow] codecov-io edited a comment on issue #6532: [AIRFLOW-5883] Don't use .count() from sqlalchemy to count
codecov-io edited a comment on issue #6532: [AIRFLOW-5883] Don't use .count() from sqlalchemy to count URL: https://github.com/apache/airflow/pull/6532#issuecomment-552059516 # [Codecov](https://codecov.io/gh/apache/airflow/pull/6532?src=pr=h1) Report > Merging [#6532](https://codecov.io/gh/apache/airflow/pull/6532?src=pr=desc) into [master](https://codecov.io/gh/apache/airflow/commit/813cd431204b71f4e7c27f2bee9fafcc910da4d0?src=pr=desc) will **decrease** coverage by `0.29%`. > The diff coverage is `100%`. [](https://codecov.io/gh/apache/airflow/pull/6532?src=pr=tree) ```diff @@Coverage Diff@@ ## master#6532 +/- ## = - Coverage 84.08% 83.78% -0.3% = Files 635 635 Lines 3684936850 +1 = - Hits3098330875-108 - Misses 5866 5975+109 ``` | [Impacted Files](https://codecov.io/gh/apache/airflow/pull/6532?src=pr=tree) | Coverage Δ | | |---|---|---| | [airflow/sensors/external\_task\_sensor.py](https://codecov.io/gh/apache/airflow/pull/6532/diff?src=pr=tree#diff-YWlyZmxvdy9zZW5zb3JzL2V4dGVybmFsX3Rhc2tfc2Vuc29yLnB5) | `85.96% <100%> (+0.25%)` | :arrow_up: | | [airflow/models/dag.py](https://codecov.io/gh/apache/airflow/pull/6532/diff?src=pr=tree#diff-YWlyZmxvdy9tb2RlbHMvZGFnLnB5) | `90.87% <100%> (ø)` | :arrow_up: | | [airflow/models/taskinstance.py](https://codecov.io/gh/apache/airflow/pull/6532/diff?src=pr=tree#diff-YWlyZmxvdy9tb2RlbHMvdGFza2luc3RhbmNlLnB5) | `93.79% <100%> (ø)` | :arrow_up: | | [airflow/kubernetes/volume\_mount.py](https://codecov.io/gh/apache/airflow/pull/6532/diff?src=pr=tree#diff-YWlyZmxvdy9rdWJlcm5ldGVzL3ZvbHVtZV9tb3VudC5weQ==) | `44.44% <0%> (-55.56%)` | :arrow_down: | | [airflow/kubernetes/volume.py](https://codecov.io/gh/apache/airflow/pull/6532/diff?src=pr=tree#diff-YWlyZmxvdy9rdWJlcm5ldGVzL3ZvbHVtZS5weQ==) | `52.94% <0%> (-47.06%)` | :arrow_down: | | [airflow/kubernetes/pod\_launcher.py](https://codecov.io/gh/apache/airflow/pull/6532/diff?src=pr=tree#diff-YWlyZmxvdy9rdWJlcm5ldGVzL3BvZF9sYXVuY2hlci5weQ==) | `45.25% <0%> (-46.72%)` | :arrow_down: | | [airflow/kubernetes/kube\_client.py](https://codecov.io/gh/apache/airflow/pull/6532/diff?src=pr=tree#diff-YWlyZmxvdy9rdWJlcm5ldGVzL2t1YmVfY2xpZW50LnB5) | `33.33% <0%> (-41.67%)` | :arrow_down: | | [...rflow/contrib/operators/kubernetes\_pod\_operator.py](https://codecov.io/gh/apache/airflow/pull/6532/diff?src=pr=tree#diff-YWlyZmxvdy9jb250cmliL29wZXJhdG9ycy9rdWJlcm5ldGVzX3BvZF9vcGVyYXRvci5weQ==) | `70.14% <0%> (-28.36%)` | :arrow_down: | | [airflow/utils/dag\_processing.py](https://codecov.io/gh/apache/airflow/pull/6532/diff?src=pr=tree#diff-YWlyZmxvdy91dGlscy9kYWdfcHJvY2Vzc2luZy5weQ==) | `58.48% <0%> (+0.32%)` | :arrow_up: | -- [Continue to review full report at Codecov](https://codecov.io/gh/apache/airflow/pull/6532?src=pr=continue). > **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta) > `Δ = absolute (impact)`, `ø = not affected`, `? = missing data` > Powered by [Codecov](https://codecov.io/gh/apache/airflow/pull/6532?src=pr=footer). Last update [813cd43...2a734d5](https://codecov.io/gh/apache/airflow/pull/6532?src=pr=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments). This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [airflow] codecov-io edited a comment on issue #6532: [AIRFLOW-5883] Don't use .count() from sqlalchemy to count
codecov-io edited a comment on issue #6532: [AIRFLOW-5883] Don't use .count() from sqlalchemy to count URL: https://github.com/apache/airflow/pull/6532#issuecomment-552059516 # [Codecov](https://codecov.io/gh/apache/airflow/pull/6532?src=pr=h1) Report > Merging [#6532](https://codecov.io/gh/apache/airflow/pull/6532?src=pr=desc) into [master](https://codecov.io/gh/apache/airflow/commit/813cd431204b71f4e7c27f2bee9fafcc910da4d0?src=pr=desc) will **decrease** coverage by `0.29%`. > The diff coverage is `100%`. [](https://codecov.io/gh/apache/airflow/pull/6532?src=pr=tree) ```diff @@Coverage Diff@@ ## master#6532 +/- ## = - Coverage 84.08% 83.78% -0.3% = Files 635 635 Lines 3684936850 +1 = - Hits3098330875-108 - Misses 5866 5975+109 ``` | [Impacted Files](https://codecov.io/gh/apache/airflow/pull/6532?src=pr=tree) | Coverage Δ | | |---|---|---| | [airflow/sensors/external\_task\_sensor.py](https://codecov.io/gh/apache/airflow/pull/6532/diff?src=pr=tree#diff-YWlyZmxvdy9zZW5zb3JzL2V4dGVybmFsX3Rhc2tfc2Vuc29yLnB5) | `85.96% <100%> (+0.25%)` | :arrow_up: | | [airflow/models/dag.py](https://codecov.io/gh/apache/airflow/pull/6532/diff?src=pr=tree#diff-YWlyZmxvdy9tb2RlbHMvZGFnLnB5) | `90.87% <100%> (ø)` | :arrow_up: | | [airflow/models/taskinstance.py](https://codecov.io/gh/apache/airflow/pull/6532/diff?src=pr=tree#diff-YWlyZmxvdy9tb2RlbHMvdGFza2luc3RhbmNlLnB5) | `93.79% <100%> (ø)` | :arrow_up: | | [airflow/kubernetes/volume\_mount.py](https://codecov.io/gh/apache/airflow/pull/6532/diff?src=pr=tree#diff-YWlyZmxvdy9rdWJlcm5ldGVzL3ZvbHVtZV9tb3VudC5weQ==) | `44.44% <0%> (-55.56%)` | :arrow_down: | | [airflow/kubernetes/volume.py](https://codecov.io/gh/apache/airflow/pull/6532/diff?src=pr=tree#diff-YWlyZmxvdy9rdWJlcm5ldGVzL3ZvbHVtZS5weQ==) | `52.94% <0%> (-47.06%)` | :arrow_down: | | [airflow/kubernetes/pod\_launcher.py](https://codecov.io/gh/apache/airflow/pull/6532/diff?src=pr=tree#diff-YWlyZmxvdy9rdWJlcm5ldGVzL3BvZF9sYXVuY2hlci5weQ==) | `45.25% <0%> (-46.72%)` | :arrow_down: | | [airflow/kubernetes/kube\_client.py](https://codecov.io/gh/apache/airflow/pull/6532/diff?src=pr=tree#diff-YWlyZmxvdy9rdWJlcm5ldGVzL2t1YmVfY2xpZW50LnB5) | `33.33% <0%> (-41.67%)` | :arrow_down: | | [...rflow/contrib/operators/kubernetes\_pod\_operator.py](https://codecov.io/gh/apache/airflow/pull/6532/diff?src=pr=tree#diff-YWlyZmxvdy9jb250cmliL29wZXJhdG9ycy9rdWJlcm5ldGVzX3BvZF9vcGVyYXRvci5weQ==) | `70.14% <0%> (-28.36%)` | :arrow_down: | | [airflow/utils/dag\_processing.py](https://codecov.io/gh/apache/airflow/pull/6532/diff?src=pr=tree#diff-YWlyZmxvdy91dGlscy9kYWdfcHJvY2Vzc2luZy5weQ==) | `58.48% <0%> (+0.32%)` | :arrow_up: | -- [Continue to review full report at Codecov](https://codecov.io/gh/apache/airflow/pull/6532?src=pr=continue). > **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta) > `Δ = absolute (impact)`, `ø = not affected`, `? = missing data` > Powered by [Codecov](https://codecov.io/gh/apache/airflow/pull/6532?src=pr=footer). Last update [813cd43...2a734d5](https://codecov.io/gh/apache/airflow/pull/6532?src=pr=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments). This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [airflow] codecov-io edited a comment on issue #6457: [AIRFLOW-5800] Add a default connection entry for PinotDbApiHook
codecov-io edited a comment on issue #6457: [AIRFLOW-5800] Add a default connection entry for PinotDbApiHook URL: https://github.com/apache/airflow/pull/6457#issuecomment-552073991 # [Codecov](https://codecov.io/gh/apache/airflow/pull/6457?src=pr=h1) Report > Merging [#6457](https://codecov.io/gh/apache/airflow/pull/6457?src=pr=desc) into [master](https://codecov.io/gh/apache/airflow/commit/813cd431204b71f4e7c27f2bee9fafcc910da4d0?src=pr=desc) will **decrease** coverage by `0.76%`. > The diff coverage is `100%`. [](https://codecov.io/gh/apache/airflow/pull/6457?src=pr=tree) ```diff @@Coverage Diff @@ ## master#6457 +/- ## == - Coverage 84.08% 83.31% -0.77% == Files 635 631 -4 Lines 3684936639 -210 == - Hits3098330527 -456 - Misses 5866 6112 +246 ``` | [Impacted Files](https://codecov.io/gh/apache/airflow/pull/6457?src=pr=tree) | Coverage Δ | | |---|---|---| | [airflow/utils/db.py](https://codecov.io/gh/apache/airflow/pull/6457/diff?src=pr=tree#diff-YWlyZmxvdy91dGlscy9kYi5weQ==) | `90.17% <100%> (ø)` | :arrow_up: | | [airflow/operators/mysql\_operator.py](https://codecov.io/gh/apache/airflow/pull/6457/diff?src=pr=tree#diff-YWlyZmxvdy9vcGVyYXRvcnMvbXlzcWxfb3BlcmF0b3IucHk=) | `0% <0%> (-100%)` | :arrow_down: | | [airflow/operators/mysql\_to\_hive.py](https://codecov.io/gh/apache/airflow/pull/6457/diff?src=pr=tree#diff-YWlyZmxvdy9vcGVyYXRvcnMvbXlzcWxfdG9faGl2ZS5weQ==) | `0% <0%> (-100%)` | :arrow_down: | | [airflow/kubernetes/volume\_mount.py](https://codecov.io/gh/apache/airflow/pull/6457/diff?src=pr=tree#diff-YWlyZmxvdy9rdWJlcm5ldGVzL3ZvbHVtZV9tb3VudC5weQ==) | `44.44% <0%> (-55.56%)` | :arrow_down: | | [airflow/executors/sequential\_executor.py](https://codecov.io/gh/apache/airflow/pull/6457/diff?src=pr=tree#diff-YWlyZmxvdy9leGVjdXRvcnMvc2VxdWVudGlhbF9leGVjdXRvci5weQ==) | `47.61% <0%> (-52.39%)` | :arrow_down: | | [...ow/contrib/example\_dags/example\_qubole\_operator.py](https://codecov.io/gh/apache/airflow/pull/6457/diff?src=pr=tree#diff-YWlyZmxvdy9jb250cmliL2V4YW1wbGVfZGFncy9leGFtcGxlX3F1Ym9sZV9vcGVyYXRvci5weQ==) | `32.5% <0%> (-47.5%)` | :arrow_down: | | [airflow/kubernetes/volume.py](https://codecov.io/gh/apache/airflow/pull/6457/diff?src=pr=tree#diff-YWlyZmxvdy9rdWJlcm5ldGVzL3ZvbHVtZS5weQ==) | `52.94% <0%> (-47.06%)` | :arrow_down: | | [airflow/kubernetes/pod\_launcher.py](https://codecov.io/gh/apache/airflow/pull/6457/diff?src=pr=tree#diff-YWlyZmxvdy9rdWJlcm5ldGVzL3BvZF9sYXVuY2hlci5weQ==) | `45.25% <0%> (-46.72%)` | :arrow_down: | | [airflow/kubernetes/kube\_client.py](https://codecov.io/gh/apache/airflow/pull/6457/diff?src=pr=tree#diff-YWlyZmxvdy9rdWJlcm5ldGVzL2t1YmVfY2xpZW50LnB5) | `33.33% <0%> (-41.67%)` | :arrow_down: | | [airflow/hooks/presto\_hook.py](https://codecov.io/gh/apache/airflow/pull/6457/diff?src=pr=tree#diff-YWlyZmxvdy9ob29rcy9wcmVzdG9faG9vay5weQ==) | `36.73% <0%> (-40.82%)` | :arrow_down: | | ... and [63 more](https://codecov.io/gh/apache/airflow/pull/6457/diff?src=pr=tree-more) | | -- [Continue to review full report at Codecov](https://codecov.io/gh/apache/airflow/pull/6457?src=pr=continue). > **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta) > `Δ = absolute (impact)`, `ø = not affected`, `? = missing data` > Powered by [Codecov](https://codecov.io/gh/apache/airflow/pull/6457?src=pr=footer). Last update [813cd43...0cb9d2d](https://codecov.io/gh/apache/airflow/pull/6457?src=pr=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments). This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [airflow] codecov-io edited a comment on issue #6457: [AIRFLOW-5800] Add a default connection entry for PinotDbApiHook
codecov-io edited a comment on issue #6457: [AIRFLOW-5800] Add a default connection entry for PinotDbApiHook URL: https://github.com/apache/airflow/pull/6457#issuecomment-552073991 # [Codecov](https://codecov.io/gh/apache/airflow/pull/6457?src=pr=h1) Report > Merging [#6457](https://codecov.io/gh/apache/airflow/pull/6457?src=pr=desc) into [master](https://codecov.io/gh/apache/airflow/commit/813cd431204b71f4e7c27f2bee9fafcc910da4d0?src=pr=desc) will **decrease** coverage by `0.76%`. > The diff coverage is `100%`. [](https://codecov.io/gh/apache/airflow/pull/6457?src=pr=tree) ```diff @@Coverage Diff @@ ## master#6457 +/- ## == - Coverage 84.08% 83.31% -0.77% == Files 635 631 -4 Lines 3684936639 -210 == - Hits3098330527 -456 - Misses 5866 6112 +246 ``` | [Impacted Files](https://codecov.io/gh/apache/airflow/pull/6457?src=pr=tree) | Coverage Δ | | |---|---|---| | [airflow/utils/db.py](https://codecov.io/gh/apache/airflow/pull/6457/diff?src=pr=tree#diff-YWlyZmxvdy91dGlscy9kYi5weQ==) | `90.17% <100%> (ø)` | :arrow_up: | | [airflow/operators/mysql\_operator.py](https://codecov.io/gh/apache/airflow/pull/6457/diff?src=pr=tree#diff-YWlyZmxvdy9vcGVyYXRvcnMvbXlzcWxfb3BlcmF0b3IucHk=) | `0% <0%> (-100%)` | :arrow_down: | | [airflow/operators/mysql\_to\_hive.py](https://codecov.io/gh/apache/airflow/pull/6457/diff?src=pr=tree#diff-YWlyZmxvdy9vcGVyYXRvcnMvbXlzcWxfdG9faGl2ZS5weQ==) | `0% <0%> (-100%)` | :arrow_down: | | [airflow/kubernetes/volume\_mount.py](https://codecov.io/gh/apache/airflow/pull/6457/diff?src=pr=tree#diff-YWlyZmxvdy9rdWJlcm5ldGVzL3ZvbHVtZV9tb3VudC5weQ==) | `44.44% <0%> (-55.56%)` | :arrow_down: | | [airflow/executors/sequential\_executor.py](https://codecov.io/gh/apache/airflow/pull/6457/diff?src=pr=tree#diff-YWlyZmxvdy9leGVjdXRvcnMvc2VxdWVudGlhbF9leGVjdXRvci5weQ==) | `47.61% <0%> (-52.39%)` | :arrow_down: | | [...ow/contrib/example\_dags/example\_qubole\_operator.py](https://codecov.io/gh/apache/airflow/pull/6457/diff?src=pr=tree#diff-YWlyZmxvdy9jb250cmliL2V4YW1wbGVfZGFncy9leGFtcGxlX3F1Ym9sZV9vcGVyYXRvci5weQ==) | `32.5% <0%> (-47.5%)` | :arrow_down: | | [airflow/kubernetes/volume.py](https://codecov.io/gh/apache/airflow/pull/6457/diff?src=pr=tree#diff-YWlyZmxvdy9rdWJlcm5ldGVzL3ZvbHVtZS5weQ==) | `52.94% <0%> (-47.06%)` | :arrow_down: | | [airflow/kubernetes/pod\_launcher.py](https://codecov.io/gh/apache/airflow/pull/6457/diff?src=pr=tree#diff-YWlyZmxvdy9rdWJlcm5ldGVzL3BvZF9sYXVuY2hlci5weQ==) | `45.25% <0%> (-46.72%)` | :arrow_down: | | [airflow/kubernetes/kube\_client.py](https://codecov.io/gh/apache/airflow/pull/6457/diff?src=pr=tree#diff-YWlyZmxvdy9rdWJlcm5ldGVzL2t1YmVfY2xpZW50LnB5) | `33.33% <0%> (-41.67%)` | :arrow_down: | | [airflow/hooks/presto\_hook.py](https://codecov.io/gh/apache/airflow/pull/6457/diff?src=pr=tree#diff-YWlyZmxvdy9ob29rcy9wcmVzdG9faG9vay5weQ==) | `36.73% <0%> (-40.82%)` | :arrow_down: | | ... and [63 more](https://codecov.io/gh/apache/airflow/pull/6457/diff?src=pr=tree-more) | | -- [Continue to review full report at Codecov](https://codecov.io/gh/apache/airflow/pull/6457?src=pr=continue). > **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta) > `Δ = absolute (impact)`, `ø = not affected`, `? = missing data` > Powered by [Codecov](https://codecov.io/gh/apache/airflow/pull/6457?src=pr=footer). Last update [813cd43...0cb9d2d](https://codecov.io/gh/apache/airflow/pull/6457?src=pr=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments). This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Created] (AIRFLOW-5879) Use JobID to monitor statuses when running a Dataflow template
Kamil Bregula created AIRFLOW-5879: -- Summary: Use JobID to monitor statuses when running a Dataflow template Key: AIRFLOW-5879 URL: https://issues.apache.org/jira/browse/AIRFLOW-5879 Project: Apache Airflow Issue Type: Sub-task Components: gcp Affects Versions: 1.10.6 Reporter: Kamil Bregula -- This message was sent by Atlassian Jira (v8.3.4#803005)
[GitHub] [airflow] codecov-io commented on issue #6482: [AIRFLOW-5823] Add a new hook for ingesting data into Apache Pinot
codecov-io commented on issue #6482: [AIRFLOW-5823] Add a new hook for ingesting data into Apache Pinot URL: https://github.com/apache/airflow/pull/6482#issuecomment-551882011 # [Codecov](https://codecov.io/gh/apache/airflow/pull/6482?src=pr=h1) Report > Merging [#6482](https://codecov.io/gh/apache/airflow/pull/6482?src=pr=desc) into [master](https://codecov.io/gh/apache/airflow/commit/6d798deec6a2c9d8460cbb00b1a565cf81c849f4?src=pr=desc) will **decrease** coverage by `0.21%`. > The diff coverage is `94.94%`. [](https://codecov.io/gh/apache/airflow/pull/6482?src=pr=tree) ```diff @@Coverage Diff @@ ## master#6482 +/- ## == - Coverage 83.99% 83.77% -0.22% == Files 635 635 Lines 3675036849 +99 == + Hits3086730872 +5 - Misses 5883 5977 +94 ``` | [Impacted Files](https://codecov.io/gh/apache/airflow/pull/6482?src=pr=tree) | Coverage Δ | | |---|---|---| | [airflow/utils/db.py](https://codecov.io/gh/apache/airflow/pull/6482/diff?src=pr=tree#diff-YWlyZmxvdy91dGlscy9kYi5weQ==) | `90.17% <100%> (+0.08%)` | :arrow_up: | | [airflow/contrib/hooks/pinot\_hook.py](https://codecov.io/gh/apache/airflow/pull/6482/diff?src=pr=tree#diff-YWlyZmxvdy9jb250cmliL2hvb2tzL3Bpbm90X2hvb2sucHk=) | `90.97% <94.89%> (+10.97%)` | :arrow_up: | | [airflow/kubernetes/volume\_mount.py](https://codecov.io/gh/apache/airflow/pull/6482/diff?src=pr=tree#diff-YWlyZmxvdy9rdWJlcm5ldGVzL3ZvbHVtZV9tb3VudC5weQ==) | `44.44% <0%> (-55.56%)` | :arrow_down: | | [airflow/kubernetes/volume.py](https://codecov.io/gh/apache/airflow/pull/6482/diff?src=pr=tree#diff-YWlyZmxvdy9rdWJlcm5ldGVzL3ZvbHVtZS5weQ==) | `52.94% <0%> (-47.06%)` | :arrow_down: | | [airflow/kubernetes/pod\_launcher.py](https://codecov.io/gh/apache/airflow/pull/6482/diff?src=pr=tree#diff-YWlyZmxvdy9rdWJlcm5ldGVzL3BvZF9sYXVuY2hlci5weQ==) | `45.25% <0%> (-46.72%)` | :arrow_down: | | [airflow/kubernetes/kube\_client.py](https://codecov.io/gh/apache/airflow/pull/6482/diff?src=pr=tree#diff-YWlyZmxvdy9rdWJlcm5ldGVzL2t1YmVfY2xpZW50LnB5) | `33.33% <0%> (-41.67%)` | :arrow_down: | | [...rflow/contrib/operators/kubernetes\_pod\_operator.py](https://codecov.io/gh/apache/airflow/pull/6482/diff?src=pr=tree#diff-YWlyZmxvdy9jb250cmliL29wZXJhdG9ycy9rdWJlcm5ldGVzX3BvZF9vcGVyYXRvci5weQ==) | `70.14% <0%> (-28.36%)` | :arrow_down: | | [airflow/utils/dag\_processing.py](https://codecov.io/gh/apache/airflow/pull/6482/diff?src=pr=tree#diff-YWlyZmxvdy91dGlscy9kYWdfcHJvY2Vzc2luZy5weQ==) | `58.15% <0%> (-0.33%)` | :arrow_down: | | [airflow/jobs/backfill\_job.py](https://codecov.io/gh/apache/airflow/pull/6482/diff?src=pr=tree#diff-YWlyZmxvdy9qb2JzL2JhY2tmaWxsX2pvYi5weQ==) | `91.43% <0%> (+1.52%)` | :arrow_up: | | [...ow/contrib/example\_dags/example\_qubole\_operator.py](https://codecov.io/gh/apache/airflow/pull/6482/diff?src=pr=tree#diff-YWlyZmxvdy9jb250cmliL2V4YW1wbGVfZGFncy9leGFtcGxlX3F1Ym9sZV9vcGVyYXRvci5weQ==) | `80% <0%> (+47.5%)` | :arrow_up: | -- [Continue to review full report at Codecov](https://codecov.io/gh/apache/airflow/pull/6482?src=pr=continue). > **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta) > `Δ = absolute (impact)`, `ø = not affected`, `? = missing data` > Powered by [Codecov](https://codecov.io/gh/apache/airflow/pull/6482?src=pr=footer). Last update [6d798de...4644b8f](https://codecov.io/gh/apache/airflow/pull/6482?src=pr=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments). This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [airflow] codecov-io edited a comment on issue #6482: [AIRFLOW-5823] Add a new hook for ingesting data into Apache Pinot
codecov-io edited a comment on issue #6482: [AIRFLOW-5823] Add a new hook for ingesting data into Apache Pinot URL: https://github.com/apache/airflow/pull/6482#issuecomment-551882011 # [Codecov](https://codecov.io/gh/apache/airflow/pull/6482?src=pr=h1) Report > Merging [#6482](https://codecov.io/gh/apache/airflow/pull/6482?src=pr=desc) into [master](https://codecov.io/gh/apache/airflow/commit/6d798deec6a2c9d8460cbb00b1a565cf81c849f4?src=pr=desc) will **decrease** coverage by `0.21%`. > The diff coverage is `94.94%`. [](https://codecov.io/gh/apache/airflow/pull/6482?src=pr=tree) ```diff @@Coverage Diff @@ ## master#6482 +/- ## == - Coverage 83.99% 83.77% -0.22% == Files 635 635 Lines 3675036849 +99 == + Hits3086730872 +5 - Misses 5883 5977 +94 ``` | [Impacted Files](https://codecov.io/gh/apache/airflow/pull/6482?src=pr=tree) | Coverage Δ | | |---|---|---| | [airflow/utils/db.py](https://codecov.io/gh/apache/airflow/pull/6482/diff?src=pr=tree#diff-YWlyZmxvdy91dGlscy9kYi5weQ==) | `90.17% <100%> (+0.08%)` | :arrow_up: | | [airflow/contrib/hooks/pinot\_hook.py](https://codecov.io/gh/apache/airflow/pull/6482/diff?src=pr=tree#diff-YWlyZmxvdy9jb250cmliL2hvb2tzL3Bpbm90X2hvb2sucHk=) | `90.97% <94.89%> (+10.97%)` | :arrow_up: | | [airflow/kubernetes/volume\_mount.py](https://codecov.io/gh/apache/airflow/pull/6482/diff?src=pr=tree#diff-YWlyZmxvdy9rdWJlcm5ldGVzL3ZvbHVtZV9tb3VudC5weQ==) | `44.44% <0%> (-55.56%)` | :arrow_down: | | [airflow/kubernetes/volume.py](https://codecov.io/gh/apache/airflow/pull/6482/diff?src=pr=tree#diff-YWlyZmxvdy9rdWJlcm5ldGVzL3ZvbHVtZS5weQ==) | `52.94% <0%> (-47.06%)` | :arrow_down: | | [airflow/kubernetes/pod\_launcher.py](https://codecov.io/gh/apache/airflow/pull/6482/diff?src=pr=tree#diff-YWlyZmxvdy9rdWJlcm5ldGVzL3BvZF9sYXVuY2hlci5weQ==) | `45.25% <0%> (-46.72%)` | :arrow_down: | | [airflow/kubernetes/kube\_client.py](https://codecov.io/gh/apache/airflow/pull/6482/diff?src=pr=tree#diff-YWlyZmxvdy9rdWJlcm5ldGVzL2t1YmVfY2xpZW50LnB5) | `33.33% <0%> (-41.67%)` | :arrow_down: | | [...rflow/contrib/operators/kubernetes\_pod\_operator.py](https://codecov.io/gh/apache/airflow/pull/6482/diff?src=pr=tree#diff-YWlyZmxvdy9jb250cmliL29wZXJhdG9ycy9rdWJlcm5ldGVzX3BvZF9vcGVyYXRvci5weQ==) | `70.14% <0%> (-28.36%)` | :arrow_down: | | [airflow/utils/dag\_processing.py](https://codecov.io/gh/apache/airflow/pull/6482/diff?src=pr=tree#diff-YWlyZmxvdy91dGlscy9kYWdfcHJvY2Vzc2luZy5weQ==) | `58.15% <0%> (-0.33%)` | :arrow_down: | | [airflow/jobs/backfill\_job.py](https://codecov.io/gh/apache/airflow/pull/6482/diff?src=pr=tree#diff-YWlyZmxvdy9qb2JzL2JhY2tmaWxsX2pvYi5weQ==) | `91.43% <0%> (+1.52%)` | :arrow_up: | | [...ow/contrib/example\_dags/example\_qubole\_operator.py](https://codecov.io/gh/apache/airflow/pull/6482/diff?src=pr=tree#diff-YWlyZmxvdy9jb250cmliL2V4YW1wbGVfZGFncy9leGFtcGxlX3F1Ym9sZV9vcGVyYXRvci5weQ==) | `80% <0%> (+47.5%)` | :arrow_up: | -- [Continue to review full report at Codecov](https://codecov.io/gh/apache/airflow/pull/6482?src=pr=continue). > **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta) > `Δ = absolute (impact)`, `ø = not affected`, `? = missing data` > Powered by [Codecov](https://codecov.io/gh/apache/airflow/pull/6482?src=pr=footer). Last update [6d798de...4644b8f](https://codecov.io/gh/apache/airflow/pull/6482?src=pr=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments). This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [airflow] baolsen commented on issue #6512: [AIRFLOW-5824] Added AWS DataSync Hook and Operator
baolsen commented on issue #6512: [AIRFLOW-5824] Added AWS DataSync Hook and Operator URL: https://github.com/apache/airflow/pull/6512#issuecomment-551882095 Sounds good, I wouldn't mind an example of the new conventions :) I'll wait for #6502 to be merged and then rebase. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [airflow] codecov-io edited a comment on issue #6482: [AIRFLOW-5823] Add a new hook for ingesting data into Apache Pinot
codecov-io edited a comment on issue #6482: [AIRFLOW-5823] Add a new hook for ingesting data into Apache Pinot URL: https://github.com/apache/airflow/pull/6482#issuecomment-551882011 # [Codecov](https://codecov.io/gh/apache/airflow/pull/6482?src=pr=h1) Report > Merging [#6482](https://codecov.io/gh/apache/airflow/pull/6482?src=pr=desc) into [master](https://codecov.io/gh/apache/airflow/commit/6d798deec6a2c9d8460cbb00b1a565cf81c849f4?src=pr=desc) will **decrease** coverage by `0.21%`. > The diff coverage is `94.94%`. [](https://codecov.io/gh/apache/airflow/pull/6482?src=pr=tree) ```diff @@Coverage Diff @@ ## master#6482 +/- ## == - Coverage 83.99% 83.77% -0.22% == Files 635 635 Lines 3675036849 +99 == + Hits3086730872 +5 - Misses 5883 5977 +94 ``` | [Impacted Files](https://codecov.io/gh/apache/airflow/pull/6482?src=pr=tree) | Coverage Δ | | |---|---|---| | [airflow/utils/db.py](https://codecov.io/gh/apache/airflow/pull/6482/diff?src=pr=tree#diff-YWlyZmxvdy91dGlscy9kYi5weQ==) | `90.17% <100%> (+0.08%)` | :arrow_up: | | [airflow/contrib/hooks/pinot\_hook.py](https://codecov.io/gh/apache/airflow/pull/6482/diff?src=pr=tree#diff-YWlyZmxvdy9jb250cmliL2hvb2tzL3Bpbm90X2hvb2sucHk=) | `90.97% <94.89%> (+10.97%)` | :arrow_up: | | [airflow/kubernetes/volume\_mount.py](https://codecov.io/gh/apache/airflow/pull/6482/diff?src=pr=tree#diff-YWlyZmxvdy9rdWJlcm5ldGVzL3ZvbHVtZV9tb3VudC5weQ==) | `44.44% <0%> (-55.56%)` | :arrow_down: | | [airflow/kubernetes/volume.py](https://codecov.io/gh/apache/airflow/pull/6482/diff?src=pr=tree#diff-YWlyZmxvdy9rdWJlcm5ldGVzL3ZvbHVtZS5weQ==) | `52.94% <0%> (-47.06%)` | :arrow_down: | | [airflow/kubernetes/pod\_launcher.py](https://codecov.io/gh/apache/airflow/pull/6482/diff?src=pr=tree#diff-YWlyZmxvdy9rdWJlcm5ldGVzL3BvZF9sYXVuY2hlci5weQ==) | `45.25% <0%> (-46.72%)` | :arrow_down: | | [airflow/kubernetes/kube\_client.py](https://codecov.io/gh/apache/airflow/pull/6482/diff?src=pr=tree#diff-YWlyZmxvdy9rdWJlcm5ldGVzL2t1YmVfY2xpZW50LnB5) | `33.33% <0%> (-41.67%)` | :arrow_down: | | [...rflow/contrib/operators/kubernetes\_pod\_operator.py](https://codecov.io/gh/apache/airflow/pull/6482/diff?src=pr=tree#diff-YWlyZmxvdy9jb250cmliL29wZXJhdG9ycy9rdWJlcm5ldGVzX3BvZF9vcGVyYXRvci5weQ==) | `70.14% <0%> (-28.36%)` | :arrow_down: | | [airflow/utils/dag\_processing.py](https://codecov.io/gh/apache/airflow/pull/6482/diff?src=pr=tree#diff-YWlyZmxvdy91dGlscy9kYWdfcHJvY2Vzc2luZy5weQ==) | `58.15% <0%> (-0.33%)` | :arrow_down: | | [airflow/jobs/backfill\_job.py](https://codecov.io/gh/apache/airflow/pull/6482/diff?src=pr=tree#diff-YWlyZmxvdy9qb2JzL2JhY2tmaWxsX2pvYi5weQ==) | `91.43% <0%> (+1.52%)` | :arrow_up: | | [...ow/contrib/example\_dags/example\_qubole\_operator.py](https://codecov.io/gh/apache/airflow/pull/6482/diff?src=pr=tree#diff-YWlyZmxvdy9jb250cmliL2V4YW1wbGVfZGFncy9leGFtcGxlX3F1Ym9sZV9vcGVyYXRvci5weQ==) | `80% <0%> (+47.5%)` | :arrow_up: | -- [Continue to review full report at Codecov](https://codecov.io/gh/apache/airflow/pull/6482?src=pr=continue). > **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta) > `Δ = absolute (impact)`, `ø = not affected`, `? = missing data` > Powered by [Codecov](https://codecov.io/gh/apache/airflow/pull/6482?src=pr=footer). Last update [6d798de...4644b8f](https://codecov.io/gh/apache/airflow/pull/6482?src=pr=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments). This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Updated] (AIRFLOW-5877) Improve job_id detection in DataflowRunner
[ https://issues.apache.org/jira/browse/AIRFLOW-5877?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kamil Bregula updated AIRFLOW-5877: --- Summary: Improve job_id detection in DataflowRunner (was: Improve job_id detection) > Improve job_id detection in DataflowRunner > -- > > Key: AIRFLOW-5877 > URL: https://issues.apache.org/jira/browse/AIRFLOW-5877 > Project: Apache Airflow > Issue Type: Sub-task > Components: gcp >Affects Versions: 1.10.6 >Reporter: Kamil Bregula >Priority: Major > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[GitHub] [airflow] mik-laj opened a new pull request #6529: [AIRFLOW-5877][depends on AIRFLOW-5876, AIRFLOW-5716][part of AIRFLOW-5697] Improve job_id detection
mik-laj opened a new pull request #6529: [AIRFLOW-5877][depends on
AIRFLOW-5876,AIRFLOW-5716][part of AIRFLOW-5697] Improve job_id detection
URL: https://github.com/apache/airflow/pull/6529
Make sure you have checked _all_ steps below.
### Jira
- [ ] My PR addresses the following [Airflow
Jira](https://issues.apache.org/jira/browse/AIRFLOW/) issues and references
them in the PR title. For example, "\[AIRFLOW-XXX\] My Airflow PR"
- https://issues.apache.org/jira/browse/AIRFLOW-XXX
- In case you are fixing a typo in the documentation you can prepend your
commit with \[AIRFLOW-XXX\], code changes always need a Jira issue.
- In case you are proposing a fundamental code change, you need to create
an Airflow Improvement Proposal
([AIP](https://cwiki.apache.org/confluence/display/AIRFLOW/Airflow+Improvements+Proposals)).
- In case you are adding a dependency, check if the license complies with
the [ASF 3rd Party License
Policy](https://www.apache.org/legal/resolved.html#category-x).
### Description
- [ ] Here are some details about my PR, including screenshots of any UI
changes:
### Tests
- [ ] My PR adds the following unit tests __OR__ does not need testing for
this extremely good reason:
### Commits
- [ ] My commits all reference Jira issues in their subject lines, and I
have squashed multiple commits if they address the same issue. In addition, my
commits follow the guidelines from "[How to write a good git commit
message](http://chris.beams.io/posts/git-commit/)":
1. Subject is separated from body by a blank line
1. Subject is limited to 50 characters (not including Jira issue reference)
1. Subject does not end with a period
1. Subject uses the imperative mood ("add", not "adding")
1. Body wraps at 72 characters
1. Body explains "what" and "why", not "how"
### Documentation
- [ ] In case of new functionality, my PR adds documentation that describes
how to use it.
- All the public functions and the classes in the PR contain docstrings
that explain what it does
- If you implement backwards incompatible changes, please leave a note in
the [Updating.md](https://github.com/apache/airflow/blob/master/UPDATING.md) so
we can assign it to a appropriate release
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[jira] [Commented] (AIRFLOW-5877) Improve job_id detection in DataflowRunner
[
https://issues.apache.org/jira/browse/AIRFLOW-5877?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16970350#comment-16970350
]
ASF GitHub Bot commented on AIRFLOW-5877:
-
mik-laj commented on pull request #6529: [AIRFLOW-5877][depends on
AIRFLOW-5876,AIRFLOW-5716][part of AIRFLOW-5697] Improve job_id detection
URL: https://github.com/apache/airflow/pull/6529
Make sure you have checked _all_ steps below.
### Jira
- [ ] My PR addresses the following [Airflow
Jira](https://issues.apache.org/jira/browse/AIRFLOW/) issues and references
them in the PR title. For example, "\[AIRFLOW-XXX\] My Airflow PR"
- https://issues.apache.org/jira/browse/AIRFLOW-XXX
- In case you are fixing a typo in the documentation you can prepend your
commit with \[AIRFLOW-XXX\], code changes always need a Jira issue.
- In case you are proposing a fundamental code change, you need to create
an Airflow Improvement Proposal
([AIP](https://cwiki.apache.org/confluence/display/AIRFLOW/Airflow+Improvements+Proposals)).
- In case you are adding a dependency, check if the license complies with
the [ASF 3rd Party License
Policy](https://www.apache.org/legal/resolved.html#category-x).
### Description
- [ ] Here are some details about my PR, including screenshots of any UI
changes:
### Tests
- [ ] My PR adds the following unit tests __OR__ does not need testing for
this extremely good reason:
### Commits
- [ ] My commits all reference Jira issues in their subject lines, and I
have squashed multiple commits if they address the same issue. In addition, my
commits follow the guidelines from "[How to write a good git commit
message](http://chris.beams.io/posts/git-commit/)":
1. Subject is separated from body by a blank line
1. Subject is limited to 50 characters (not including Jira issue reference)
1. Subject does not end with a period
1. Subject uses the imperative mood ("add", not "adding")
1. Body wraps at 72 characters
1. Body explains "what" and "why", not "how"
### Documentation
- [ ] In case of new functionality, my PR adds documentation that describes
how to use it.
- All the public functions and the classes in the PR contain docstrings
that explain what it does
- If you implement backwards incompatible changes, please leave a note in
the [Updating.md](https://github.com/apache/airflow/blob/master/UPDATING.md) so
we can assign it to a appropriate release
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
> Improve job_id detection in DataflowRunner
> --
>
> Key: AIRFLOW-5877
> URL: https://issues.apache.org/jira/browse/AIRFLOW-5877
> Project: Apache Airflow
> Issue Type: Sub-task
> Components: gcp
>Affects Versions: 1.10.6
>Reporter: Kamil Bregula
>Priority: Major
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Closed] (AIRFLOW-5878) Use JobID to monitor statuses when running a Dataflow template
[ https://issues.apache.org/jira/browse/AIRFLOW-5878?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kamil Bregula closed AIRFLOW-5878. -- Resolution: Fixed > Use JobID to monitor statuses when running a Dataflow template > -- > > Key: AIRFLOW-5878 > URL: https://issues.apache.org/jira/browse/AIRFLOW-5878 > Project: Apache Airflow > Issue Type: Bug > Components: gcp >Affects Versions: 1.10.6 >Reporter: Kamil Bregula >Priority: Major > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (AIRFLOW-2285) Need to be able to stop a running DAG
[ https://issues.apache.org/jira/browse/AIRFLOW-2285?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16970356#comment-16970356 ] Vasudha Putta commented on AIRFLOW-2285: Stopping a Dag run is not actually terminating the job processes running. > Need to be able to stop a running DAG > - > > Key: AIRFLOW-2285 > URL: https://issues.apache.org/jira/browse/AIRFLOW-2285 > Project: Apache Airflow > Issue Type: New Feature > Components: DAG >Affects Versions: 1.9.0 >Reporter: Gregory Dougherty >Priority: Major > Labels: features > > There is no way to stop a running DAG. As DAGs can have serious bugs (for > example, uploading data to the wrong location, when being charged for every > MB uploaded), need a way to force a DAG to stop running -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (AIRFLOW-5871) Stopping/Clearing a running airflow instance doesnt't terminate the job actually.
[ https://issues.apache.org/jira/browse/AIRFLOW-5871?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vasudha Putta updated AIRFLOW-5871: --- Priority: Critical (was: Major) > Stopping/Clearing a running airflow instance doesnt't terminate the job > actually. > - > > Key: AIRFLOW-5871 > URL: https://issues.apache.org/jira/browse/AIRFLOW-5871 > Project: Apache Airflow > Issue Type: Bug > Components: scheduler >Affects Versions: 1.10.1 >Reporter: Vasudha Putta >Priority: Critical > > Hi Team, > When I change the state of a running job instance to cleared/failed, it > doesn't completely terminate the existing job. I tried using pythonOperator, > bashOperator. The job connects to oracle and executes a package. Even > terminating/killing airflow job process won't terminate the oracle sessions. > This is an issue as whenever we would need to compile the package we would > have to stop the dags, marks existing dag runs to clear state and then kill > the oracle sessions. Is there a way to clean stop dag runs in airflow. > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[GitHub] [airflow] mik-laj opened a new pull request #6530: [AIRFLOW-5878] Use JobID to monitor statuses when running a Dataflow …
mik-laj opened a new pull request #6530: [AIRFLOW-5878] Use JobID to monitor
statuses when running a Dataflow …
URL: https://github.com/apache/airflow/pull/6530
…template
Make sure you have checked _all_ steps below.
### Jira
- [ ] My PR addresses the following [Airflow
Jira](https://issues.apache.org/jira/browse/AIRFLOW/) issues and references
them in the PR title. For example, "\[AIRFLOW-XXX\] My Airflow PR"
- https://issues.apache.org/jira/browse/AIRFLOW-5878
- In case you are fixing a typo in the documentation you can prepend your
commit with \[AIRFLOW-XXX\], code changes always need a Jira issue.
- In case you are proposing a fundamental code change, you need to create
an Airflow Improvement Proposal
([AIP](https://cwiki.apache.org/confluence/display/AIRFLOW/Airflow+Improvements+Proposals)).
- In case you are adding a dependency, check if the license complies with
the [ASF 3rd Party License
Policy](https://www.apache.org/legal/resolved.html#category-x).
### Description
- [ ] Here are some details about my PR, including screenshots of any UI
changes:
### Tests
- [ ] My PR adds the following unit tests __OR__ does not need testing for
this extremely good reason:
### Commits
- [ ] My commits all reference Jira issues in their subject lines, and I
have squashed multiple commits if they address the same issue. In addition, my
commits follow the guidelines from "[How to write a good git commit
message](http://chris.beams.io/posts/git-commit/)":
1. Subject is separated from body by a blank line
1. Subject is limited to 50 characters (not including Jira issue reference)
1. Subject does not end with a period
1. Subject uses the imperative mood ("add", not "adding")
1. Body wraps at 72 characters
1. Body explains "what" and "why", not "how"
### Documentation
- [ ] In case of new functionality, my PR adds documentation that describes
how to use it.
- All the public functions and the classes in the PR contain docstrings
that explain what it does
- If you implement backwards incompatible changes, please leave a note in
the [Updating.md](https://github.com/apache/airflow/blob/master/UPDATING.md) so
we can assign it to a appropriate release
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[jira] [Commented] (AIRFLOW-5878) Use JobID to monitor statuses when running a Dataflow template
[
https://issues.apache.org/jira/browse/AIRFLOW-5878?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16970363#comment-16970363
]
ASF GitHub Bot commented on AIRFLOW-5878:
-
mik-laj commented on pull request #6530: [AIRFLOW-5878] Use JobID to monitor
statuses when running a Dataflow …
URL: https://github.com/apache/airflow/pull/6530
…template
Make sure you have checked _all_ steps below.
### Jira
- [ ] My PR addresses the following [Airflow
Jira](https://issues.apache.org/jira/browse/AIRFLOW/) issues and references
them in the PR title. For example, "\[AIRFLOW-XXX\] My Airflow PR"
- https://issues.apache.org/jira/browse/AIRFLOW-5878
- In case you are fixing a typo in the documentation you can prepend your
commit with \[AIRFLOW-XXX\], code changes always need a Jira issue.
- In case you are proposing a fundamental code change, you need to create
an Airflow Improvement Proposal
([AIP](https://cwiki.apache.org/confluence/display/AIRFLOW/Airflow+Improvements+Proposals)).
- In case you are adding a dependency, check if the license complies with
the [ASF 3rd Party License
Policy](https://www.apache.org/legal/resolved.html#category-x).
### Description
- [ ] Here are some details about my PR, including screenshots of any UI
changes:
### Tests
- [ ] My PR adds the following unit tests __OR__ does not need testing for
this extremely good reason:
### Commits
- [ ] My commits all reference Jira issues in their subject lines, and I
have squashed multiple commits if they address the same issue. In addition, my
commits follow the guidelines from "[How to write a good git commit
message](http://chris.beams.io/posts/git-commit/)":
1. Subject is separated from body by a blank line
1. Subject is limited to 50 characters (not including Jira issue reference)
1. Subject does not end with a period
1. Subject uses the imperative mood ("add", not "adding")
1. Body wraps at 72 characters
1. Body explains "what" and "why", not "how"
### Documentation
- [ ] In case of new functionality, my PR adds documentation that describes
how to use it.
- All the public functions and the classes in the PR contain docstrings
that explain what it does
- If you implement backwards incompatible changes, please leave a note in
the [Updating.md](https://github.com/apache/airflow/blob/master/UPDATING.md) so
we can assign it to a appropriate release
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
> Use JobID to monitor statuses when running a Dataflow template
> --
>
> Key: AIRFLOW-5878
> URL: https://issues.apache.org/jira/browse/AIRFLOW-5878
> Project: Apache Airflow
> Issue Type: Bug
> Components: gcp
>Affects Versions: 1.10.6
>Reporter: Kamil Bregula
>Priority: Major
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (AIRFLOW-5844) [AIRFLOW-5844] Rename BaseSQLToGoogleCloudStorageOperator to BaseSqlToGcsOperator
[ https://issues.apache.org/jira/browse/AIRFLOW-5844?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16970095#comment-16970095 ] ASF GitHub Bot commented on AIRFLOW-5844: - blcksrx commented on pull request #6522: [AIRFLOW-5844] Rename BaseSQLToGoogleCloudStorageOperator URL: https://github.com/apache/airflow/pull/6522 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org > [AIRFLOW-5844] Rename BaseSQLToGoogleCloudStorageOperator to > BaseSqlToGcsOperator > - > > Key: AIRFLOW-5844 > URL: https://issues.apache.org/jira/browse/AIRFLOW-5844 > Project: Apache Airflow > Issue Type: Sub-task > Components: gcp >Affects Versions: 1.10.6 >Reporter: Michał Słowikowski >Assignee: Michał Słowikowski >Priority: Minor > > part of AIP-21 -- This message was sent by Atlassian Jira (v8.3.4#803005)
[GitHub] [airflow] blcksrx closed pull request #6522: [AIRFLOW-5844] Rename BaseSQLToGoogleCloudStorageOperator
blcksrx closed pull request #6522: [AIRFLOW-5844] Rename BaseSQLToGoogleCloudStorageOperator URL: https://github.com/apache/airflow/pull/6522 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [airflow] blcksrx commented on issue #6522: [AIRFLOW-5844] Rename BaseSQLToGoogleCloudStorageOperator
blcksrx commented on issue #6522: [AIRFLOW-5844] Rename BaseSQLToGoogleCloudStorageOperator URL: https://github.com/apache/airflow/pull/6522#issuecomment-551658869 > Hello @blcksrx - thanks for those. Not sure if you are fully aware about the discussion we have now in the devlist about AIP-21. The most recent agreement about AIP-21 is that we should move this operator to airflow/providers/google/cloud @michalslowikowski00 -> can you please take a look at this - I guess it somehow overlaps with the work you are currently doing ? > > @blcksrx - @michalslowikowski00 works currently on automated renaming/moving those operators for GCP and if he already has plans for this operator maybe it's better to close this one and wait for Michal's change. It's okay, I was doing the Jira issues only. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [airflow] ddelange opened a new pull request #6523: [AIRFLOW-5873] KubernetesPodOperator fixes and test
ddelange opened a new pull request #6523: [AIRFLOW-5873] KubernetesPodOperator fixes and test URL: https://github.com/apache/airflow/pull/6523 Make sure you have checked _all_ steps below. ### Jira - [x] My PR addresses the following [Airflow Jira] - https://issues.apache.org/jira/browse/AIRFLOW-5873 ### Description - `KubernetesPodOperator` kwarg `resources` is erroneously passed to `base_operator`, instead should only go to `PodGenerator`. The two have different syntax. (both on `master` and `v1-10-test` branches) - `KubernetesPodOperator` kwarg `in_cluster` erroneously defaults to False in comparison to `default_args.py` - `kubernetes/pod.py`: `Resources` does not have `__slots__` so accepts arbitrary values in `setattr` (present on `v1-10-test` branch https://github.com/apache/airflow/blame/v1-10-test/airflow/contrib/operators/kubernetes_pod_operator.py#L166) - `v1-10-test` is behind `master` with KubernetesPodOperator fixes and refactors - e.g. move kubernetes folder one level up from `/contrib` https://github.com/apache/airflow/blame/4dd24a2c595d4042ffe745aed947eaaea6abb652/airflow/contrib/operators/kubernetes_pod_operator.py#L21 - fix `xcom_push` to `do_xcom_push` https://github.com/apache/airflow/blame/4dd24a2c595d4042ffe745aed947eaaea6abb652/airflow/contrib/operators/kubernetes_pod_operator.py#L90 ### Tests - [x] My PR adds the following unit tests __OR__ does not need testing for this extremely good reason: `contrib/operators/test_kubernetes_pod_operator.py` This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [airflow] KKcorps commented on issue #6515: [AIRFLOW-XXX] GSoD: How to make DAGs production ready
KKcorps commented on issue #6515: [AIRFLOW-XXX] GSoD: How to make DAGs production ready URL: https://github.com/apache/airflow/pull/6515#issuecomment-551690166 @potiuk Yes, I am all for it. Personally, I also faced this issue. I have mentioned in the documentation that you shouldn't write a lot of code outside the DAG because of frequent parsing but I'll this point as well. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [airflow] ddelange commented on issue #6523: [AIRFLOW-5873] KubernetesPodOperator fixes and test
ddelange commented on issue #6523: [AIRFLOW-5873] KubernetesPodOperator fixes and test URL: https://github.com/apache/airflow/pull/6523#issuecomment-551694985 @ashb how should I cherry pick the relevant changes made to kubernetes in `master` onto `v1-10-test`? how/where is your branch structure documented (flow to pick commits for releases etc)? cc https://github.com/apache/airflow/pull/6506#issuecomment-550051457 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [airflow] KKcorps edited a comment on issue #6515: [AIRFLOW-XXX] GSoD: How to make DAGs production ready
KKcorps edited a comment on issue #6515: [AIRFLOW-XXX] GSoD: How to make DAGs production ready URL: https://github.com/apache/airflow/pull/6515#issuecomment-551690166 @potiuk Yes, I am all for it. Personally, I also faced this issue. I have mentioned in the documentation that you shouldn't write a lot of code outside the DAG because of frequent parsing but I'll elaborate that point to include this as well. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [airflow] feluelle commented on issue #6269: [AIRFLOW-5874] Use poke_interval of 0 to make EMR tests run green
feluelle commented on issue #6269: [AIRFLOW-5874] Use poke_interval of 0 to make EMR tests run green URL: https://github.com/apache/airflow/pull/6269#issuecomment-551712040 I created a Sub-Task https://issues.apache.org/jira/browse/AIRFLOW-5874 for this changes, because your changes are related to 3897 but don't completely fix them. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [airflow] feluelle edited a comment on issue #6269: [AIRFLOW-5874] Use poke_interval of 0 to make EMR tests run green
feluelle edited a comment on issue #6269: [AIRFLOW-5874] Use poke_interval of 0 to make EMR tests run green URL: https://github.com/apache/airflow/pull/6269#issuecomment-551712040 I created a Sub-Task https://issues.apache.org/jira/browse/AIRFLOW-5874 for these changes, because your changes are related to 3897 but don't completely fix them. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
