[GitHub] [airflow] boring-cyborg[bot] commented on issue #8237: dag will never start if we set a cron string with a clear day of week to the schedule_interval arg
boring-cyborg[bot] commented on issue #8237: dag will never start if we set a cron string with a clear day of week to the schedule_interval arg URL: https://github.com/apache/airflow/issues/8237#issuecomment-611899522 Thanks for opening your first issue here! Be sure to follow the issue template! This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [airflow] forbxy opened a new issue #8237: dag will never start if we set a cron string with a clear day of week to the schedule_interval arg
forbxy opened a new issue #8237: dag will never start if we set a cron string with a clear day of week to the schedule_interval arg URL: https://github.com/apache/airflow/issues/8237 dag like this ``` dag = DAG( dag_id='sch_test', default_args=args, schedule_interval='12 14 * * 5', dagrun_timeout=timedelta(minutes=60), tags=['bxy'] ) ``` This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Resolved] (AIRFLOW-7096) `apache-airflow[gcp]` can't be installed using pipenv
[
https://issues.apache.org/jira/browse/AIRFLOW-7096?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Igor Khrol resolved AIRFLOW-7096.
-
Fix Version/s: 1.10.10
Resolution: Fixed
Tested with 1.10.10 release. It's not reproducible today.
> `apache-airflow[gcp]` can't be installed using pipenv
> -
>
> Key: AIRFLOW-7096
> URL: https://issues.apache.org/jira/browse/AIRFLOW-7096
> Project: Apache Airflow
> Issue Type: Bug
> Components: dependencies
>Affects Versions: 1.10.9
>Reporter: Igor Khrol
>Priority: Major
> Fix For: 1.10.10
>
>
> There are some conflicting versions in {{gcp}} extra.
> {{$ pipenv install apache-airflow[gcp]}} is failing.
> See logs below:
> bq. Igors-MacBook-Pro:airflow-gcp igor_khrol$ pipenv --python 3.7
> bq. Warning: the environment variable LANG is not set!
> bq. We recommend setting this in ~/.profile (or equivalent) for proper
> expected behavior.
> bq. Creating a virtualenv for this project…
> bq. Pipfile: /Users/igor_khrol/tmp/airflow-gcp/Pipfile
> bq. Using /usr/local/bin/python3 (3.7.5) to create virtualenv…
> bq. ⠸ Creating virtual environment...Already using interpreter
> /usr/local/opt/python/bin/python3.7
> bq. Using base prefix
> '/usr/local/Cellar/python/3.7.5/Frameworks/Python.framework/Versions/3.7'
> bq. New python executable in
> /Users/igor_khrol/.local/share/virtualenvs/airflow-gcp-7pYi2_I1/bin/python3.7
> bq. Also creating executable in
> /Users/igor_khrol/.local/share/virtualenvs/airflow-gcp-7pYi2_I1/bin/python
> bq. Installing setuptools, pip, wheel...
> bq. done.
> bq. Running virtualenv with interpreter /usr/local/bin/python3
> bq.
> bq. ✔ Successfully created virtual environment!
> bq. Virtualenv location:
> /Users/igor_khrol/.local/share/virtualenvs/airflow-gcp-7pYi2_I1
> bq. Creating a Pipfile for this project…
> bq. Igors-MacBook-Pro:airflow-gcp igor_khrol$ pipenv install
> apache-airflow[gcp]
> bq. Installing apache-airflow[gcp]…
> bq. Adding apache-airflow to Pipfile's [packages]…
> bq. ✔ Installation Succeeded
> bq. Pipfile.lock not found, creating…
> bq. Locking [dev-packages] dependencies…
> bq. Locking [packages] dependencies…
> bq. ✘ Locking Failed!
> bq. [pipenv.exceptions.ResolutionFailure]: File
> "/usr/local/lib/python3.7/site-packages/pipenv/resolver.py", line 69, in
> resolve
> bq. [pipenv.exceptions.ResolutionFailure]: req_dir=requirements_dir
> bq. [pipenv.exceptions.ResolutionFailure]: File
> "/usr/local/lib/python3.7/site-packages/pipenv/utils.py", line 726, in
> resolve_deps
> bq. [pipenv.exceptions.ResolutionFailure]: req_dir=req_dir,
> bq. [pipenv.exceptions.ResolutionFailure]: File
> "/usr/local/lib/python3.7/site-packages/pipenv/utils.py", line 480, in
> actually_resolve_deps
> bq. [pipenv.exceptions.ResolutionFailure]: resolved_tree =
> resolver.resolve()
> bq. [pipenv.exceptions.ResolutionFailure]: File
> "/usr/local/lib/python3.7/site-packages/pipenv/utils.py", line 395, in resolve
> bq. [pipenv.exceptions.ResolutionFailure]: raise
> ResolutionFailure(message=str(e))
> bq. [pipenv.exceptions.ResolutionFailure]:
> pipenv.exceptions.ResolutionFailure: ERROR: ERROR: Could not find a version
> that matches grpc-google-iam-v1<0.12dev,<0.13dev,>=0.11.4,>=0.12.3
> bq. [pipenv.exceptions.ResolutionFailure]: Tried: 0.9.0, 0.10.0,
> 0.10.1, 0.11.1, 0.11.3, 0.11.4, 0.12.0, 0.12.1, 0.12.2, 0.12.3
> bq. [pipenv.exceptions.ResolutionFailure]: Warning: Your dependencies could
> not be resolved. You likely have a mismatch in your sub-dependencies.
> bq. First try clearing your dependency cache with $ pipenv lock --clear,
> then try the original command again.
> bq. Alternatively, you can use $ pipenv install --skip-lock to bypass this
> mechanism, then run $ pipenv graph to inspect the situation.
> bq. Hint: try $ pipenv lock --pre if it is a pre-release dependency.
> bq. ERROR: ERROR: Could not find a version that matches
> grpc-google-iam-v1<0.12dev,<0.13dev,>=0.11.4,>=0.12.3
> bq. Tried: 0.9.0, 0.10.0, 0.10.1, 0.11.1, 0.11.3, 0.11.4, 0.12.0, 0.12.1,
> 0.12.2, 0.12.3
> bq. There are incompatible versions in the resolved dependencies.
> bq. [pipenv.exceptions.ResolutionFailure]: req_dir=requirements_dir
> bq. [pipenv.exceptions.ResolutionFailure]: File
> "/usr/local/lib/python3.7/site-packages/pipenv/utils.py", line 726, in
> resolve_deps
> bq. [pipenv.exceptions.ResolutionFailure]: req_dir=req_dir,
> bq. [pipenv.exceptions.ResolutionFailure]: File
> "/usr/local/lib/python3.7/site-packages/pipenv/utils.py", line 480, in
> actually_resolve_deps
> bq. [pipenv.exceptions.ResolutionFailure]: resolved_tree =
> resolver.resolve()
> bq. [pipenv.exceptions.ResolutionFailure]: File
> "/usr/local/lib/python3.7/site-packages/pipe
[GitHub] [airflow] whynick1 commented on a change in pull request #8090: [AIRFLOW-7118] Honor schema type for mysql to gcs data pre-process
whynick1 commented on a change in pull request #8090: [AIRFLOW-7118] Honor schema type for mysql to gcs data pre-process URL: https://github.com/apache/airflow/pull/8090#discussion_r406601342 ## File path: airflow/providers/google/cloud/operators/mysql_to_gcs.py ## @@ -102,10 +102,12 @@ def field_to_bigquery(self, field): def convert_type(self, value, schema_type): """ Takes a value from MySQLdb, and converts it to a value that's safe for -JSON/Google Cloud Storage/BigQuery. Dates are converted to UTC seconds. -Decimals are converted to floats. Binary type fields are encoded with base64, -as imported BYTES data must be base64-encoded according to Bigquery SQL -date type documentation: https://cloud.google.com/bigquery/data-types +JSON/Google Cloud Storage/BigQuery. Datetimes are converted to UTC seconds. Review comment: Done! Can I get another review please? 😃 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [airflow] whynick1 commented on a change in pull request #8090: [AIRFLOW-7118] Honor schema type for mysql to gcs data pre-process
whynick1 commented on a change in pull request #8090: [AIRFLOW-7118] Honor schema type for mysql to gcs data pre-process URL: https://github.com/apache/airflow/pull/8090#discussion_r406601342 ## File path: airflow/providers/google/cloud/operators/mysql_to_gcs.py ## @@ -102,10 +102,12 @@ def field_to_bigquery(self, field): def convert_type(self, value, schema_type): """ Takes a value from MySQLdb, and converts it to a value that's safe for -JSON/Google Cloud Storage/BigQuery. Dates are converted to UTC seconds. -Decimals are converted to floats. Binary type fields are encoded with base64, -as imported BYTES data must be base64-encoded according to Bigquery SQL -date type documentation: https://cloud.google.com/bigquery/data-types +JSON/Google Cloud Storage/BigQuery. Datetimes are converted to UTC seconds. Review comment: Done! Can I get another review please? 😃 @mik-laj This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [airflow] whynick1 edited a comment on issue #8090: [AIRFLOW-7118] Honor schema type for mysql to gcs data pre-process
whynick1 edited a comment on issue #8090: [AIRFLOW-7118] Honor schema type for mysql to gcs data pre-process URL: https://github.com/apache/airflow/pull/8090#issuecomment-608239076 ~~Testing locally. Please hold.~~ 😃 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [airflow] whynick1 edited a comment on issue #8090: [AIRFLOW-7118] Honor schema type for mysql to gcs data pre-process
whynick1 edited a comment on issue #8090: [AIRFLOW-7118] Honor schema type for mysql to gcs data pre-process URL: https://github.com/apache/airflow/pull/8090#issuecomment-608239076 Testing locally. Please hold. 😃 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [airflow] codecov-io edited a comment on issue #8090: [AIRFLOW-7118] Honor schema type for mysql to gcs data pre-process
codecov-io edited a comment on issue #8090: [AIRFLOW-7118] Honor schema type for mysql to gcs data pre-process URL: https://github.com/apache/airflow/pull/8090#issuecomment-611873167 # [Codecov](https://codecov.io/gh/apache/airflow/pull/8090?src=pr&el=h1) Report > Merging [#8090](https://codecov.io/gh/apache/airflow/pull/8090?src=pr&el=desc) into [master](https://codecov.io/gh/apache/airflow/commit/94480a731d141df4dd7ed91b4b93952d8e07c5c4&el=desc) will **increase** coverage by `0.40%`. > The diff coverage is `92.85%`. [](https://codecov.io/gh/apache/airflow/pull/8090?src=pr&el=tree) ```diff @@Coverage Diff @@ ## master#8090 +/- ## == + Coverage 87.18% 87.58% +0.40% == Files 937 936 -1 Lines 4555645323 -233 == - Hits3971939698 -21 + Misses 5837 5625 -212 ``` | [Impacted Files](https://codecov.io/gh/apache/airflow/pull/8090?src=pr&el=tree) | Coverage Δ | | |---|---|---| | [...w/providers/google/cloud/operators/mysql\_to\_gcs.py](https://codecov.io/gh/apache/airflow/pull/8090/diff?src=pr&el=tree#diff-YWlyZmxvdy9wcm92aWRlcnMvZ29vZ2xlL2Nsb3VkL29wZXJhdG9ycy9teXNxbF90b19nY3MucHk=) | `98.00% <92.85%> (+0.27%)` | :arrow_up: | | [...flow/providers/apache/cassandra/hooks/cassandra.py](https://codecov.io/gh/apache/airflow/pull/8090/diff?src=pr&el=tree#diff-YWlyZmxvdy9wcm92aWRlcnMvYXBhY2hlL2Nhc3NhbmRyYS9ob29rcy9jYXNzYW5kcmEucHk=) | `21.25% <0.00%> (-72.50%)` | :arrow_down: | | [...w/providers/apache/hive/operators/mysql\_to\_hive.py](https://codecov.io/gh/apache/airflow/pull/8090/diff?src=pr&el=tree#diff-YWlyZmxvdy9wcm92aWRlcnMvYXBhY2hlL2hpdmUvb3BlcmF0b3JzL215c3FsX3RvX2hpdmUucHk=) | `35.84% <0.00%> (-64.16%)` | :arrow_down: | | [airflow/kubernetes/volume\_mount.py](https://codecov.io/gh/apache/airflow/pull/8090/diff?src=pr&el=tree#diff-YWlyZmxvdy9rdWJlcm5ldGVzL3ZvbHVtZV9tb3VudC5weQ==) | `44.44% <0.00%> (-55.56%)` | :arrow_down: | | [airflow/providers/postgres/operators/postgres.py](https://codecov.io/gh/apache/airflow/pull/8090/diff?src=pr&el=tree#diff-YWlyZmxvdy9wcm92aWRlcnMvcG9zdGdyZXMvb3BlcmF0b3JzL3Bvc3RncmVzLnB5) | `47.82% <0.00%> (-52.18%)` | :arrow_down: | | [airflow/providers/redis/operators/redis\_publish.py](https://codecov.io/gh/apache/airflow/pull/8090/diff?src=pr&el=tree#diff-YWlyZmxvdy9wcm92aWRlcnMvcmVkaXMvb3BlcmF0b3JzL3JlZGlzX3B1Ymxpc2gucHk=) | `50.00% <0.00%> (-50.00%)` | :arrow_down: | | [airflow/kubernetes/volume.py](https://codecov.io/gh/apache/airflow/pull/8090/diff?src=pr&el=tree#diff-YWlyZmxvdy9rdWJlcm5ldGVzL3ZvbHVtZS5weQ==) | `52.94% <0.00%> (-47.06%)` | :arrow_down: | | [airflow/providers/mongo/sensors/mongo.py](https://codecov.io/gh/apache/airflow/pull/8090/diff?src=pr&el=tree#diff-YWlyZmxvdy9wcm92aWRlcnMvbW9uZ28vc2Vuc29ycy9tb25nby5weQ==) | `53.33% <0.00%> (-46.67%)` | :arrow_down: | | [airflow/kubernetes/pod\_launcher.py](https://codecov.io/gh/apache/airflow/pull/8090/diff?src=pr&el=tree#diff-YWlyZmxvdy9rdWJlcm5ldGVzL3BvZF9sYXVuY2hlci5weQ==) | `47.18% <0.00%> (-45.08%)` | :arrow_down: | | [airflow/providers/mysql/operators/mysql.py](https://codecov.io/gh/apache/airflow/pull/8090/diff?src=pr&el=tree#diff-YWlyZmxvdy9wcm92aWRlcnMvbXlzcWwvb3BlcmF0b3JzL215c3FsLnB5) | `55.00% <0.00%> (-45.00%)` | :arrow_down: | | ... and [78 more](https://codecov.io/gh/apache/airflow/pull/8090/diff?src=pr&el=tree-more) | | -- [Continue to review full report at Codecov](https://codecov.io/gh/apache/airflow/pull/8090?src=pr&el=continue). > **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta) > `Δ = absolute (impact)`, `ø = not affected`, `? = missing data` > Powered by [Codecov](https://codecov.io/gh/apache/airflow/pull/8090?src=pr&el=footer). Last update [94480a7...7a87782](https://codecov.io/gh/apache/airflow/pull/8090?src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments). This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [airflow] codecov-io edited a comment on issue #8090: [AIRFLOW-7118] Honor schema type for mysql to gcs data pre-process
codecov-io edited a comment on issue #8090: [AIRFLOW-7118] Honor schema type for mysql to gcs data pre-process URL: https://github.com/apache/airflow/pull/8090#issuecomment-611873167 # [Codecov](https://codecov.io/gh/apache/airflow/pull/8090?src=pr&el=h1) Report > Merging [#8090](https://codecov.io/gh/apache/airflow/pull/8090?src=pr&el=desc) into [master](https://codecov.io/gh/apache/airflow/commit/94480a731d141df4dd7ed91b4b93952d8e07c5c4&el=desc) will **increase** coverage by `0.40%`. > The diff coverage is `92.85%`. [](https://codecov.io/gh/apache/airflow/pull/8090?src=pr&el=tree) ```diff @@Coverage Diff @@ ## master#8090 +/- ## == + Coverage 87.18% 87.58% +0.40% == Files 937 936 -1 Lines 4555645323 -233 == - Hits3971939698 -21 + Misses 5837 5625 -212 ``` | [Impacted Files](https://codecov.io/gh/apache/airflow/pull/8090?src=pr&el=tree) | Coverage Δ | | |---|---|---| | [...w/providers/google/cloud/operators/mysql\_to\_gcs.py](https://codecov.io/gh/apache/airflow/pull/8090/diff?src=pr&el=tree#diff-YWlyZmxvdy9wcm92aWRlcnMvZ29vZ2xlL2Nsb3VkL29wZXJhdG9ycy9teXNxbF90b19nY3MucHk=) | `98.00% <92.85%> (+0.27%)` | :arrow_up: | | [...flow/providers/apache/cassandra/hooks/cassandra.py](https://codecov.io/gh/apache/airflow/pull/8090/diff?src=pr&el=tree#diff-YWlyZmxvdy9wcm92aWRlcnMvYXBhY2hlL2Nhc3NhbmRyYS9ob29rcy9jYXNzYW5kcmEucHk=) | `21.25% <0.00%> (-72.50%)` | :arrow_down: | | [...w/providers/apache/hive/operators/mysql\_to\_hive.py](https://codecov.io/gh/apache/airflow/pull/8090/diff?src=pr&el=tree#diff-YWlyZmxvdy9wcm92aWRlcnMvYXBhY2hlL2hpdmUvb3BlcmF0b3JzL215c3FsX3RvX2hpdmUucHk=) | `35.84% <0.00%> (-64.16%)` | :arrow_down: | | [airflow/kubernetes/volume\_mount.py](https://codecov.io/gh/apache/airflow/pull/8090/diff?src=pr&el=tree#diff-YWlyZmxvdy9rdWJlcm5ldGVzL3ZvbHVtZV9tb3VudC5weQ==) | `44.44% <0.00%> (-55.56%)` | :arrow_down: | | [airflow/providers/postgres/operators/postgres.py](https://codecov.io/gh/apache/airflow/pull/8090/diff?src=pr&el=tree#diff-YWlyZmxvdy9wcm92aWRlcnMvcG9zdGdyZXMvb3BlcmF0b3JzL3Bvc3RncmVzLnB5) | `47.82% <0.00%> (-52.18%)` | :arrow_down: | | [airflow/providers/redis/operators/redis\_publish.py](https://codecov.io/gh/apache/airflow/pull/8090/diff?src=pr&el=tree#diff-YWlyZmxvdy9wcm92aWRlcnMvcmVkaXMvb3BlcmF0b3JzL3JlZGlzX3B1Ymxpc2gucHk=) | `50.00% <0.00%> (-50.00%)` | :arrow_down: | | [airflow/kubernetes/volume.py](https://codecov.io/gh/apache/airflow/pull/8090/diff?src=pr&el=tree#diff-YWlyZmxvdy9rdWJlcm5ldGVzL3ZvbHVtZS5weQ==) | `52.94% <0.00%> (-47.06%)` | :arrow_down: | | [airflow/providers/mongo/sensors/mongo.py](https://codecov.io/gh/apache/airflow/pull/8090/diff?src=pr&el=tree#diff-YWlyZmxvdy9wcm92aWRlcnMvbW9uZ28vc2Vuc29ycy9tb25nby5weQ==) | `53.33% <0.00%> (-46.67%)` | :arrow_down: | | [airflow/kubernetes/pod\_launcher.py](https://codecov.io/gh/apache/airflow/pull/8090/diff?src=pr&el=tree#diff-YWlyZmxvdy9rdWJlcm5ldGVzL3BvZF9sYXVuY2hlci5weQ==) | `47.18% <0.00%> (-45.08%)` | :arrow_down: | | [airflow/providers/mysql/operators/mysql.py](https://codecov.io/gh/apache/airflow/pull/8090/diff?src=pr&el=tree#diff-YWlyZmxvdy9wcm92aWRlcnMvbXlzcWwvb3BlcmF0b3JzL215c3FsLnB5) | `55.00% <0.00%> (-45.00%)` | :arrow_down: | | ... and [78 more](https://codecov.io/gh/apache/airflow/pull/8090/diff?src=pr&el=tree-more) | | -- [Continue to review full report at Codecov](https://codecov.io/gh/apache/airflow/pull/8090?src=pr&el=continue). > **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta) > `Δ = absolute (impact)`, `ø = not affected`, `? = missing data` > Powered by [Codecov](https://codecov.io/gh/apache/airflow/pull/8090?src=pr&el=footer). Last update [94480a7...7a87782](https://codecov.io/gh/apache/airflow/pull/8090?src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments). This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [airflow] codecov-io commented on issue #8090: [AIRFLOW-7118] Honor schema type for mysql to gcs data pre-process
codecov-io commented on issue #8090: [AIRFLOW-7118] Honor schema type for mysql to gcs data pre-process URL: https://github.com/apache/airflow/pull/8090#issuecomment-611873167 # [Codecov](https://codecov.io/gh/apache/airflow/pull/8090?src=pr&el=h1) Report > Merging [#8090](https://codecov.io/gh/apache/airflow/pull/8090?src=pr&el=desc) into [master](https://codecov.io/gh/apache/airflow/commit/94480a731d141df4dd7ed91b4b93952d8e07c5c4&el=desc) will **increase** coverage by `0.40%`. > The diff coverage is `92.85%`. [](https://codecov.io/gh/apache/airflow/pull/8090?src=pr&el=tree) ```diff @@Coverage Diff @@ ## master#8090 +/- ## == + Coverage 87.18% 87.58% +0.40% == Files 937 936 -1 Lines 4555645323 -233 == - Hits3971939698 -21 + Misses 5837 5625 -212 ``` | [Impacted Files](https://codecov.io/gh/apache/airflow/pull/8090?src=pr&el=tree) | Coverage Δ | | |---|---|---| | [...w/providers/google/cloud/operators/mysql\_to\_gcs.py](https://codecov.io/gh/apache/airflow/pull/8090/diff?src=pr&el=tree#diff-YWlyZmxvdy9wcm92aWRlcnMvZ29vZ2xlL2Nsb3VkL29wZXJhdG9ycy9teXNxbF90b19nY3MucHk=) | `98.00% <92.85%> (+0.27%)` | :arrow_up: | | [...flow/providers/apache/cassandra/hooks/cassandra.py](https://codecov.io/gh/apache/airflow/pull/8090/diff?src=pr&el=tree#diff-YWlyZmxvdy9wcm92aWRlcnMvYXBhY2hlL2Nhc3NhbmRyYS9ob29rcy9jYXNzYW5kcmEucHk=) | `21.25% <0.00%> (-72.50%)` | :arrow_down: | | [...w/providers/apache/hive/operators/mysql\_to\_hive.py](https://codecov.io/gh/apache/airflow/pull/8090/diff?src=pr&el=tree#diff-YWlyZmxvdy9wcm92aWRlcnMvYXBhY2hlL2hpdmUvb3BlcmF0b3JzL215c3FsX3RvX2hpdmUucHk=) | `35.84% <0.00%> (-64.16%)` | :arrow_down: | | [airflow/kubernetes/volume\_mount.py](https://codecov.io/gh/apache/airflow/pull/8090/diff?src=pr&el=tree#diff-YWlyZmxvdy9rdWJlcm5ldGVzL3ZvbHVtZV9tb3VudC5weQ==) | `44.44% <0.00%> (-55.56%)` | :arrow_down: | | [airflow/providers/postgres/operators/postgres.py](https://codecov.io/gh/apache/airflow/pull/8090/diff?src=pr&el=tree#diff-YWlyZmxvdy9wcm92aWRlcnMvcG9zdGdyZXMvb3BlcmF0b3JzL3Bvc3RncmVzLnB5) | `47.82% <0.00%> (-52.18%)` | :arrow_down: | | [airflow/providers/redis/operators/redis\_publish.py](https://codecov.io/gh/apache/airflow/pull/8090/diff?src=pr&el=tree#diff-YWlyZmxvdy9wcm92aWRlcnMvcmVkaXMvb3BlcmF0b3JzL3JlZGlzX3B1Ymxpc2gucHk=) | `50.00% <0.00%> (-50.00%)` | :arrow_down: | | [airflow/kubernetes/volume.py](https://codecov.io/gh/apache/airflow/pull/8090/diff?src=pr&el=tree#diff-YWlyZmxvdy9rdWJlcm5ldGVzL3ZvbHVtZS5weQ==) | `52.94% <0.00%> (-47.06%)` | :arrow_down: | | [airflow/providers/mongo/sensors/mongo.py](https://codecov.io/gh/apache/airflow/pull/8090/diff?src=pr&el=tree#diff-YWlyZmxvdy9wcm92aWRlcnMvbW9uZ28vc2Vuc29ycy9tb25nby5weQ==) | `53.33% <0.00%> (-46.67%)` | :arrow_down: | | [airflow/kubernetes/pod\_launcher.py](https://codecov.io/gh/apache/airflow/pull/8090/diff?src=pr&el=tree#diff-YWlyZmxvdy9rdWJlcm5ldGVzL3BvZF9sYXVuY2hlci5weQ==) | `47.18% <0.00%> (-45.08%)` | :arrow_down: | | [airflow/providers/mysql/operators/mysql.py](https://codecov.io/gh/apache/airflow/pull/8090/diff?src=pr&el=tree#diff-YWlyZmxvdy9wcm92aWRlcnMvbXlzcWwvb3BlcmF0b3JzL215c3FsLnB5) | `55.00% <0.00%> (-45.00%)` | :arrow_down: | | ... and [78 more](https://codecov.io/gh/apache/airflow/pull/8090/diff?src=pr&el=tree-more) | | -- [Continue to review full report at Codecov](https://codecov.io/gh/apache/airflow/pull/8090?src=pr&el=continue). > **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta) > `Δ = absolute (impact)`, `ø = not affected`, `? = missing data` > Powered by [Codecov](https://codecov.io/gh/apache/airflow/pull/8090?src=pr&el=footer). Last update [94480a7...7a87782](https://codecov.io/gh/apache/airflow/pull/8090?src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments). This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [airflow] vanducng commented on issue #8046: [AIRFLOW-7048] Allow user to chose timezone to use in UI
vanducng commented on issue #8046: [AIRFLOW-7048] Allow user to chose timezone to use in UI URL: https://github.com/apache/airflow/pull/8046#issuecomment-611872084 Im waiting this feature quite so long (though it would be available in v2), thank you so much @ashb @kaxil  This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Commented] (AIRFLOW-6266) Asserts in Test view should give more information
[ https://issues.apache.org/jira/browse/AIRFLOW-6266?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17080191#comment-17080191 ] ASF GitHub Bot commented on AIRFLOW-6266: - stale[bot] commented on pull request #6837: [AIRFLOW-6266] Better asserts in test_views.py URL: https://github.com/apache/airflow/pull/6837 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org > Asserts in Test view should give more information > - > > Key: AIRFLOW-6266 > URL: https://issues.apache.org/jira/browse/AIRFLOW-6266 > Project: Apache Airflow > Issue Type: Improvement > Components: ci >Affects Versions: 2.0.0 >Reporter: Jarek Potiuk >Priority: Major > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[GitHub] [airflow] stale[bot] closed pull request #6837: [AIRFLOW-6266] Better asserts in test_views.py
stale[bot] closed pull request #6837: [AIRFLOW-6266] Better asserts in test_views.py URL: https://github.com/apache/airflow/pull/6837 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [airflow] bcjonescbt commented on issue #8046: [AIRFLOW-7048] Allow user to chose timezone to use in UI
bcjonescbt commented on issue #8046: [AIRFLOW-7048] Allow user to chose timezone to use in UI URL: https://github.com/apache/airflow/pull/8046#issuecomment-611851627 Ah - it's a brave new world. Thanks for the pointer. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [airflow] jithin97 commented on issue #7431: [AIRFLOW-6808]: Add GCP GKE Operators and Hooks
jithin97 commented on issue #7431: [AIRFLOW-6808]: Add GCP GKE Operators and Hooks URL: https://github.com/apache/airflow/pull/7431#issuecomment-611835662 > @jithin97 Do you plan to continue working on it? Do you need any support? > > I don't know if you like cats, but I love them. To encourage you to continue working on this PR, I am sending you a beautiful cat. I hope you enjoy it. >  > CC: @mschickensoup @mik-laj Thanks for the motivation. I got sidetracked a little bit. I would continue to work from now 🙇♂️ This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [airflow] kaxil commented on issue #8046: [AIRFLOW-7048] Allow user to chose timezone to use in UI
kaxil commented on issue #8046: [AIRFLOW-7048] Allow user to chose timezone to use in UI URL: https://github.com/apache/airflow/pull/8046#issuecomment-611806912 > My entire header row is different than the picture in this PR. I don't have the pretty icons, and I don't have a user on the right. The menu items don't match either. Is there some config setting that I need to keep up with to get these new features? > >  Please enable RBAC UI - https://airflow.apache.org/blog/airflow-1.10.8-1.10.9/#use-airflow-rbac-ui to use that feature. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [airflow] bcjonescbt commented on issue #8046: [AIRFLOW-7048] Allow user to chose timezone to use in UI
bcjonescbt commented on issue #8046: [AIRFLOW-7048] Allow user to chose timezone to use in UI URL: https://github.com/apache/airflow/pull/8046#issuecomment-611806238 My entire header row is different than the picture in this PR. I don't have the pretty icons, and I don't have a user on the right. The menu items don't match either. Is there some config setting that I need to keep up with to get these new features?  This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [airflow] bcjonescbt commented on issue #8046: [AIRFLOW-7048] Allow user to chose timezone to use in UI
bcjonescbt commented on issue #8046: [AIRFLOW-7048] Allow user to chose timezone to use in UI URL: https://github.com/apache/airflow/pull/8046#issuecomment-611804869 I upgraded to version 1.10.10 but I don't see the new GUI tooltips in the upper right. I added the default_ui_timezone setting to my config file, but still no luck. It looks the same as it ever did (just showing UTC). Do I have to burn down my database to get this new feature? This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Commented] (AIRFLOW-6207) Dag run twice in airflow
[ https://issues.apache.org/jira/browse/AIRFLOW-6207?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17080082#comment-17080082 ] Sam Rogers commented on AIRFLOW-6207: - Thanks for your response, I will give it a try. > Dag run twice in airflow > > > Key: AIRFLOW-6207 > URL: https://issues.apache.org/jira/browse/AIRFLOW-6207 > Project: Apache Airflow > Issue Type: Bug > Components: DagRun >Affects Versions: 1.10.6 >Reporter: anilkumar >Priority: Critical > Attachments: airflow1.PNG, airflow10.png, airflow11.PNG, > airflow2.PNG, airflow3.PNG, airflow4.PNG, airflow5.PNG, airflow6.PNG, > airflow7.PNG, airflow8.PNG, airflow9.PNG > > > [!https://user-images.githubusercontent.com/26398961/70459068-67356780-1ad9-11ea-86a8-44d1bbfb574d.png!|https://user-images.githubusercontent.com/26398961/70459068-67356780-1ad9-11ea-86a8-44d1bbfb574d.png] > As you can see the dag xyz_dag run twice a day and XYZ dag is the scheduled > dag, not manual trigger for the first run it runs at 5:10:16 and the second > run it ran at 5:10:58 similarly this behavior has been observed for my all > 400 dags. I don't know why this behavior has occurred and I don't know how > this can be solved any help will be appreciated. below I have shared > xyx_dag.py file. > [!https://user-images.githubusercontent.com/26398961/70460387-30ad1c00-1adc-11ea-8241-ce3dd6d7556c.png!|https://user-images.githubusercontent.com/26398961/70460387-30ad1c00-1adc-11ea-8241-ce3dd6d7556c.png] -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (AIRFLOW-6207) Dag run twice in airflow
[ https://issues.apache.org/jira/browse/AIRFLOW-6207?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17080078#comment-17080078 ] anilkumar commented on AIRFLOW-6207: Issue occured Due to cron expression don't put 6 digit cron expression try with five digit. In my sceneraio I have just change the cron expression and every thing works for me. > Dag run twice in airflow > > > Key: AIRFLOW-6207 > URL: https://issues.apache.org/jira/browse/AIRFLOW-6207 > Project: Apache Airflow > Issue Type: Bug > Components: DagRun >Affects Versions: 1.10.6 >Reporter: anilkumar >Priority: Critical > Attachments: airflow1.PNG, airflow10.png, airflow11.PNG, > airflow2.PNG, airflow3.PNG, airflow4.PNG, airflow5.PNG, airflow6.PNG, > airflow7.PNG, airflow8.PNG, airflow9.PNG > > > [!https://user-images.githubusercontent.com/26398961/70459068-67356780-1ad9-11ea-86a8-44d1bbfb574d.png!|https://user-images.githubusercontent.com/26398961/70459068-67356780-1ad9-11ea-86a8-44d1bbfb574d.png] > As you can see the dag xyz_dag run twice a day and XYZ dag is the scheduled > dag, not manual trigger for the first run it runs at 5:10:16 and the second > run it ran at 5:10:58 similarly this behavior has been observed for my all > 400 dags. I don't know why this behavior has occurred and I don't know how > this can be solved any help will be appreciated. below I have shared > xyx_dag.py file. > [!https://user-images.githubusercontent.com/26398961/70460387-30ad1c00-1adc-11ea-8241-ce3dd6d7556c.png!|https://user-images.githubusercontent.com/26398961/70460387-30ad1c00-1adc-11ea-8241-ce3dd6d7556c.png] -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (AIRFLOW-6207) Dag run twice in airflow
[ https://issues.apache.org/jira/browse/AIRFLOW-6207?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17080070#comment-17080070 ] Sam Rogers commented on AIRFLOW-6207: - Hi [~anilkumar13] were you ever able to find the cause of this problem? Running into a seemingly very similar bug myself. > Dag run twice in airflow > > > Key: AIRFLOW-6207 > URL: https://issues.apache.org/jira/browse/AIRFLOW-6207 > Project: Apache Airflow > Issue Type: Bug > Components: DagRun >Affects Versions: 1.10.6 >Reporter: anilkumar >Priority: Critical > Attachments: airflow1.PNG, airflow10.png, airflow11.PNG, > airflow2.PNG, airflow3.PNG, airflow4.PNG, airflow5.PNG, airflow6.PNG, > airflow7.PNG, airflow8.PNG, airflow9.PNG > > > [!https://user-images.githubusercontent.com/26398961/70459068-67356780-1ad9-11ea-86a8-44d1bbfb574d.png!|https://user-images.githubusercontent.com/26398961/70459068-67356780-1ad9-11ea-86a8-44d1bbfb574d.png] > As you can see the dag xyz_dag run twice a day and XYZ dag is the scheduled > dag, not manual trigger for the first run it runs at 5:10:16 and the second > run it ran at 5:10:58 similarly this behavior has been observed for my all > 400 dags. I don't know why this behavior has occurred and I don't know how > this can be solved any help will be appreciated. below I have shared > xyx_dag.py file. > [!https://user-images.githubusercontent.com/26398961/70460387-30ad1c00-1adc-11ea-8241-ce3dd6d7556c.png!|https://user-images.githubusercontent.com/26398961/70460387-30ad1c00-1adc-11ea-8241-ce3dd6d7556c.png] -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Resolved] (AIRFLOW-7117) Honor self.schema in sql_to_gcs as schema to upload
[ https://issues.apache.org/jira/browse/AIRFLOW-7117?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hongyi Wang resolved AIRFLOW-7117. -- Resolution: Fixed > Honor self.schema in sql_to_gcs as schema to upload > --- > > Key: AIRFLOW-7117 > URL: https://issues.apache.org/jira/browse/AIRFLOW-7117 > Project: Apache Airflow > Issue Type: Improvement > Components: operators >Affects Versions: 1.10.5, 1.10.6, 1.10.7, 1.10.8, 1.10.9 >Reporter: Hongyi Wang >Assignee: Hongyi Wang >Priority: Minor > > In > [sql_to_gcs|https://github.com/apache/airflow/blob/master/airflow/providers/google/cloud/operators/sql_to_gcs.py], > there is a > [self.schema|https://github.com/apache/airflow/blob/master/airflow/providers/google/cloud/operators/sql_to_gcs.py#L61-L65], > which was introduced in > [pr-3091|https://github.com/apache/airflow/pull/3091/files] to allow user to > provide their own schema while loading from sql to Google BigQuery. > This is very useful when user want custom type mapping. For example, > [mysql.date is default to map > bigquery.TIMESTAMP|https://github.com/apache/airflow/blob/master/airflow/providers/google/cloud/operators/mysql_to_gcs.py#L49], > but sometimes, people want it to be bigquery.DATE. > However, during > [mysql_to_gcs|https://github.com/apache/airflow/blob/master/airflow/providers/google/cloud/operators/mysql_to_gcs.py] > re-organization, we lost that ability. (Only [use self.schema for > pre-processing|https://github.com/apache/airflow/blob/master/airflow/providers/google/cloud/operators/sql_to_gcs.py#L229-L232], > while data type is no longer configurable). > As a conclusion, we need to honor self.schema (if provided) to override > default schema, like we did before. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (AIRFLOW-7117) Honor self.schema in sql_to_gcs as schema to upload
[ https://issues.apache.org/jira/browse/AIRFLOW-7117?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17080064#comment-17080064 ] Hongyi Wang commented on AIRFLOW-7117: -- Issue solved with [GitHub Pull Request #8049|https://github.com/apache/airflow/pull/8049] > Honor self.schema in sql_to_gcs as schema to upload > --- > > Key: AIRFLOW-7117 > URL: https://issues.apache.org/jira/browse/AIRFLOW-7117 > Project: Apache Airflow > Issue Type: Improvement > Components: operators >Affects Versions: 1.10.5, 1.10.6, 1.10.7, 1.10.8, 1.10.9 >Reporter: Hongyi Wang >Assignee: Hongyi Wang >Priority: Minor > > In > [sql_to_gcs|https://github.com/apache/airflow/blob/master/airflow/providers/google/cloud/operators/sql_to_gcs.py], > there is a > [self.schema|https://github.com/apache/airflow/blob/master/airflow/providers/google/cloud/operators/sql_to_gcs.py#L61-L65], > which was introduced in > [pr-3091|https://github.com/apache/airflow/pull/3091/files] to allow user to > provide their own schema while loading from sql to Google BigQuery. > This is very useful when user want custom type mapping. For example, > [mysql.date is default to map > bigquery.TIMESTAMP|https://github.com/apache/airflow/blob/master/airflow/providers/google/cloud/operators/mysql_to_gcs.py#L49], > but sometimes, people want it to be bigquery.DATE. > However, during > [mysql_to_gcs|https://github.com/apache/airflow/blob/master/airflow/providers/google/cloud/operators/mysql_to_gcs.py] > re-organization, we lost that ability. (Only [use self.schema for > pre-processing|https://github.com/apache/airflow/blob/master/airflow/providers/google/cloud/operators/sql_to_gcs.py#L229-L232], > while data type is no longer configurable). > As a conclusion, we need to honor self.schema (if provided) to override > default schema, like we did before. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[GitHub] [airflow] madison-ookla commented on issue #7364: [AIRFLOW-6728] Change various DAG info methods to POST
madison-ookla commented on issue #7364: [AIRFLOW-6728] Change various DAG info methods to POST URL: https://github.com/apache/airflow/pull/7364#issuecomment-611795797 > @madison-ookla And included in 1.10.10 which was just released :) Very excited about this release! 😁 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [airflow] madison-ookla edited a comment on issue #7364: [AIRFLOW-6728] Change various DAG info methods to POST
madison-ookla edited a comment on issue #7364: [AIRFLOW-6728] Change various DAG info methods to POST URL: https://github.com/apache/airflow/pull/7364#issuecomment-611795797 > @madison-ookla And included in 1.10.10 which was just released :) Thanks, very excited about this release! 😁 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [airflow] ashb commented on issue #7364: [AIRFLOW-6728] Change various DAG info methods to POST
ashb commented on issue #7364: [AIRFLOW-6728] Change various DAG info methods to POST URL: https://github.com/apache/airflow/pull/7364#issuecomment-611793964 @madison-ookla And included in 1.10.10 which was just released :) This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Commented] (AIRFLOW-1455) Move logging related configs out of airflow.cfg
[
https://issues.apache.org/jira/browse/AIRFLOW-1455?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17080048#comment-17080048
]
Amith Gopal commented on AIRFLOW-1455:
--
[~vait] [~allisonwang]
i am trying to set up custom loggin for Azure blob storage and I have specified
all the configs in my custom log_config.py
But it is not even parsing any of the logs nor it is pushing the logs to azure
blob storage.
Can you please help ?
> Move logging related configs out of airflow.cfg
> ---
>
> Key: AIRFLOW-1455
> URL: https://issues.apache.org/jira/browse/AIRFLOW-1455
> Project: Apache Airflow
> Issue Type: Sub-task
>Reporter: Allison Wang
>Assignee: Igors Vaitkus
>Priority: Major
>
> All logging related configurations including {{LOG_BASE_FOLDER}},
> {{REMOTE_LOG_BASE_FOLDER}}, {{LOG_LEVEL}} and {{LOG_FORMAT}} should be placed
> inside {{default_airflow_logging}}. This task also includes refactoring all
> occurrence of those variables and make them handler specific rather than
> global.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[GitHub] [airflow] kaxil merged pull request #8236: Fix stylelint violation
kaxil merged pull request #8236: Fix stylelint violation URL: https://github.com/apache/airflow/pull/8236 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [airflow] ryanahamilton opened a new pull request #8236: Fix stylelint violation
ryanahamilton opened a new pull request #8236: Fix stylelint violation URL: https://github.com/apache/airflow/pull/8236 --- Make sure to mark the boxes below before creating PR: [x] - [x] Description above provides context of the change - [x] Unit tests coverage for changes (not needed for documentation changes) - [x] Commits follow "[How to write a good git commit message](http://chris.beams.io/posts/git-commit/)" - [x] Relevant documentation is updated including usage instructions. - [x] I will engage committers as explained in [Contribution Workflow Example](https://github.com/apache/airflow/blob/master/CONTRIBUTING.rst#contribution-workflow-example). --- In case of fundamental code change, Airflow Improvement Proposal ([AIP](https://cwiki.apache.org/confluence/display/AIRFLOW/Airflow+Improvements+Proposals)) is needed. In case of a new dependency, check compliance with the [ASF 3rd Party License Policy](https://www.apache.org/legal/resolved.html#category-x). In case of backwards incompatible changes please leave a note in [UPDATING.md](https://github.com/apache/airflow/blob/master/UPDATING.md). Read the [Pull Request Guidelines](https://github.com/apache/airflow/blob/master/CONTRIBUTING.rst#pull-request-guidelines) for more information. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [airflow] stale[bot] commented on issue #7119: [AIRFLOW-5840] Add operator extra link to external task sensor
stale[bot] commented on issue #7119: [AIRFLOW-5840] Add operator extra link to external task sensor URL: https://github.com/apache/airflow/pull/7119#issuecomment-611787068 This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [airflow] kaxil opened a new pull request #8235: Add Changelog and Updating note for 1.10.10
kaxil opened a new pull request #8235: Add Changelog and Updating note for 1.10.10 URL: https://github.com/apache/airflow/pull/8235 Add Changelog and Updating note for 1.10.10 --- Make sure to mark the boxes below before creating PR: [x] - [x] Description above provides context of the change - [x] Unit tests coverage for changes (not needed for documentation changes) - [x] Commits follow "[How to write a good git commit message](http://chris.beams.io/posts/git-commit/)" - [x] Relevant documentation is updated including usage instructions. - [x] I will engage committers as explained in [Contribution Workflow Example](https://github.com/apache/airflow/blob/master/CONTRIBUTING.rst#contribution-workflow-example). --- In case of fundamental code change, Airflow Improvement Proposal ([AIP](https://cwiki.apache.org/confluence/display/AIRFLOW/Airflow+Improvements+Proposals)) is needed. In case of a new dependency, check compliance with the [ASF 3rd Party License Policy](https://www.apache.org/legal/resolved.html#category-x). In case of backwards incompatible changes please leave a note in [UPDATING.md](https://github.com/apache/airflow/blob/master/UPDATING.md). Read the [Pull Request Guidelines](https://github.com/apache/airflow/blob/master/CONTRIBUTING.rst#pull-request-guidelines) for more information. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Commented] (AIRFLOW-6515) log level of INFO/WARN when ERROR happened
[
https://issues.apache.org/jira/browse/AIRFLOW-6515?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17080025#comment-17080025
]
ASF subversion and git services commented on AIRFLOW-6515:
--
Commit 87969a350ddd41e9e6af6d780b31e363eaca in airflow's branch
refs/heads/master from William Hudgins
[ https://gitbox.apache.org/repos/asf?p=airflow.git;h=87969a3 ]
[AIRFLOW-6515] Change Log Levels from Info/Warn to Error (#8170)
> log level of INFO/WARN when ERROR happened
> --
>
> Key: AIRFLOW-6515
> URL: https://issues.apache.org/jira/browse/AIRFLOW-6515
> Project: Apache Airflow
> Issue Type: Bug
> Components: logging
>Affects Versions: 1.10.7
>Reporter: t oo
>Assignee: Will Hudgins
>Priority: Major
> Fix For: 1.10.11
>
>
> log level should be error on some (but there are false positives):
> grep -iE
> 'log\.(info|warn).*(error|exceptio|fail|unab|couldn|lost|gone|missing|not
> fou|abort|exit|could not)' -R *
> airflow/sensors/base_sensor_operator.py:self.log.info("Success
> criteria met. Exiting.")
> airflow/logging_config.py:log.warning('Unable to load the config,
> contains a configuration error.')
> airflow/operators/check_operator.py:self.log.warning("The
> following %s tests out of %s failed:", j, n)
> airflow/operators/sql_to_gcs.py:self.log.warning('Using default
> schema due to missing name or type. Please '
> airflow/operators/bash_operator.py:self.log.info('Command exited
> with return code %s', self.sub_process.returncode)
> airflow/serialization/serialized_objects.py:LOG.warning('Failed
> to stringify.', exc_info=True)
> airflow/providers/amazon/aws/operators/batch.py:
> self.log.info("AWS Batch Job has failed")
> airflow/providers/amazon/aws/hooks/s3.py:
> self.log.info(e.response["Error"]["Message"])
> airflow/providers/amazon/aws/hooks/s3.py:
> self.log.info(e.response["Error"]["Message"])
> airflow/utils/dag_processing.py:self.log.info("Exiting gracefully
> upon receiving signal %s", signum)
> airflow/utils/dag_processing.py:self.log.info("Exiting dag
> parsing loop as all files "
> airflow/utils/dag_processing.py:self.log.info("Failing jobs
> without heartbeat after %s", limit_dttm)
> airflow/utils/dag_processing.py:self.log.info("Waiting up to %s
> seconds for processes to exit...", timeout)
> airflow/utils/helpers.py:log.info("Process %s (%s) terminated with
> exit code %s", p, p.pid, p.returncode)
> airflow/models/dagrun.py:self.log.info('Marking run %s failed',
> self)
> airflow/models/dagrun.py:self.log.info('Deadlock; marking run %s
> failed', self)
> airflow/models/dagrun.py:self.log.warning("Failed to get
> task '{}' for dag '{}'. "
> airflow/gcp/sensors/gcs.py:self.log.warning("FAILURE: Inactivity
> Period passed, not enough objects found in %s", path)
> airflow/gcp/operators/spanner.py:self.log.info("The Cloud Spanner
> database was missing: "
> airflow/gcp/hooks/kubernetes_engine.py:self.log.info('Assuming
> Success: %s', error.message)
> airflow/gcp/hooks/kubernetes_engine.py:self.log.info('Assuming
> Success: %s', error.message)
> airflow/gcp/hooks/cloud_memorystore.py:self.log.info("Failovering
> Instance: %s", name)
> airflow/gcp/hooks/cloud_memorystore.py:self.log.info("Instance
> failovered: %s", name)
> airflow/gcp/hooks/bigquery.py:self.log.info(error_msg)
> airflow/gcp/hooks/bigtable.py:self.log.info("The instance '%s'
> does not exist in project '%s'. Exiting", instance_id,
> airflow/contrib/sensors/bash_sensor.py:self.log.info("Command
> exited with return code %s", sp.returncode)
> airflow/contrib/sensors/ftp_sensor.py:self.log.info('Ftp
> error encountered: %s', str(e))
> airflow/contrib/operators/azure_container_instances_operator.py:
> self.log.info("Container had exit code: %s", exit_code)
> airflow/contrib/operators/azure_container_instances_operator.py:
> self.log.info("Container exited with detail_status %s", detail_status)
> airflow/contrib/operators/azure_container_instances_operator.py:
> self.log.info("Azure provision failure")
> airflow/contrib/operators/winrm_operator.py:self.log.info("Hook
> not found, creating...")
> airflow/contrib/operators/docker_swarm_operator.py:
> self.log.info('Service status before exiting: %s', status)
> airflow/contrib/auth/backends/ldap_auth.py:log.warning("Unable to
> find group for %s %s", search_base, search_filter)
> airflow/contrib/auth
[jira] [Commented] (AIRFLOW-6515) log level of INFO/WARN when ERROR happened
[
https://issues.apache.org/jira/browse/AIRFLOW-6515?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17080024#comment-17080024
]
ASF GitHub Bot commented on AIRFLOW-6515:
-
kaxil commented on pull request #8170: [AIRFLOW-6515] Change Log Levels from
Info/Warn to Error
URL: https://github.com/apache/airflow/pull/8170
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
> log level of INFO/WARN when ERROR happened
> --
>
> Key: AIRFLOW-6515
> URL: https://issues.apache.org/jira/browse/AIRFLOW-6515
> Project: Apache Airflow
> Issue Type: Bug
> Components: logging
>Affects Versions: 1.10.7
>Reporter: t oo
>Assignee: Will Hudgins
>Priority: Major
>
> log level should be error on some (but there are false positives):
> grep -iE
> 'log\.(info|warn).*(error|exceptio|fail|unab|couldn|lost|gone|missing|not
> fou|abort|exit|could not)' -R *
> airflow/sensors/base_sensor_operator.py:self.log.info("Success
> criteria met. Exiting.")
> airflow/logging_config.py:log.warning('Unable to load the config,
> contains a configuration error.')
> airflow/operators/check_operator.py:self.log.warning("The
> following %s tests out of %s failed:", j, n)
> airflow/operators/sql_to_gcs.py:self.log.warning('Using default
> schema due to missing name or type. Please '
> airflow/operators/bash_operator.py:self.log.info('Command exited
> with return code %s', self.sub_process.returncode)
> airflow/serialization/serialized_objects.py:LOG.warning('Failed
> to stringify.', exc_info=True)
> airflow/providers/amazon/aws/operators/batch.py:
> self.log.info("AWS Batch Job has failed")
> airflow/providers/amazon/aws/hooks/s3.py:
> self.log.info(e.response["Error"]["Message"])
> airflow/providers/amazon/aws/hooks/s3.py:
> self.log.info(e.response["Error"]["Message"])
> airflow/utils/dag_processing.py:self.log.info("Exiting gracefully
> upon receiving signal %s", signum)
> airflow/utils/dag_processing.py:self.log.info("Exiting dag
> parsing loop as all files "
> airflow/utils/dag_processing.py:self.log.info("Failing jobs
> without heartbeat after %s", limit_dttm)
> airflow/utils/dag_processing.py:self.log.info("Waiting up to %s
> seconds for processes to exit...", timeout)
> airflow/utils/helpers.py:log.info("Process %s (%s) terminated with
> exit code %s", p, p.pid, p.returncode)
> airflow/models/dagrun.py:self.log.info('Marking run %s failed',
> self)
> airflow/models/dagrun.py:self.log.info('Deadlock; marking run %s
> failed', self)
> airflow/models/dagrun.py:self.log.warning("Failed to get
> task '{}' for dag '{}'. "
> airflow/gcp/sensors/gcs.py:self.log.warning("FAILURE: Inactivity
> Period passed, not enough objects found in %s", path)
> airflow/gcp/operators/spanner.py:self.log.info("The Cloud Spanner
> database was missing: "
> airflow/gcp/hooks/kubernetes_engine.py:self.log.info('Assuming
> Success: %s', error.message)
> airflow/gcp/hooks/kubernetes_engine.py:self.log.info('Assuming
> Success: %s', error.message)
> airflow/gcp/hooks/cloud_memorystore.py:self.log.info("Failovering
> Instance: %s", name)
> airflow/gcp/hooks/cloud_memorystore.py:self.log.info("Instance
> failovered: %s", name)
> airflow/gcp/hooks/bigquery.py:self.log.info(error_msg)
> airflow/gcp/hooks/bigtable.py:self.log.info("The instance '%s'
> does not exist in project '%s'. Exiting", instance_id,
> airflow/contrib/sensors/bash_sensor.py:self.log.info("Command
> exited with return code %s", sp.returncode)
> airflow/contrib/sensors/ftp_sensor.py:self.log.info('Ftp
> error encountered: %s', str(e))
> airflow/contrib/operators/azure_container_instances_operator.py:
> self.log.info("Container had exit code: %s", exit_code)
> airflow/contrib/operators/azure_container_instances_operator.py:
> self.log.info("Container exited with detail_status %s", detail_status)
> airflow/contrib/operators/azure_container_instances_operator.py:
> self.log.info("Azure provision failure")
> airflow/contrib/operators/winrm_operator.py:self.log.info("Hook
> not found, creating...")
> airflow/contrib/operators/docker_swarm_operator.py:
> self.log.info('Service status before exitin
[jira] [Resolved] (AIRFLOW-6515) log level of INFO/WARN when ERROR happened
[
https://issues.apache.org/jira/browse/AIRFLOW-6515?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Kaxil Naik resolved AIRFLOW-6515.
-
Fix Version/s: 1.10.11
Resolution: Fixed
> log level of INFO/WARN when ERROR happened
> --
>
> Key: AIRFLOW-6515
> URL: https://issues.apache.org/jira/browse/AIRFLOW-6515
> Project: Apache Airflow
> Issue Type: Bug
> Components: logging
>Affects Versions: 1.10.7
>Reporter: t oo
>Assignee: Will Hudgins
>Priority: Major
> Fix For: 1.10.11
>
>
> log level should be error on some (but there are false positives):
> grep -iE
> 'log\.(info|warn).*(error|exceptio|fail|unab|couldn|lost|gone|missing|not
> fou|abort|exit|could not)' -R *
> airflow/sensors/base_sensor_operator.py:self.log.info("Success
> criteria met. Exiting.")
> airflow/logging_config.py:log.warning('Unable to load the config,
> contains a configuration error.')
> airflow/operators/check_operator.py:self.log.warning("The
> following %s tests out of %s failed:", j, n)
> airflow/operators/sql_to_gcs.py:self.log.warning('Using default
> schema due to missing name or type. Please '
> airflow/operators/bash_operator.py:self.log.info('Command exited
> with return code %s', self.sub_process.returncode)
> airflow/serialization/serialized_objects.py:LOG.warning('Failed
> to stringify.', exc_info=True)
> airflow/providers/amazon/aws/operators/batch.py:
> self.log.info("AWS Batch Job has failed")
> airflow/providers/amazon/aws/hooks/s3.py:
> self.log.info(e.response["Error"]["Message"])
> airflow/providers/amazon/aws/hooks/s3.py:
> self.log.info(e.response["Error"]["Message"])
> airflow/utils/dag_processing.py:self.log.info("Exiting gracefully
> upon receiving signal %s", signum)
> airflow/utils/dag_processing.py:self.log.info("Exiting dag
> parsing loop as all files "
> airflow/utils/dag_processing.py:self.log.info("Failing jobs
> without heartbeat after %s", limit_dttm)
> airflow/utils/dag_processing.py:self.log.info("Waiting up to %s
> seconds for processes to exit...", timeout)
> airflow/utils/helpers.py:log.info("Process %s (%s) terminated with
> exit code %s", p, p.pid, p.returncode)
> airflow/models/dagrun.py:self.log.info('Marking run %s failed',
> self)
> airflow/models/dagrun.py:self.log.info('Deadlock; marking run %s
> failed', self)
> airflow/models/dagrun.py:self.log.warning("Failed to get
> task '{}' for dag '{}'. "
> airflow/gcp/sensors/gcs.py:self.log.warning("FAILURE: Inactivity
> Period passed, not enough objects found in %s", path)
> airflow/gcp/operators/spanner.py:self.log.info("The Cloud Spanner
> database was missing: "
> airflow/gcp/hooks/kubernetes_engine.py:self.log.info('Assuming
> Success: %s', error.message)
> airflow/gcp/hooks/kubernetes_engine.py:self.log.info('Assuming
> Success: %s', error.message)
> airflow/gcp/hooks/cloud_memorystore.py:self.log.info("Failovering
> Instance: %s", name)
> airflow/gcp/hooks/cloud_memorystore.py:self.log.info("Instance
> failovered: %s", name)
> airflow/gcp/hooks/bigquery.py:self.log.info(error_msg)
> airflow/gcp/hooks/bigtable.py:self.log.info("The instance '%s'
> does not exist in project '%s'. Exiting", instance_id,
> airflow/contrib/sensors/bash_sensor.py:self.log.info("Command
> exited with return code %s", sp.returncode)
> airflow/contrib/sensors/ftp_sensor.py:self.log.info('Ftp
> error encountered: %s', str(e))
> airflow/contrib/operators/azure_container_instances_operator.py:
> self.log.info("Container had exit code: %s", exit_code)
> airflow/contrib/operators/azure_container_instances_operator.py:
> self.log.info("Container exited with detail_status %s", detail_status)
> airflow/contrib/operators/azure_container_instances_operator.py:
> self.log.info("Azure provision failure")
> airflow/contrib/operators/winrm_operator.py:self.log.info("Hook
> not found, creating...")
> airflow/contrib/operators/docker_swarm_operator.py:
> self.log.info('Service status before exiting: %s', status)
> airflow/contrib/auth/backends/ldap_auth.py:log.warning("Unable to
> find group for %s %s", search_base, search_filter)
> airflow/contrib/auth/backends/ldap_auth.py:log.warning("""Missing
> attribute "%s" when looked-up in Ldap database.
> airflow/contrib/auth/backends/ldap_auth.py:log.warning("Parsing error
> when retrieving the user's group(s)."
> airflow/contrib/utils/sendgrid.py:log.warning('Fai
[jira] [Commented] (AIRFLOW-7110) Refactor the way logs are written to/read from Elasticsearch
[
https://issues.apache.org/jira/browse/AIRFLOW-7110?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17080023#comment-17080023
]
Patrick Gaudreau commented on AIRFLOW-7110:
---
We ran into a similar issue as well when using Airflow deployed on an Azure AKS
with the following configuration:
{noformat}
AIRFLOW__CORE__REMOTE_LOGGING: true
AIRFLOW__ELASTICSEARCH__WRITE_STDOUT: true
{noformat}
Our logs are properly shipped to elasticsearch (because we write to stdout and
are using bkpr that sends everything from stdout to elasticsearch) but the
Airflow UI can't get the logs back because there is no 'log_id' field created.
One possible solution to our problem would be if the 'log_id' field name was
configurable. We could then add an extra kubernetes label called
'kubernetes.labels.log_id' to every log using for example a Pod Mutation Hook.
> Refactor the way logs are written to/read from Elasticsearch
>
>
> Key: AIRFLOW-7110
> URL: https://issues.apache.org/jira/browse/AIRFLOW-7110
> Project: Apache Airflow
> Issue Type: Improvement
> Components: configuration, logging, operators, ui
>Affects Versions: 1.10.9
>Reporter: Noël BARDELOT
>Priority: Major
>
> See:
> * in the configuration,
> - under section 'core' option named 'remote_logging'
> - under section 'elasticsearch' options named 'host', 'log_id_template',
> 'end_of_log_mark', 'frontend', 'write_stdout' and 'json_format'
> * in the documentation,
> - https://airflow.apache.org/docs/stable/howto/write-logs.html
> * in the Airflow code,
> - BaseOperator in airflow/models/baseoperator.py
> - ElasticsearchTaskHandler in airflow/utils/log/es_task_handler.py
> - the detail of a task's view in airflow/www/views.py
> Current behaviour:
> If one sets the options as stated in the documentation in order to write logs
> from the task to the worker stdout logs in JSON format, and upload those logs
> in Elasticsearch (using ELK or an equivalent stack), the field 'log_id'
> needed by the handler and the view to work is not written by default to the
> logs.
> One needs to adapt the log collection from the worker's container stdout logs
> in order to provide this 'log_id' field, which agregates other usual fields
> (see 'log_id_template' which is used by the handler to request logs from
> Elasticsearch to be shown in the view).
> An example for fluentd:
> {code}
> # Also add log_id to airflow.task logs
>
> @type record_transformer
> enable_ruby
>
> log_id
> ${record["dag_id"]}_${record["task_id"]}_${record["execution_date"]}_${record["try_number"]}
> offset ${time = Time.now; time.to_i * (10 ** 9) + time.nsec}
>
>
> {code}
> What we expect:
> It should be the job of the Operator to log the needed 'log_id' field
> (Airflow reads it and needs it, so Airflow should write it itself...). That
> should be done in the BaseOperator, so that it's available to all inheriting
> subclasses. This field could be added only if the 'remote_logging' is set to
> True for example in order to avoid writing more logs than necessary.
> Plus, there is no benefit to make the 'log_id_template' configurable. As is,
> there is no way to make anything else than the current value work, and nobody
> should ever modify it (if someone modified that value, the view wouldn't work
> anyway).
> Plus, the 'frontend' configuration option should not presume that 'https://'
> is the protoocol to use in order to make a link to the external log viewer
> (Kibana or other...). The 'frontend' option should contain the whole URL
> templated by 'log_id'.
> Plus, all this behaviour should be far more documented in the 'write-logs'
> documentation.
> Plus, a DAG example should be added to Airflow's already existing example, in
> order to show how a subclass of BaseOperator should use the logging mechanism
> to write a log.
> Plus, Elasticsearch's 'host' option under Airflow's configuration is wrongly
> named and documented. It is not a host, but a server with host and port.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[GitHub] [airflow] kaxil merged pull request #8170: [AIRFLOW-6515] Change Log Levels from Info/Warn to Error
kaxil merged pull request #8170: [AIRFLOW-6515] Change Log Levels from Info/Warn to Error URL: https://github.com/apache/airflow/pull/8170 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [airflow] kaxil commented on issue #8170: [AIRFLOW-6515] Change Log Levels from Info/Warn to Error
kaxil commented on issue #8170: [AIRFLOW-6515] Change Log Levels from Info/Warn to Error URL: https://github.com/apache/airflow/pull/8170#issuecomment-611776575 > > Another alternative could be clear guidelines on logging in some kind of easy-to-access contributor documentation. We could even make it a checkbox on the PR template. > > That's what I'm thinking. However, I am not sure if anyone will search for or follow it... Exactly. If we want we can write something at https://github.com/apache/airflow/blob/master/CONTRIBUTING.rst#coding-style-and-best-practices This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [airflow] boring-cyborg[bot] commented on issue #8170: [AIRFLOW-6515] Change Log Levels from Info/Warn to Error
boring-cyborg[bot] commented on issue #8170: [AIRFLOW-6515] Change Log Levels from Info/Warn to Error URL: https://github.com/apache/airflow/pull/8170#issuecomment-611776691 Awesome work, congrats on your first merged pull request! This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [airflow] wkhudgins92 commented on issue #8170: [AIRFLOW-6515] Change Log Levels from Info/Warn to Error
wkhudgins92 commented on issue #8170: [AIRFLOW-6515] Change Log Levels from Info/Warn to Error URL: https://github.com/apache/airflow/pull/8170#issuecomment-611772348 @turbaszek I think the most effective thing we could try would be a checkbox on the PR template, similar to "Commits follow "How to write a good git commit message", it could link to some markdown file about logging. We would want to have community review and consensus concerning the contents of such a guide. If this is something worth pursuing forward, it might make sense to create a followup ticket in JIRA for it and move this PR and ticket along. I would be willing to write a draft of such a document. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [airflow] turbaszek commented on issue #8170: [AIRFLOW-6515] Change Log Levels from Info/Warn to Error
turbaszek commented on issue #8170: [AIRFLOW-6515] Change Log Levels from Info/Warn to Error URL: https://github.com/apache/airflow/pull/8170#issuecomment-611765011 > Another alternative could be clear guidelines on logging in some kind of easy-to-access contributor documentation. We could even make it a checkbox on the PR template. That's what I'm thinking. However, I am not sure if anyone will search for or follow it... This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [airflow] wkhudgins92 commented on issue #8170: [AIRFLOW-6515] Change Log Levels from Info/Warn to Error
wkhudgins92 commented on issue #8170: [AIRFLOW-6515] Change Log Levels from
Info/Warn to Error
URL: https://github.com/apache/airflow/pull/8170#issuecomment-611762956
@turbaszek I would love to add a regression test for this but I agree with
@kaxil , it likely is not possible. You could flag certain words in an info
message, like I did with the grep statements, but at the end of the day there's
always a change that those keywords will appear in a valid info/debug log
statement. For example,
```
self.log.info("Secret %s not found in Path: %s", secret_id, secret_path)
```
not found in path sounds like an error, or at least a warn, but in reality,
in this specific context it should be a debug.
The best we could do would be a flag/warning but I don't think we can build
a sufficiently smart CI check to incorporate it into the official CI processes.
Another alternative could be clear guidelines on logging in some kind of
easy-to-access contributor documentation. We could even make it a checkbox on
the PR template.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [airflow] aostapenko commented on issue #8005: Use annotated tags for releases?
aostapenko commented on issue #8005: Use annotated tags for releases? URL: https://github.com/apache/airflow/issues/8005#issuecomment-611762323 Who created tag and when is rather a helpful info. We don't discuss why commits have an author, date and commit message. Why should this be different for tags? Creating release tag using particular commit is an accountable decision. And additional info may help to better understand occurrences of tag recreation. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [airflow] turbaszek commented on issue #8170: [AIRFLOW-6515] Change Log Levels from Info/Warn to Error
turbaszek commented on issue #8170: [AIRFLOW-6515] Change Log Levels from Info/Warn to Error URL: https://github.com/apache/airflow/pull/8170#issuecomment-611761887 > Not sure you can avoid regression here. It depends on a case-by-case basis if a particular LOG is DEBUG, WARN, ERROR or INFO True. But some basic check can help. Do we have any guide on selecting log level? This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [airflow] kaxil commented on issue #8170: [AIRFLOW-6515] Change Log Levels from Info/Warn to Error
kaxil commented on issue #8170: [AIRFLOW-6515] Change Log Levels from Info/Warn to Error URL: https://github.com/apache/airflow/pull/8170#issuecomment-611759649 > > @wkhudgins92 thanks for the PR! I'm wondering, can we add pre-commit check to avoid regression (i.e. using pygrep)? Also, what in your opinion is the difference between error and warning levels? > > @wkhudgins92 Not sure you can avoid regression here. It depends on a case-by-case basis if a particular LOG is DEBUG, WARN, ERROR or INFO This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [airflow] kaxil commented on a change in pull request #8222: github action tests
kaxil commented on a change in pull request #8222: github action tests
URL: https://github.com/apache/airflow/pull/8222#discussion_r406485461
##
File path: scripts/ci/ci_run_airflow_testing.sh
##
@@ -79,36 +79,3 @@ done
RUN_INTEGRATION_TESTS=${RUN_INTEGRATION_TESTS:=""}
-if [[ ${RUNTIME:=} == "kubernetes" ]]; then
-export KUBERNETES_MODE=${KUBERNETES_MODE:="git_mode"}
-export KUBERNETES_VERSION=${KUBERNETES_VERSION:="v1.15.3"}
-
-set +u
-# shellcheck disable=SC2016
-docker-compose --log-level INFO \
- -f "${MY_DIR}/docker-compose/base.yml" \
- -f "${MY_DIR}/docker-compose/backend-${BACKEND}.yml" \
- -f "${MY_DIR}/docker-compose/runtime-kubernetes.yml" \
- "${INTEGRATIONS[@]}" \
- "${DOCKER_COMPOSE_LOCAL[@]}" \
- run airflow \
- '/opt/airflow/scripts/ci/in_container/entrypoint_ci.sh "${@}"' \
- /opt/airflow/scripts/ci/in_container/entrypoint_ci.sh "${@}"
- # Note the command is there twice (!) because it is passed via bash -c
- # and bash -c starts passing parameters from $0. TODO: fixme
-set -u
-else
-set +u
-# shellcheck disable=SC2016
-docker-compose --log-level INFO \
- -f "${MY_DIR}/docker-compose/base.yml" \
- -f "${MY_DIR}/docker-compose/backend-${BACKEND}.yml" \
- "${INTEGRATIONS[@]}" \
- "${DOCKER_COMPOSE_LOCAL[@]}" \
- run airflow \
- '/opt/airflow/scripts/ci/in_container/entrypoint_ci.sh "${@}"' \
- /opt/airflow/scripts/ci/in_container/entrypoint_ci.sh "${@}"
- # Note the command is there twice (!) because it is passed via bash -c
- # and bash -c starts passing parameters from $0. TODO: fixme
-set -u
Review comment:
I thought only the "if" part ran Kubernetes tests
cc @potiuk Can you verify if we need the else part or is it safe to remove,
please :)
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [airflow-site] kaxil merged pull request #261: Docs for 1.10.10
kaxil merged pull request #261: Docs for 1.10.10 URL: https://github.com/apache/airflow-site/pull/261 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [airflow] turbaszek commented on a change in pull request #8145: Drop redundant project id null checks.
turbaszek commented on a change in pull request #8145: Drop redundant project id null checks. URL: https://github.com/apache/airflow/pull/8145#discussion_r406473607 ## File path: airflow/providers/google/cloud/hooks/automl.py ## @@ -499,9 +481,6 @@ def update_dataset( :param update_mask: The update mask applies to the resource. If a dict is provided, it must be of the same form as the protobuf message FieldMask. :type update_mask: Union[dict, FieldMask] -:param project_id: ID of the Google Cloud project where dataset is located if None then Review comment: @potiuk no need I think, automl is master only This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [airflow] turbaszek commented on a change in pull request #8145: Drop redundant project id null checks.
turbaszek commented on a change in pull request #8145: Drop redundant project id null checks. URL: https://github.com/apache/airflow/pull/8145#discussion_r406473921 ## File path: airflow/providers/google/cloud/hooks/stackdriver.py ## @@ -500,8 +500,8 @@ def enable_notification_channels( @GoogleBaseHook.fallback_to_default_project_id def disable_notification_channels( self, +project_id: str, Review comment: @potiuk was this hook released? This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [airflow] turbaszek commented on issue #8170: [AIRFLOW-6515] Change Log Levels from Info/Warn to Error
turbaszek commented on issue #8170: [AIRFLOW-6515] Change Log Levels from Info/Warn to Error URL: https://github.com/apache/airflow/pull/8170#issuecomment-611749011 > @wkhudgins92 thanks for the PR! I'm wondering, can we add pre-commit check to avoid regression (i.e. using pygrep)? Also, what in your opinion is the difference between error and warning levels? @wkhudgins92 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [airflow] dimberman commented on a change in pull request #8222: github action tests
dimberman commented on a change in pull request #8222: github action tests
URL: https://github.com/apache/airflow/pull/8222#discussion_r406471990
##
File path: scripts/ci/ci_run_airflow_testing.sh
##
@@ -79,36 +79,3 @@ done
RUN_INTEGRATION_TESTS=${RUN_INTEGRATION_TESTS:=""}
-if [[ ${RUNTIME:=} == "kubernetes" ]]; then
-export KUBERNETES_MODE=${KUBERNETES_MODE:="git_mode"}
-export KUBERNETES_VERSION=${KUBERNETES_VERSION:="v1.15.3"}
-
-set +u
-# shellcheck disable=SC2016
-docker-compose --log-level INFO \
- -f "${MY_DIR}/docker-compose/base.yml" \
- -f "${MY_DIR}/docker-compose/backend-${BACKEND}.yml" \
- -f "${MY_DIR}/docker-compose/runtime-kubernetes.yml" \
- "${INTEGRATIONS[@]}" \
- "${DOCKER_COMPOSE_LOCAL[@]}" \
- run airflow \
- '/opt/airflow/scripts/ci/in_container/entrypoint_ci.sh "${@}"' \
- /opt/airflow/scripts/ci/in_container/entrypoint_ci.sh "${@}"
- # Note the command is there twice (!) because it is passed via bash -c
- # and bash -c starts passing parameters from $0. TODO: fixme
-set -u
-else
-set +u
-# shellcheck disable=SC2016
-docker-compose --log-level INFO \
- -f "${MY_DIR}/docker-compose/base.yml" \
- -f "${MY_DIR}/docker-compose/backend-${BACKEND}.yml" \
- "${INTEGRATIONS[@]}" \
- "${DOCKER_COMPOSE_LOCAL[@]}" \
- run airflow \
- '/opt/airflow/scripts/ci/in_container/entrypoint_ci.sh "${@}"' \
- /opt/airflow/scripts/ci/in_container/entrypoint_ci.sh "${@}"
- # Note the command is there twice (!) because it is passed via bash -c
- # and bash -c starts passing parameters from $0. TODO: fixme
-set -u
Review comment:
How come? We're not running those travis tests anymore
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [airflow-site] kaxil opened a new pull request #261: Docs for 1.10.10
kaxil opened a new pull request #261: Docs for 1.10.10 URL: https://github.com/apache/airflow-site/pull/261 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [airflow] turbaszek commented on a change in pull request #8222: github action tests
turbaszek commented on a change in pull request #8222: github action tests
URL: https://github.com/apache/airflow/pull/8222#discussion_r406470186
##
File path: .github/workflows/ci-k8s.yml
##
@@ -0,0 +1,68 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+
+name: "k8s tests"
+on: [pull_request, push]
+
+jobs:
+ create-airflow-dockerfile:
+runs-on: ubuntu-latest
+strategy:
+ matrix:
+k8s-version: [v1.15.7, v1.16.4, v1.17.2, v1.18.0]
+steps:
+ - uses: actions/checkout@v2
+name: checkout
+ - name: create k8s Kind Cluster
+uses: helm/kind-action@v1.0.0-rc.1
+with:
+ cluster_name: kind
+ node_image: kindest/node:${{ matrix.k8s-version }}
+# config: ./kind-cluster-conf.yaml
+ - name: test cluster running
+run: |
+ kubectl cluster-info
+ kubectl get pods -n kube-system
+ - name: Set up Python 3.7
+uses: actions/setup-python@v1
+with:
+ python-version: '3.7'
+ - name: pip install airflow
+run: pip install -e .
+ - name: download helm
+run: |
+ curl -fsSL -o get_helm.sh
https://raw.githubusercontent.com/helm/helm/master/scripts/get-helm-3
+ chmod 700 get_helm.sh
+ ./get_helm.sh
+ - name: create namespace
+run: kubectl create namespace airflow
+ - name: pull astronomer helm chart
+run: |
+ helm repo add astronomer https://helm.astronomer.io
+ helm install airflow --namespace airflow astronomer/airflow
+ kubectl get pods --namespace airflow
+ kubectl port-forward svc/airflow-webserver 8080:8080 &
+ - name: pip install airflow
+run: pip install -e .[devel]
+ - name: run k8sexecutor tests
+run: pytest tests/runtime/kubernetes/test_kubernetes_executor.py
Review comment:
```suggestion
run: pytest --runtime=kubernetes
tests/runtime/kubernetes/test_kubernetes_executor.py
```
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [airflow] ashb merged pull request #8218: CSS linting integrated into pre-commit
ashb merged pull request #8218: CSS linting integrated into pre-commit URL: https://github.com/apache/airflow/pull/8218 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[airflow-site] branch docs-for-1.10.10 created (now 7e4ce41)
This is an automated email from the ASF dual-hosted git repository. kaxilnaik pushed a change to branch docs-for-1.10.10 in repository https://gitbox.apache.org/repos/asf/airflow-site.git. at 7e4ce41 Docs for 1.10.10 This branch includes the following new commits: new 7e4ce41 Docs for 1.10.10 The 1 revisions listed above as "new" are entirely new to this repository and will be described in separate emails. The revisions listed as "add" were already present in the repository and have only been added to this reference.
[GitHub] [airflow] codecov-io edited a comment on issue #8232: Simplify DAG.set_dag_runs_state method
codecov-io edited a comment on issue #8232: Simplify DAG.set_dag_runs_state method URL: https://github.com/apache/airflow/pull/8232#issuecomment-611742609 # [Codecov](https://codecov.io/gh/apache/airflow/pull/8232?src=pr&el=h1) Report > Merging [#8232](https://codecov.io/gh/apache/airflow/pull/8232?src=pr&el=desc) into [master](https://codecov.io/gh/apache/airflow/commit/ad42c4076479fe7420dc329ee8bfd51d9fb0a2ae&el=desc) will **decrease** coverage by `0.77%`. > The diff coverage is `100.00%`. [](https://codecov.io/gh/apache/airflow/pull/8232?src=pr&el=tree) ```diff @@Coverage Diff @@ ## master#8232 +/- ## == - Coverage 88.35% 87.58% -0.78% == Files 936 936 Lines 4530545312 +7 == - Hits4003039687 -343 - Misses 5275 5625 +350 ``` | [Impacted Files](https://codecov.io/gh/apache/airflow/pull/8232?src=pr&el=tree) | Coverage Δ | | |---|---|---| | [airflow/models/dag.py](https://codecov.io/gh/apache/airflow/pull/8232/diff?src=pr&el=tree#diff-YWlyZmxvdy9tb2RlbHMvZGFnLnB5) | `91.61% <100.00%> (-0.05%)` | :arrow_down: | | [...flow/providers/apache/cassandra/hooks/cassandra.py](https://codecov.io/gh/apache/airflow/pull/8232/diff?src=pr&el=tree#diff-YWlyZmxvdy9wcm92aWRlcnMvYXBhY2hlL2Nhc3NhbmRyYS9ob29rcy9jYXNzYW5kcmEucHk=) | `21.25% <0.00%> (-72.50%)` | :arrow_down: | | [...w/providers/apache/hive/operators/mysql\_to\_hive.py](https://codecov.io/gh/apache/airflow/pull/8232/diff?src=pr&el=tree#diff-YWlyZmxvdy9wcm92aWRlcnMvYXBhY2hlL2hpdmUvb3BlcmF0b3JzL215c3FsX3RvX2hpdmUucHk=) | `35.84% <0.00%> (-64.16%)` | :arrow_down: | | [airflow/kubernetes/volume\_mount.py](https://codecov.io/gh/apache/airflow/pull/8232/diff?src=pr&el=tree#diff-YWlyZmxvdy9rdWJlcm5ldGVzL3ZvbHVtZV9tb3VudC5weQ==) | `44.44% <0.00%> (-55.56%)` | :arrow_down: | | [airflow/providers/postgres/operators/postgres.py](https://codecov.io/gh/apache/airflow/pull/8232/diff?src=pr&el=tree#diff-YWlyZmxvdy9wcm92aWRlcnMvcG9zdGdyZXMvb3BlcmF0b3JzL3Bvc3RncmVzLnB5) | `47.82% <0.00%> (-52.18%)` | :arrow_down: | | [airflow/providers/redis/operators/redis\_publish.py](https://codecov.io/gh/apache/airflow/pull/8232/diff?src=pr&el=tree#diff-YWlyZmxvdy9wcm92aWRlcnMvcmVkaXMvb3BlcmF0b3JzL3JlZGlzX3B1Ymxpc2gucHk=) | `50.00% <0.00%> (-50.00%)` | :arrow_down: | | [airflow/kubernetes/volume.py](https://codecov.io/gh/apache/airflow/pull/8232/diff?src=pr&el=tree#diff-YWlyZmxvdy9rdWJlcm5ldGVzL3ZvbHVtZS5weQ==) | `52.94% <0.00%> (-47.06%)` | :arrow_down: | | [airflow/providers/mongo/sensors/mongo.py](https://codecov.io/gh/apache/airflow/pull/8232/diff?src=pr&el=tree#diff-YWlyZmxvdy9wcm92aWRlcnMvbW9uZ28vc2Vuc29ycy9tb25nby5weQ==) | `53.33% <0.00%> (-46.67%)` | :arrow_down: | | [airflow/kubernetes/pod\_launcher.py](https://codecov.io/gh/apache/airflow/pull/8232/diff?src=pr&el=tree#diff-YWlyZmxvdy9rdWJlcm5ldGVzL3BvZF9sYXVuY2hlci5weQ==) | `47.18% <0.00%> (-45.08%)` | :arrow_down: | | [airflow/providers/mysql/operators/mysql.py](https://codecov.io/gh/apache/airflow/pull/8232/diff?src=pr&el=tree#diff-YWlyZmxvdy9wcm92aWRlcnMvbXlzcWwvb3BlcmF0b3JzL215c3FsLnB5) | `55.00% <0.00%> (-45.00%)` | :arrow_down: | | ... and [12 more](https://codecov.io/gh/apache/airflow/pull/8232/diff?src=pr&el=tree-more) | | -- [Continue to review full report at Codecov](https://codecov.io/gh/apache/airflow/pull/8232?src=pr&el=continue). > **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta) > `Δ = absolute (impact)`, `ø = not affected`, `? = missing data` > Powered by [Codecov](https://codecov.io/gh/apache/airflow/pull/8232?src=pr&el=footer). Last update [ad42c40...31cd70c](https://codecov.io/gh/apache/airflow/pull/8232?src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments). This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [airflow] ashb commented on issue #8005: Use annotated tags for releases?
ashb commented on issue #8005: Use annotated tags for releases? URL: https://github.com/apache/airflow/issues/8005#issuecomment-611742756 @aostapenko Show me something other than git's own docs that claims annotated tags are best practice and we can revaluate it. Git's docs are primarilry written by git+linux kernel developers. What does `git push --follow-tags` give that `git push --tags` doesn't? @cwedgwood `git describe --tags` works with non-annotated tags. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [airflow] codecov-io commented on issue #8232: Simplify DAG.set_dag_runs_state method
codecov-io commented on issue #8232: Simplify DAG.set_dag_runs_state method URL: https://github.com/apache/airflow/pull/8232#issuecomment-611742609 # [Codecov](https://codecov.io/gh/apache/airflow/pull/8232?src=pr&el=h1) Report > Merging [#8232](https://codecov.io/gh/apache/airflow/pull/8232?src=pr&el=desc) into [master](https://codecov.io/gh/apache/airflow/commit/ad42c4076479fe7420dc329ee8bfd51d9fb0a2ae&el=desc) will **decrease** coverage by `0.77%`. > The diff coverage is `100.00%`. [](https://codecov.io/gh/apache/airflow/pull/8232?src=pr&el=tree) ```diff @@Coverage Diff @@ ## master#8232 +/- ## == - Coverage 88.35% 87.58% -0.78% == Files 936 936 Lines 4530545312 +7 == - Hits4003039687 -343 - Misses 5275 5625 +350 ``` | [Impacted Files](https://codecov.io/gh/apache/airflow/pull/8232?src=pr&el=tree) | Coverage Δ | | |---|---|---| | [airflow/models/dag.py](https://codecov.io/gh/apache/airflow/pull/8232/diff?src=pr&el=tree#diff-YWlyZmxvdy9tb2RlbHMvZGFnLnB5) | `91.61% <100.00%> (-0.05%)` | :arrow_down: | | [...flow/providers/apache/cassandra/hooks/cassandra.py](https://codecov.io/gh/apache/airflow/pull/8232/diff?src=pr&el=tree#diff-YWlyZmxvdy9wcm92aWRlcnMvYXBhY2hlL2Nhc3NhbmRyYS9ob29rcy9jYXNzYW5kcmEucHk=) | `21.25% <0.00%> (-72.50%)` | :arrow_down: | | [...w/providers/apache/hive/operators/mysql\_to\_hive.py](https://codecov.io/gh/apache/airflow/pull/8232/diff?src=pr&el=tree#diff-YWlyZmxvdy9wcm92aWRlcnMvYXBhY2hlL2hpdmUvb3BlcmF0b3JzL215c3FsX3RvX2hpdmUucHk=) | `35.84% <0.00%> (-64.16%)` | :arrow_down: | | [airflow/kubernetes/volume\_mount.py](https://codecov.io/gh/apache/airflow/pull/8232/diff?src=pr&el=tree#diff-YWlyZmxvdy9rdWJlcm5ldGVzL3ZvbHVtZV9tb3VudC5weQ==) | `44.44% <0.00%> (-55.56%)` | :arrow_down: | | [airflow/providers/postgres/operators/postgres.py](https://codecov.io/gh/apache/airflow/pull/8232/diff?src=pr&el=tree#diff-YWlyZmxvdy9wcm92aWRlcnMvcG9zdGdyZXMvb3BlcmF0b3JzL3Bvc3RncmVzLnB5) | `47.82% <0.00%> (-52.18%)` | :arrow_down: | | [airflow/providers/redis/operators/redis\_publish.py](https://codecov.io/gh/apache/airflow/pull/8232/diff?src=pr&el=tree#diff-YWlyZmxvdy9wcm92aWRlcnMvcmVkaXMvb3BlcmF0b3JzL3JlZGlzX3B1Ymxpc2gucHk=) | `50.00% <0.00%> (-50.00%)` | :arrow_down: | | [airflow/kubernetes/volume.py](https://codecov.io/gh/apache/airflow/pull/8232/diff?src=pr&el=tree#diff-YWlyZmxvdy9rdWJlcm5ldGVzL3ZvbHVtZS5weQ==) | `52.94% <0.00%> (-47.06%)` | :arrow_down: | | [airflow/providers/mongo/sensors/mongo.py](https://codecov.io/gh/apache/airflow/pull/8232/diff?src=pr&el=tree#diff-YWlyZmxvdy9wcm92aWRlcnMvbW9uZ28vc2Vuc29ycy9tb25nby5weQ==) | `53.33% <0.00%> (-46.67%)` | :arrow_down: | | [airflow/kubernetes/pod\_launcher.py](https://codecov.io/gh/apache/airflow/pull/8232/diff?src=pr&el=tree#diff-YWlyZmxvdy9rdWJlcm5ldGVzL3BvZF9sYXVuY2hlci5weQ==) | `47.18% <0.00%> (-45.08%)` | :arrow_down: | | [airflow/providers/mysql/operators/mysql.py](https://codecov.io/gh/apache/airflow/pull/8232/diff?src=pr&el=tree#diff-YWlyZmxvdy9wcm92aWRlcnMvbXlzcWwvb3BlcmF0b3JzL215c3FsLnB5) | `55.00% <0.00%> (-45.00%)` | :arrow_down: | | ... and [12 more](https://codecov.io/gh/apache/airflow/pull/8232/diff?src=pr&el=tree-more) | | -- [Continue to review full report at Codecov](https://codecov.io/gh/apache/airflow/pull/8232?src=pr&el=continue). > **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta) > `Δ = absolute (impact)`, `ø = not affected`, `? = missing data` > Powered by [Codecov](https://codecov.io/gh/apache/airflow/pull/8232?src=pr&el=footer). Last update [ad42c40...31cd70c](https://codecov.io/gh/apache/airflow/pull/8232?src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments). This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [airflow] codecov-io edited a comment on issue #8218: CSS linting integrated into pre-commit
codecov-io edited a comment on issue #8218: CSS linting integrated into pre-commit URL: https://github.com/apache/airflow/pull/8218#issuecomment-611740217 # [Codecov](https://codecov.io/gh/apache/airflow/pull/8218?src=pr&el=h1) Report > Merging [#8218](https://codecov.io/gh/apache/airflow/pull/8218?src=pr&el=desc) into [master](https://codecov.io/gh/apache/airflow/commit/1317cdc63546a35ad9cf229f9026ac0d08f01dfb&el=desc) will **decrease** coverage by `0.47%`. > The diff coverage is `n/a`. [](https://codecov.io/gh/apache/airflow/pull/8218?src=pr&el=tree) ```diff @@Coverage Diff @@ ## master#8218 +/- ## == - Coverage 88.32% 87.85% -0.48% == Files 936 936 Lines 4531645319 +3 == - Hits4002739813 -214 - Misses 5289 5506 +217 ``` | [Impacted Files](https://codecov.io/gh/apache/airflow/pull/8218?src=pr&el=tree) | Coverage Δ | | |---|---|---| | [...flow/providers/apache/cassandra/hooks/cassandra.py](https://codecov.io/gh/apache/airflow/pull/8218/diff?src=pr&el=tree#diff-YWlyZmxvdy9wcm92aWRlcnMvYXBhY2hlL2Nhc3NhbmRyYS9ob29rcy9jYXNzYW5kcmEucHk=) | `21.25% <0.00%> (-72.50%)` | :arrow_down: | | [...w/providers/apache/hive/operators/mysql\_to\_hive.py](https://codecov.io/gh/apache/airflow/pull/8218/diff?src=pr&el=tree#diff-YWlyZmxvdy9wcm92aWRlcnMvYXBhY2hlL2hpdmUvb3BlcmF0b3JzL215c3FsX3RvX2hpdmUucHk=) | `35.84% <0.00%> (-64.16%)` | :arrow_down: | | [airflow/providers/postgres/operators/postgres.py](https://codecov.io/gh/apache/airflow/pull/8218/diff?src=pr&el=tree#diff-YWlyZmxvdy9wcm92aWRlcnMvcG9zdGdyZXMvb3BlcmF0b3JzL3Bvc3RncmVzLnB5) | `47.82% <0.00%> (-52.18%)` | :arrow_down: | | [airflow/providers/redis/operators/redis\_publish.py](https://codecov.io/gh/apache/airflow/pull/8218/diff?src=pr&el=tree#diff-YWlyZmxvdy9wcm92aWRlcnMvcmVkaXMvb3BlcmF0b3JzL3JlZGlzX3B1Ymxpc2gucHk=) | `50.00% <0.00%> (-50.00%)` | :arrow_down: | | [airflow/providers/mongo/sensors/mongo.py](https://codecov.io/gh/apache/airflow/pull/8218/diff?src=pr&el=tree#diff-YWlyZmxvdy9wcm92aWRlcnMvbW9uZ28vc2Vuc29ycy9tb25nby5weQ==) | `53.33% <0.00%> (-46.67%)` | :arrow_down: | | [airflow/providers/mysql/operators/mysql.py](https://codecov.io/gh/apache/airflow/pull/8218/diff?src=pr&el=tree#diff-YWlyZmxvdy9wcm92aWRlcnMvbXlzcWwvb3BlcmF0b3JzL215c3FsLnB5) | `55.00% <0.00%> (-45.00%)` | :arrow_down: | | [airflow/providers/redis/sensors/redis\_key.py](https://codecov.io/gh/apache/airflow/pull/8218/diff?src=pr&el=tree#diff-YWlyZmxvdy9wcm92aWRlcnMvcmVkaXMvc2Vuc29ycy9yZWRpc19rZXkucHk=) | `61.53% <0.00%> (-38.47%)` | :arrow_down: | | [airflow/executors/celery\_executor.py](https://codecov.io/gh/apache/airflow/pull/8218/diff?src=pr&el=tree#diff-YWlyZmxvdy9leGVjdXRvcnMvY2VsZXJ5X2V4ZWN1dG9yLnB5) | `50.67% <0.00%> (-37.84%)` | :arrow_down: | | [...roviders/google/cloud/operators/postgres\_to\_gcs.py](https://codecov.io/gh/apache/airflow/pull/8218/diff?src=pr&el=tree#diff-YWlyZmxvdy9wcm92aWRlcnMvZ29vZ2xlL2Nsb3VkL29wZXJhdG9ycy9wb3N0Z3Jlc190b19nY3MucHk=) | `54.28% <0.00%> (-31.43%)` | :arrow_down: | | [airflow/providers/postgres/hooks/postgres.py](https://codecov.io/gh/apache/airflow/pull/8218/diff?src=pr&el=tree#diff-YWlyZmxvdy9wcm92aWRlcnMvcG9zdGdyZXMvaG9va3MvcG9zdGdyZXMucHk=) | `78.87% <0.00%> (-14.09%)` | :arrow_down: | | ... and [5 more](https://codecov.io/gh/apache/airflow/pull/8218/diff?src=pr&el=tree-more) | | -- [Continue to review full report at Codecov](https://codecov.io/gh/apache/airflow/pull/8218?src=pr&el=continue). > **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta) > `Δ = absolute (impact)`, `ø = not affected`, `? = missing data` > Powered by [Codecov](https://codecov.io/gh/apache/airflow/pull/8218?src=pr&el=footer). Last update [1317cdc...bce6797](https://codecov.io/gh/apache/airflow/pull/8218?src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments). This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [airflow] KevinYang21 commented on issue #7133: [AIRFLOW-6535] add exception to fail without retry
KevinYang21 commented on issue #7133: [AIRFLOW-6535] add exception to fail without retry URL: https://github.com/apache/airflow/pull/7133#issuecomment-611741165 @jstern sry I was distracted, yes I think your plan is good. Saw that you got a PR fixing doc, will look into that one too. Thank you! This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [airflow] mik-laj opened a new issue #8108: Basic OpenAPI spec
mik-laj opened a new issue #8108: Basic OpenAPI spec URL: https://github.com/apache/airflow/issues/8108 **Description** We need to develop an OpenAPI spec that contains at least the following elements: - endpoints - scheme - response bodies Subsequent PR may extend the Open API spec of subsequent elements, e.g. HATEOAS, but HATEOAS must be defined after the endpoints, so it seems logical to split this task. PR is available: https://github.com/apache/airflow/issues/8107 More information about the REST API is available: [AIP-32 - Airflow REST API - High-level info](https://github.com/apache/airflow/issues/8107) **Use case / motivation** N/A **Related Issues** https://github.com/apache/airflow/issues/8107 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [airflow] codecov-io commented on issue #8218: CSS linting integrated into pre-commit
codecov-io commented on issue #8218: CSS linting integrated into pre-commit URL: https://github.com/apache/airflow/pull/8218#issuecomment-611740217 # [Codecov](https://codecov.io/gh/apache/airflow/pull/8218?src=pr&el=h1) Report > Merging [#8218](https://codecov.io/gh/apache/airflow/pull/8218?src=pr&el=desc) into [master](https://codecov.io/gh/apache/airflow/commit/1317cdc63546a35ad9cf229f9026ac0d08f01dfb&el=desc) will **decrease** coverage by `0.47%`. > The diff coverage is `n/a`. [](https://codecov.io/gh/apache/airflow/pull/8218?src=pr&el=tree) ```diff @@Coverage Diff @@ ## master#8218 +/- ## == - Coverage 88.32% 87.85% -0.48% == Files 936 936 Lines 4531645319 +3 == - Hits4002739813 -214 - Misses 5289 5506 +217 ``` | [Impacted Files](https://codecov.io/gh/apache/airflow/pull/8218?src=pr&el=tree) | Coverage Δ | | |---|---|---| | [...flow/providers/apache/cassandra/hooks/cassandra.py](https://codecov.io/gh/apache/airflow/pull/8218/diff?src=pr&el=tree#diff-YWlyZmxvdy9wcm92aWRlcnMvYXBhY2hlL2Nhc3NhbmRyYS9ob29rcy9jYXNzYW5kcmEucHk=) | `21.25% <0.00%> (-72.50%)` | :arrow_down: | | [...w/providers/apache/hive/operators/mysql\_to\_hive.py](https://codecov.io/gh/apache/airflow/pull/8218/diff?src=pr&el=tree#diff-YWlyZmxvdy9wcm92aWRlcnMvYXBhY2hlL2hpdmUvb3BlcmF0b3JzL215c3FsX3RvX2hpdmUucHk=) | `35.84% <0.00%> (-64.16%)` | :arrow_down: | | [airflow/providers/postgres/operators/postgres.py](https://codecov.io/gh/apache/airflow/pull/8218/diff?src=pr&el=tree#diff-YWlyZmxvdy9wcm92aWRlcnMvcG9zdGdyZXMvb3BlcmF0b3JzL3Bvc3RncmVzLnB5) | `47.82% <0.00%> (-52.18%)` | :arrow_down: | | [airflow/providers/redis/operators/redis\_publish.py](https://codecov.io/gh/apache/airflow/pull/8218/diff?src=pr&el=tree#diff-YWlyZmxvdy9wcm92aWRlcnMvcmVkaXMvb3BlcmF0b3JzL3JlZGlzX3B1Ymxpc2gucHk=) | `50.00% <0.00%> (-50.00%)` | :arrow_down: | | [airflow/providers/mongo/sensors/mongo.py](https://codecov.io/gh/apache/airflow/pull/8218/diff?src=pr&el=tree#diff-YWlyZmxvdy9wcm92aWRlcnMvbW9uZ28vc2Vuc29ycy9tb25nby5weQ==) | `53.33% <0.00%> (-46.67%)` | :arrow_down: | | [airflow/providers/mysql/operators/mysql.py](https://codecov.io/gh/apache/airflow/pull/8218/diff?src=pr&el=tree#diff-YWlyZmxvdy9wcm92aWRlcnMvbXlzcWwvb3BlcmF0b3JzL215c3FsLnB5) | `55.00% <0.00%> (-45.00%)` | :arrow_down: | | [airflow/providers/redis/sensors/redis\_key.py](https://codecov.io/gh/apache/airflow/pull/8218/diff?src=pr&el=tree#diff-YWlyZmxvdy9wcm92aWRlcnMvcmVkaXMvc2Vuc29ycy9yZWRpc19rZXkucHk=) | `61.53% <0.00%> (-38.47%)` | :arrow_down: | | [airflow/executors/celery\_executor.py](https://codecov.io/gh/apache/airflow/pull/8218/diff?src=pr&el=tree#diff-YWlyZmxvdy9leGVjdXRvcnMvY2VsZXJ5X2V4ZWN1dG9yLnB5) | `50.67% <0.00%> (-37.84%)` | :arrow_down: | | [...roviders/google/cloud/operators/postgres\_to\_gcs.py](https://codecov.io/gh/apache/airflow/pull/8218/diff?src=pr&el=tree#diff-YWlyZmxvdy9wcm92aWRlcnMvZ29vZ2xlL2Nsb3VkL29wZXJhdG9ycy9wb3N0Z3Jlc190b19nY3MucHk=) | `54.28% <0.00%> (-31.43%)` | :arrow_down: | | [airflow/providers/postgres/hooks/postgres.py](https://codecov.io/gh/apache/airflow/pull/8218/diff?src=pr&el=tree#diff-YWlyZmxvdy9wcm92aWRlcnMvcG9zdGdyZXMvaG9va3MvcG9zdGdyZXMucHk=) | `78.87% <0.00%> (-14.09%)` | :arrow_down: | | ... and [5 more](https://codecov.io/gh/apache/airflow/pull/8218/diff?src=pr&el=tree-more) | | -- [Continue to review full report at Codecov](https://codecov.io/gh/apache/airflow/pull/8218?src=pr&el=continue). > **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta) > `Δ = absolute (impact)`, `ø = not affected`, `? = missing data` > Powered by [Codecov](https://codecov.io/gh/apache/airflow/pull/8218?src=pr&el=footer). Last update [1317cdc...bce6797](https://codecov.io/gh/apache/airflow/pull/8218?src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments). This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Updated] (AIRFLOW-6167) Escape col name in MysqlToHive operator
[ https://issues.apache.org/jira/browse/AIRFLOW-6167?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kaxil Naik updated AIRFLOW-6167: Fix Version/s: (was: 1.10.10) 1.10.11 > Escape col name in MysqlToHive operator > --- > > Key: AIRFLOW-6167 > URL: https://issues.apache.org/jira/browse/AIRFLOW-6167 > Project: Apache Airflow > Issue Type: Bug > Components: operators >Affects Versions: 1.10.4 >Reporter: Ping Zhang >Assignee: Ping Zhang >Priority: Major > Fix For: 1.10.11 > > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (AIRFLOW-2516) Deadlock found when trying to update task_instance table
[
https://issues.apache.org/jira/browse/AIRFLOW-2516?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Kaxil Naik updated AIRFLOW-2516:
Fix Version/s: (was: 1.10.10)
1.10.11
> Deadlock found when trying to update task_instance table
>
>
> Key: AIRFLOW-2516
> URL: https://issues.apache.org/jira/browse/AIRFLOW-2516
> Project: Apache Airflow
> Issue Type: Bug
> Components: DagRun
>Affects Versions: 1.8.0, 1.9.0, 1.10.0, 1.10.1, 1.10.2, 1.10.3, 1.10.4,
> 1.10.5, 1.10.6, 1.10.7
>Reporter: Jeff Liu
>Assignee: Jarek Potiuk
>Priority: Major
> Fix For: 1.10.11
>
> Attachments: Screenshot 2019-12-30 at 10.42.52.png,
> image-2019-12-30-10-48-41-313.png, image-2019-12-30-10-58-02-610.png,
> jobs.py, jobs_fixed_deadlock_possibly_1.9.py,
> scheduler_job_fixed_deadlock_possibly_1.10.6.py
>
>
>
>
> {code:java}
> [2018-05-23 17:59:57,218] {base_task_runner.py:98} INFO - Subtask:
> [2018-05-23 17:59:57,217] {base_executor.py:49} INFO - Adding to queue:
> airflow run production_wipeout_wipe_manager.Carat Carat_20180227
> 2018-05-23T17:41:18.815809 --local -sd DAGS_FOLDER/wipeout/wipeout.py
> [2018-05-23 17:59:57,231] {base_task_runner.py:98} INFO - Subtask: Traceback
> (most recent call last):
> [2018-05-23 17:59:57,232] {base_task_runner.py:98} INFO - Subtask: File
> "/usr/local/bin/airflow", line 27, in
> [2018-05-23 17:59:57,232] {base_task_runner.py:98} INFO - Subtask:
> args.func(args)
> [2018-05-23 17:59:57,232] {base_task_runner.py:98} INFO - Subtask: File
> "/usr/local/lib/python2.7/site-packages/airflow/bin/cli.py", line 392, in run
> [2018-05-23 17:59:57,232] {base_task_runner.py:98} INFO - Subtask:
> pool=args.pool,
> [2018-05-23 17:59:57,233] {base_task_runner.py:98} INFO - Subtask: File
> "/usr/local/lib/python2.7/site-packages/airflow/utils/db.py", line 50, in
> wrapper
> [2018-05-23 17:59:57,233] {base_task_runner.py:98} INFO - Subtask: result =
> func(*args, **kwargs)
> [2018-05-23 17:59:57,233] {base_task_runner.py:98} INFO - Subtask: File
> "/usr/local/lib/python2.7/site-packages/airflow/models.py", line 1532, in
> _run_raw_task
> [2018-05-23 17:59:57,234] {base_task_runner.py:98} INFO - Subtask:
> self.handle_failure(e, test_mode, context)
> [2018-05-23 17:59:57,234] {base_task_runner.py:98} INFO - Subtask: File
> "/usr/local/lib/python2.7/site-packages/airflow/models.py", line 1641, in
> handle_failure
> [2018-05-23 17:59:57,234] {base_task_runner.py:98} INFO - Subtask:
> session.merge(self)
> [2018-05-23 17:59:57,235] {base_task_runner.py:98} INFO - Subtask: File
> "/usr/local/lib/python2.7/site-packages/sqlalchemy/orm/session.py", line
> 1920, in merge
> [2018-05-23 17:59:57,235] {base_task_runner.py:98} INFO - Subtask:
> _resolve_conflict_map=_resolve_conflict_map)
> [2018-05-23 17:59:57,235] {base_task_runner.py:98} INFO - Subtask: File
> "/usr/local/lib/python2.7/site-packages/sqlalchemy/orm/session.py", line
> 1974, in _merge
> [2018-05-23 17:59:57,236] {base_task_runner.py:98} INFO - Subtask: merged =
> self.query(mapper.class_).get(key[1])
> [2018-05-23 17:59:57,236] {base_task_runner.py:98} INFO - Subtask: File
> "/usr/local/lib/python2.7/site-packages/sqlalchemy/orm/query.py", line 882,
> in get
> [2018-05-23 17:59:57,236] {base_task_runner.py:98} INFO - Subtask: ident,
> loading.load_on_pk_identity)
> [2018-05-23 17:59:57,236] {base_task_runner.py:98} INFO - Subtask: File
> "/usr/local/lib/python2.7/site-packages/sqlalchemy/orm/query.py", line 952,
> in _get_impl
> [2018-05-23 17:59:57,237] {base_task_runner.py:98} INFO - Subtask: return
> db_load_fn(self, primary_key_identity)
> [2018-05-23 17:59:57,237] {base_task_runner.py:98} INFO - Subtask: File
> "/usr/local/lib/python2.7/site-packages/sqlalchemy/orm/loading.py", line 247,
> in load_on_pk_i
> dentity
> [2018-05-23 17:59:57,237] {base_task_runner.py:98} INFO - Subtask: return
> q.one()
> [2018-05-23 17:59:57,238] {base_task_runner.py:98} INFO - Subtask: File
> "/usr/local/lib/python2.7/site-packages/sqlalchemy/orm/query.py", line 2884,
> in one
> [2018-05-23 17:59:57,238] {base_task_runner.py:98} INFO - Subtask: ret =
> self.one_or_none()
> [2018-05-23 17:59:57,238] {base_task_runner.py:98} INFO - Subtask: File
> "/usr/local/lib/python2.7/site-packages/sqlalchemy/orm/query.py", line 2854,
> in one_or_none
> [2018-05-23 17:59:57,238] {base_task_runner.py:98} INFO - Subtask: ret =
> list(self)
> [2018-05-23 17:59:57,239] {base_task_runner.py:98} INFO - Subtask: File
> "/usr/local/lib/python2.7/site-packages/sqlalchemy/orm/query.py", line 2925,
> in __iter__
> [2018-05-23 17:59:57,239] {base_task_runner.py:98} INFO - Subtask: return
> self._execute_and_instances(context)
> [2018-05-23 17:59:57,239] {base_task_runner.py:98} INFO -
[GitHub] [airflow] ashb closed issue #7968: Add dag_run to the task_instance table or create new taskuuid column and use this to uniquely identify a task
ashb closed issue #7968: Add dag_run to the task_instance table or create new taskuuid column and use this to uniquely identify a task URL: https://github.com/apache/airflow/issues/7968 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [airflow] ashb commented on issue #7968: Add dag_run to the task_instance table or create new taskuuid column and use this to uniquely identify a task
ashb commented on issue #7968: Add dag_run to the task_instance table or create new taskuuid column and use this to uniquely identify a task URL: https://github.com/apache/airflow/issues/7968#issuecomment-611739916 Duplicate of #7967 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Updated] (AIRFLOW-7046) Support Locale-formatted datetimes in Webserver
[ https://issues.apache.org/jira/browse/AIRFLOW-7046?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kaxil Naik updated AIRFLOW-7046: Fix Version/s: (was: 1.10.10) 1.10.11 > Support Locale-formatted datetimes in Webserver > --- > > Key: AIRFLOW-7046 > URL: https://issues.apache.org/jira/browse/AIRFLOW-7046 > Project: Apache Airflow > Issue Type: Sub-task > Components: ui, webserver >Affects Versions: 1.10.10 >Reporter: Kaxil Naik >Assignee: Samantha Black >Priority: Minor > Fix For: 1.10.11 > > > Support human-readable time in local language -- This message was sent by Atlassian Jira (v8.3.4#803005)
[GitHub] [airflow] ashb commented on issue #8032: Security: Variable level ownership/permission
ashb commented on issue #8032: Security: Variable level ownership/permission URL: https://github.com/apache/airflow/issues/8032#issuecomment-611738465 This is a nice feature, but how would this interact with DAGs trying to access variables? And also for DAGs accessing variables: it would be best effort and not an actual protection as DAGs run with full database access, so could by-pass any code-level protection on Variable. (This would be true if/until we move workers from not having direct database access.) This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [airflow] ashb commented on issue #8062: 'run_as_user' failed to run in controlled environment
ashb commented on issue #8062: 'run_as_user' failed to run in controlled environment URL: https://github.com/apache/airflow/issues/8062#issuecomment-611737498 I think this is a won't fix. Without sudo how can you run tasks as another user anyway? This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [airflow] ashb commented on issue #8061: Support cross environment setup with Celery
ashb commented on issue #8061: Support cross environment setup with Celery URL: https://github.com/apache/airflow/issues/8061#issuecomment-611736891 I think this got broken when the "SimpleDagBag" concept was introduced. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [airflow] ashb commented on issue #8160: SimpleHttpOperator aborts connection after 5 minutes
ashb commented on issue #8160: SimpleHttpOperator aborts connection after 5 minutes URL: https://github.com/apache/airflow/issues/8160#issuecomment-611736250 This one appears to be something specific to your environment or the request you are making, as I can't reproduce this behavour, nor could khyurri. Without more detailed reproduction steps that we can try ourselves we won't be able to help with this one. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [airflow] ashb commented on issue #8179: Airflow LDAP authentication with RBAC features
ashb commented on issue #8179: Airflow LDAP authentication with RBAC features URL: https://github.com/apache/airflow/issues/8179#issuecomment-611735117 This is a question/feature request for the framework we use, Flask-AppBuilder and has been asked for upstream as https://github.com/dpgaspar/Flask-AppBuilder/issues/956 Closing this one for now. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [airflow] ashb closed issue #8179: Airflow LDAP authentication with RBAC features
ashb closed issue #8179: Airflow LDAP authentication with RBAC features URL: https://github.com/apache/airflow/issues/8179 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [airflow] ashb merged pull request #8220: Make Gantt tooltip the same as Tree and Graph view
ashb merged pull request #8220: Make Gantt tooltip the same as Tree and Graph view URL: https://github.com/apache/airflow/pull/8220 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [airflow] ashb closed issue #8210: UI time zone not used in Gantt charts tooltips
ashb closed issue #8210: UI time zone not used in Gantt charts tooltips URL: https://github.com/apache/airflow/issues/8210 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
svn commit: r38876 - /release/airflow/1.10.9/
Author: kaxilnaik Date: Thu Apr 9 19:51:35 2020 New Revision: 38876 Log: Remove old release: 1.10.9 Removed: release/airflow/1.10.9/
svn commit: r38875 - /release/airflow/1.10.10/
Author: kaxilnaik Date: Thu Apr 9 19:51:13 2020 New Revision: 38875 Log: Release Airflow 1.10.10 from 1.10.10rc5 Added: release/airflow/1.10.10/ release/airflow/1.10.10/apache-airflow-1.10.10-bin.tar.gz - copied unchanged from r38872, dev/airflow/1.10.10rc5/apache-airflow-1.10.10rc5-bin.tar.gz release/airflow/1.10.10/apache-airflow-1.10.10-bin.tar.gz.asc - copied unchanged from r38872, dev/airflow/1.10.10rc5/apache-airflow-1.10.10rc5-bin.tar.gz.asc release/airflow/1.10.10/apache-airflow-1.10.10-bin.tar.gz.sha512 - copied unchanged from r38872, dev/airflow/1.10.10rc5/apache-airflow-1.10.10rc5-bin.tar.gz.sha512 release/airflow/1.10.10/apache-airflow-1.10.10-source.tar.gz - copied unchanged from r38872, dev/airflow/1.10.10rc5/apache-airflow-1.10.10rc5-source.tar.gz release/airflow/1.10.10/apache-airflow-1.10.10-source.tar.gz.asc - copied unchanged from r38872, dev/airflow/1.10.10rc5/apache-airflow-1.10.10rc5-source.tar.gz.asc release/airflow/1.10.10/apache-airflow-1.10.10-source.tar.gz.sha512 - copied unchanged from r38872, dev/airflow/1.10.10rc5/apache-airflow-1.10.10rc5-source.tar.gz.sha512 release/airflow/1.10.10/apache_airflow-1.10.10-py2.py3-none-any.whl - copied unchanged from r38872, dev/airflow/1.10.10rc5/apache_airflow-1.10.10rc5-py2.py3-none-any.whl release/airflow/1.10.10/apache_airflow-1.10.10-py2.py3-none-any.whl.asc - copied unchanged from r38872, dev/airflow/1.10.10rc5/apache_airflow-1.10.10rc5-py2.py3-none-any.whl.asc release/airflow/1.10.10/apache_airflow-1.10.10-py2.py3-none-any.whl.sha512 - copied unchanged from r38872, dev/airflow/1.10.10rc5/apache_airflow-1.10.10rc5-py2.py3-none-any.whl.sha512
[GitHub] [airflow] turbaszek edited a comment on issue #8222: github action tests
turbaszek edited a comment on issue #8222: github action tests URL: https://github.com/apache/airflow/pull/8222#issuecomment-611713402 @dimberman one question: do we run any tests? I checked one job and it seems all were skipped. It seems that we need integration flag and runtime. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [airflow] turbaszek commented on a change in pull request #8222: github action tests
turbaszek commented on a change in pull request #8222: github action tests URL: https://github.com/apache/airflow/pull/8222#discussion_r406426416 ## File path: .github/workflows/ci-k8s.yml ## @@ -0,0 +1,47 @@ +name: "Create cluster using KinD" +on: [pull_request, push] + +jobs: + create-airflow-dockerfile: +runs-on: ubuntu-latest +strategy: + matrix: +k8s-version: [v1.15.7, v1.16.4, v1.17.2, v1.18.0] +steps: + - uses: actions/checkout@v2 +name: checkout + - name: Create k8s Kind Cluster Review comment: ```suggestion - name: create k8s Kind Cluster ``` consistency :) This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [airflow] turbaszek commented on issue #8222: github action tests
turbaszek commented on issue #8222: github action tests URL: https://github.com/apache/airflow/pull/8222#issuecomment-611713402 @dimberman one question: do we run any tests? I checked one job and it seems all were skipped This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [airflow] kaxil commented on a change in pull request #8222: github action tests
kaxil commented on a change in pull request #8222: github action tests
URL: https://github.com/apache/airflow/pull/8222#discussion_r406428567
##
File path: scripts/ci/ci_run_airflow_testing.sh
##
@@ -79,36 +79,3 @@ done
RUN_INTEGRATION_TESTS=${RUN_INTEGRATION_TESTS:=""}
-if [[ ${RUNTIME:=} == "kubernetes" ]]; then
-export KUBERNETES_MODE=${KUBERNETES_MODE:="git_mode"}
-export KUBERNETES_VERSION=${KUBERNETES_VERSION:="v1.15.3"}
-
-set +u
-# shellcheck disable=SC2016
-docker-compose --log-level INFO \
- -f "${MY_DIR}/docker-compose/base.yml" \
- -f "${MY_DIR}/docker-compose/backend-${BACKEND}.yml" \
- -f "${MY_DIR}/docker-compose/runtime-kubernetes.yml" \
- "${INTEGRATIONS[@]}" \
- "${DOCKER_COMPOSE_LOCAL[@]}" \
- run airflow \
- '/opt/airflow/scripts/ci/in_container/entrypoint_ci.sh "${@}"' \
- /opt/airflow/scripts/ci/in_container/entrypoint_ci.sh "${@}"
- # Note the command is there twice (!) because it is passed via bash -c
- # and bash -c starts passing parameters from $0. TODO: fixme
-set -u
-else
-set +u
-# shellcheck disable=SC2016
-docker-compose --log-level INFO \
- -f "${MY_DIR}/docker-compose/base.yml" \
- -f "${MY_DIR}/docker-compose/backend-${BACKEND}.yml" \
- "${INTEGRATIONS[@]}" \
- "${DOCKER_COMPOSE_LOCAL[@]}" \
- run airflow \
- '/opt/airflow/scripts/ci/in_container/entrypoint_ci.sh "${@}"' \
- /opt/airflow/scripts/ci/in_container/entrypoint_ci.sh "${@}"
- # Note the command is there twice (!) because it is passed via bash -c
- # and bash -c starts passing parameters from $0. TODO: fixme
-set -u
Review comment:
This (`else` part) should still remain I think
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [airflow] turbaszek commented on a change in pull request #8222: github action tests
turbaszek commented on a change in pull request #8222: github action tests URL: https://github.com/apache/airflow/pull/8222#discussion_r406427507 ## File path: .github/workflows/ci-k8s.yml ## @@ -0,0 +1,47 @@ +name: "Create cluster using KinD" +on: [pull_request, push] + +jobs: + create-airflow-dockerfile: Review comment: Should we adjust this name? This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [airflow] turbaszek commented on a change in pull request #8222: github action tests
turbaszek commented on a change in pull request #8222: github action tests URL: https://github.com/apache/airflow/pull/8222#discussion_r406426416 ## File path: .github/workflows/ci-k8s.yml ## @@ -0,0 +1,47 @@ +name: "Create cluster using KinD" +on: [pull_request, push] + +jobs: + create-airflow-dockerfile: +runs-on: ubuntu-latest +strategy: + matrix: +k8s-version: [v1.15.7, v1.16.4, v1.17.2, v1.18.0] +steps: + - uses: actions/checkout@v2 +name: checkout + - name: Create k8s Kind Cluster Review comment: consistency :) This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [airflow] turbaszek commented on a change in pull request #8222: github action tests
turbaszek commented on a change in pull request #8222: github action tests URL: https://github.com/apache/airflow/pull/8222#discussion_r406425874 ## File path: .github/workflows/ci-k8s.yml ## @@ -0,0 +1,47 @@ +name: "Create cluster using KinD" Review comment: ```suggestion name: "k8s tests" ``` What do you think? This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [airflow] turbaszek commented on a change in pull request #8222: github action tests
turbaszek commented on a change in pull request #8222: github action tests
URL: https://github.com/apache/airflow/pull/8222#discussion_r406426382
##
File path: .github/workflows/ci-k8s.yml
##
@@ -0,0 +1,47 @@
+name: "Create cluster using KinD"
+on: [pull_request, push]
+
+jobs:
+ create-airflow-dockerfile:
+runs-on: ubuntu-latest
+strategy:
+ matrix:
+k8s-version: [v1.15.7, v1.16.4, v1.17.2, v1.18.0]
+steps:
+ - uses: actions/checkout@v2
+name: checkout
+ - name: Create k8s Kind Cluster
+uses: helm/kind-action@v1.0.0-rc.1
+with:
+ cluster_name: kind
+ node_image: kindest/node:${{ matrix.k8s-version }}
+# config: ./kind-cluster-conf.yaml
+ - name: Test cluster running
Review comment:
```suggestion
- name: test cluster running
```
consistency :)
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [airflow] wkhudgins92 commented on a change in pull request #8170: [AIRFLOW-6515] Change Log Levels from Info/Warn to Error
wkhudgins92 commented on a change in pull request #8170: [AIRFLOW-6515] Change
Log Levels from Info/Warn to Error
URL: https://github.com/apache/airflow/pull/8170#discussion_r406422935
##
File path: airflow/providers/hashicorp/secrets/vault.py
##
@@ -189,7 +189,7 @@ def _get_secret(self, path_prefix: str, secret_id: str) ->
Optional[dict]:
response = self.client.secrets.kv.v2.read_secret_version(
path=secret_path, mount_point=self.mount_point)
except InvalidPath:
-self.log.info("Secret %s not found in Path: %s", secret_id,
secret_path)
+self.log.error("Secret %s not found in Path: %s", secret_id,
secret_path)
Review comment:
Thanks, good catch!
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [airflow] codecov-io commented on issue #8233: Dag tags use set instead of list
codecov-io commented on issue #8233: Dag tags use set instead of list URL: https://github.com/apache/airflow/pull/8233#issuecomment-611694409 # [Codecov](https://codecov.io/gh/apache/airflow/pull/8233?src=pr&el=h1) Report > Merging [#8233](https://codecov.io/gh/apache/airflow/pull/8233?src=pr&el=desc) into [master](https://codecov.io/gh/apache/airflow/commit/ad42c4076479fe7420dc329ee8bfd51d9fb0a2ae&el=desc) will **decrease** coverage by `0.29%`. > The diff coverage is `100.00%`. [](https://codecov.io/gh/apache/airflow/pull/8233?src=pr&el=tree) ```diff @@Coverage Diff @@ ## master#8233 +/- ## == - Coverage 88.35% 88.05% -0.30% == Files 936 936 Lines 4530545316 +11 == - Hits4003039905 -125 - Misses 5275 5411 +136 ``` | [Impacted Files](https://codecov.io/gh/apache/airflow/pull/8233?src=pr&el=tree) | Coverage Δ | | |---|---|---| | [airflow/example\_dags/example\_bash\_operator.py](https://codecov.io/gh/apache/airflow/pull/8233/diff?src=pr&el=tree#diff-YWlyZmxvdy9leGFtcGxlX2RhZ3MvZXhhbXBsZV9iYXNoX29wZXJhdG9yLnB5) | `94.44% <ø> (ø)` | | | [airflow/example\_dags/example\_branch\_operator.py](https://codecov.io/gh/apache/airflow/pull/8233/diff?src=pr&el=tree#diff-YWlyZmxvdy9leGFtcGxlX2RhZ3MvZXhhbXBsZV9icmFuY2hfb3BlcmF0b3IucHk=) | `100.00% <ø> (ø)` | | | [...ample\_dags/example\_branch\_python\_dop\_operator\_3.py](https://codecov.io/gh/apache/airflow/pull/8233/diff?src=pr&el=tree#diff-YWlyZmxvdy9leGFtcGxlX2RhZ3MvZXhhbXBsZV9icmFuY2hfcHl0aG9uX2RvcF9vcGVyYXRvcl8zLnB5) | `75.00% <ø> (ø)` | | | [airflow/example\_dags/example\_complex.py](https://codecov.io/gh/apache/airflow/pull/8233/diff?src=pr&el=tree#diff-YWlyZmxvdy9leGFtcGxlX2RhZ3MvZXhhbXBsZV9jb21wbGV4LnB5) | `100.00% <ø> (ø)` | | | [...w/example\_dags/example\_external\_task\_marker\_dag.py](https://codecov.io/gh/apache/airflow/pull/8233/diff?src=pr&el=tree#diff-YWlyZmxvdy9leGFtcGxlX2RhZ3MvZXhhbXBsZV9leHRlcm5hbF90YXNrX21hcmtlcl9kYWcucHk=) | `100.00% <ø> (ø)` | | | [...irflow/example\_dags/example\_kubernetes\_executor.py](https://codecov.io/gh/apache/airflow/pull/8233/diff?src=pr&el=tree#diff-YWlyZmxvdy9leGFtcGxlX2RhZ3MvZXhhbXBsZV9rdWJlcm5ldGVzX2V4ZWN1dG9yLnB5) | `85.00% <ø> (ø)` | | | [...example\_dags/example\_kubernetes\_executor\_config.py](https://codecov.io/gh/apache/airflow/pull/8233/diff?src=pr&el=tree#diff-YWlyZmxvdy9leGFtcGxlX2RhZ3MvZXhhbXBsZV9rdWJlcm5ldGVzX2V4ZWN1dG9yX2NvbmZpZy5weQ==) | `75.00% <ø> (ø)` | | | [airflow/example\_dags/example\_latest\_only.py](https://codecov.io/gh/apache/airflow/pull/8233/diff?src=pr&el=tree#diff-YWlyZmxvdy9leGFtcGxlX2RhZ3MvZXhhbXBsZV9sYXRlc3Rfb25seS5weQ==) | `100.00% <ø> (ø)` | | | [...w/example\_dags/example\_latest\_only\_with\_trigger.py](https://codecov.io/gh/apache/airflow/pull/8233/diff?src=pr&el=tree#diff-YWlyZmxvdy9leGFtcGxlX2RhZ3MvZXhhbXBsZV9sYXRlc3Rfb25seV93aXRoX3RyaWdnZXIucHk=) | `100.00% <ø> (ø)` | | | [...le\_dags/example\_passing\_params\_via\_test\_command.py](https://codecov.io/gh/apache/airflow/pull/8233/diff?src=pr&el=tree#diff-YWlyZmxvdy9leGFtcGxlX2RhZ3MvZXhhbXBsZV9wYXNzaW5nX3BhcmFtc192aWFfdGVzdF9jb21tYW5kLnB5) | `100.00% <ø> (ø)` | | | ... and [76 more](https://codecov.io/gh/apache/airflow/pull/8233/diff?src=pr&el=tree-more) | | -- [Continue to review full report at Codecov](https://codecov.io/gh/apache/airflow/pull/8233?src=pr&el=continue). > **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta) > `Δ = absolute (impact)`, `ø = not affected`, `? = missing data` > Powered by [Codecov](https://codecov.io/gh/apache/airflow/pull/8233?src=pr&el=footer). Last update [ad42c40...f1b794f](https://codecov.io/gh/apache/airflow/pull/8233?src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments). This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [airflow] kaxil merged pull request #8225: Move DAG._schedule_interval logic out of DAG.__init__
kaxil merged pull request #8225: Move DAG._schedule_interval logic out of DAG.__init__ URL: https://github.com/apache/airflow/pull/8225 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [airflow] kaxil closed issue #8166: Task Duration Bug: Doesn't respect number of runs
kaxil closed issue #8166: Task Duration Bug: Doesn't respect number of runs URL: https://github.com/apache/airflow/issues/8166 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [airflow] codecov-io commented on issue #8225: Move DAG._schedule_interval logic out of DAG.__init__
codecov-io commented on issue #8225: Move DAG._schedule_interval logic out of DAG.__init__ URL: https://github.com/apache/airflow/pull/8225#issuecomment-611691257 # [Codecov](https://codecov.io/gh/apache/airflow/pull/8225?src=pr&el=h1) Report > Merging [#8225](https://codecov.io/gh/apache/airflow/pull/8225?src=pr&el=desc) into [master](https://codecov.io/gh/apache/airflow/commit/9468187c5a4abb4b060e6a61928a9673f98b556f&el=desc) will **decrease** coverage by `0.72%`. > The diff coverage is `100.00%`. [](https://codecov.io/gh/apache/airflow/pull/8225?src=pr&el=tree) ```diff @@Coverage Diff @@ ## master#8225 +/- ## == - Coverage 88.31% 87.58% -0.73% == Files 936 936 Lines 4527245319 +47 == - Hits3998039694 -286 - Misses 5292 5625 +333 ``` | [Impacted Files](https://codecov.io/gh/apache/airflow/pull/8225?src=pr&el=tree) | Coverage Δ | | |---|---|---| | [airflow/models/dag.py](https://codecov.io/gh/apache/airflow/pull/8225/diff?src=pr&el=tree#diff-YWlyZmxvdy9tb2RlbHMvZGFnLnB5) | `91.68% <100.00%> (+0.03%)` | :arrow_up: | | [airflow/models/dagbag.py](https://codecov.io/gh/apache/airflow/pull/8225/diff?src=pr&el=tree#diff-YWlyZmxvdy9tb2RlbHMvZGFnYmFnLnB5) | `89.83% <100.00%> (ø)` | | | [...flow/providers/apache/cassandra/hooks/cassandra.py](https://codecov.io/gh/apache/airflow/pull/8225/diff?src=pr&el=tree#diff-YWlyZmxvdy9wcm92aWRlcnMvYXBhY2hlL2Nhc3NhbmRyYS9ob29rcy9jYXNzYW5kcmEucHk=) | `21.25% <0.00%> (-72.50%)` | :arrow_down: | | [...w/providers/apache/hive/operators/mysql\_to\_hive.py](https://codecov.io/gh/apache/airflow/pull/8225/diff?src=pr&el=tree#diff-YWlyZmxvdy9wcm92aWRlcnMvYXBhY2hlL2hpdmUvb3BlcmF0b3JzL215c3FsX3RvX2hpdmUucHk=) | `35.84% <0.00%> (-64.16%)` | :arrow_down: | | [airflow/kubernetes/volume\_mount.py](https://codecov.io/gh/apache/airflow/pull/8225/diff?src=pr&el=tree#diff-YWlyZmxvdy9rdWJlcm5ldGVzL3ZvbHVtZV9tb3VudC5weQ==) | `44.44% <0.00%> (-55.56%)` | :arrow_down: | | [airflow/providers/postgres/operators/postgres.py](https://codecov.io/gh/apache/airflow/pull/8225/diff?src=pr&el=tree#diff-YWlyZmxvdy9wcm92aWRlcnMvcG9zdGdyZXMvb3BlcmF0b3JzL3Bvc3RncmVzLnB5) | `47.82% <0.00%> (-52.18%)` | :arrow_down: | | [airflow/providers/redis/operators/redis\_publish.py](https://codecov.io/gh/apache/airflow/pull/8225/diff?src=pr&el=tree#diff-YWlyZmxvdy9wcm92aWRlcnMvcmVkaXMvb3BlcmF0b3JzL3JlZGlzX3B1Ymxpc2gucHk=) | `50.00% <0.00%> (-50.00%)` | :arrow_down: | | [airflow/kubernetes/volume.py](https://codecov.io/gh/apache/airflow/pull/8225/diff?src=pr&el=tree#diff-YWlyZmxvdy9rdWJlcm5ldGVzL3ZvbHVtZS5weQ==) | `52.94% <0.00%> (-47.06%)` | :arrow_down: | | [airflow/providers/mongo/sensors/mongo.py](https://codecov.io/gh/apache/airflow/pull/8225/diff?src=pr&el=tree#diff-YWlyZmxvdy9wcm92aWRlcnMvbW9uZ28vc2Vuc29ycy9tb25nby5weQ==) | `53.33% <0.00%> (-46.67%)` | :arrow_down: | | [airflow/kubernetes/pod\_launcher.py](https://codecov.io/gh/apache/airflow/pull/8225/diff?src=pr&el=tree#diff-YWlyZmxvdy9rdWJlcm5ldGVzL3BvZF9sYXVuY2hlci5weQ==) | `47.18% <0.00%> (-45.08%)` | :arrow_down: | | ... and [15 more](https://codecov.io/gh/apache/airflow/pull/8225/diff?src=pr&el=tree-more) | | -- [Continue to review full report at Codecov](https://codecov.io/gh/apache/airflow/pull/8225?src=pr&el=continue). > **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta) > `Δ = absolute (impact)`, `ø = not affected`, `? = missing data` > Powered by [Codecov](https://codecov.io/gh/apache/airflow/pull/8225?src=pr&el=footer). Last update [9468187...99e10a7](https://codecov.io/gh/apache/airflow/pull/8225?src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments). This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [airflow] kaxil commented on a change in pull request #8232: Simplify DAG.set_dag_runs_state method
kaxil commented on a change in pull request #8232: Simplify
DAG.set_dag_runs_state method
URL: https://github.com/apache/airflow/pull/8232#discussion_r406400542
##
File path: airflow/models/dag.py
##
@@ -912,12 +912,7 @@ def set_dag_runs_state(
query = query.filter(DagRun.execution_date >= start_date)
if end_date:
query = query.filter(DagRun.execution_date <= end_date)
-drs = query.all()
-
-dirty_ids = []
-for dr in drs:
-dr.state = state
-dirty_ids.append(dr.dag_id)
+query.update({DagRun.state: state})
Review comment:
> > Can you just double-check & confirm that tests for this function are
already covered?
>
> @kaxil I've added a test for this method
Awesome, feel free to merge once the test passes
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [airflow] kaxil commented on a change in pull request #8218: CSS linting integrated into pre-commit
kaxil commented on a change in pull request #8218: CSS linting integrated into pre-commit URL: https://github.com/apache/airflow/pull/8218#discussion_r406399984 ## File path: airflow/providers/amazon/aws/secrets/secrets_manager.py ## @@ -0,0 +1,125 @@ +# Review comment: Done :) This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [airflow] kaxil commented on a change in pull request #8232: Simplify DAG.set_dag_runs_state method
kaxil commented on a change in pull request #8232: Simplify DAG.set_dag_runs_state method URL: https://github.com/apache/airflow/pull/8232#discussion_r406400299 ## File path: airflow/models/dag.py ## @@ -912,12 +912,7 @@ def set_dag_runs_state( query = query.filter(DagRun.execution_date >= start_date) if end_date: query = query.filter(DagRun.execution_date <= end_date) -drs = query.all() - -dirty_ids = [] -for dr in drs: -dr.state = state -dirty_ids.append(dr.dag_id) Review comment: Yeah that is strange This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [airflow] potiuk commented on a change in pull request #8218: CSS linting integrated into pre-commit
potiuk commented on a change in pull request #8218: CSS linting integrated into pre-commit URL: https://github.com/apache/airflow/pull/8218#discussion_r406398081 ## File path: airflow/providers/amazon/aws/secrets/secrets_manager.py ## @@ -0,0 +1,125 @@ +# Review comment: I think it needs rebase - those are unrelated changes. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [airflow] potiuk commented on issue #8218: CSS linting integrated into pre-commit
potiuk commented on issue #8218: CSS linting integrated into pre-commit URL: https://github.com/apache/airflow/pull/8218#issuecomment-611685195 Love it :) This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [airflow] turbaszek commented on a change in pull request #8232: Simplify DAG.set_dag_runs_state method
turbaszek commented on a change in pull request #8232: Simplify DAG.set_dag_runs_state method URL: https://github.com/apache/airflow/pull/8232#discussion_r406396586 ## File path: airflow/models/dag.py ## @@ -912,12 +912,7 @@ def set_dag_runs_state( query = query.filter(DagRun.execution_date >= start_date) if end_date: query = query.filter(DagRun.execution_date <= end_date) -drs = query.all() - -dirty_ids = [] -for dr in drs: -dr.state = state -dirty_ids.append(dr.dag_id) Review comment: That was my main reason to do this change... This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [airflow] turbaszek commented on issue #8232: Simplify DAG.set_dag_runs_state method
turbaszek commented on issue #8232: Simplify DAG.set_dag_runs_state method URL: https://github.com/apache/airflow/pull/8232#issuecomment-611683855 > Can you just double-check & confirm that tests for this function are already covered? @kaxil I've added a test for this method This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [airflow] turbaszek commented on a change in pull request #8232: Simplify DAG.set_dag_runs_state method
turbaszek commented on a change in pull request #8232: Simplify
DAG.set_dag_runs_state method
URL: https://github.com/apache/airflow/pull/8232#discussion_r406396004
##
File path: airflow/models/dag.py
##
@@ -912,12 +912,7 @@ def set_dag_runs_state(
query = query.filter(DagRun.execution_date >= start_date)
if end_date:
query = query.filter(DagRun.execution_date <= end_date)
-drs = query.all()
-
-dirty_ids = []
-for dr in drs:
-dr.state = state
-dirty_ids.append(dr.dag_id)
+query.update({DagRun.state: state})
Review comment:
Yes, we are using this in many different places
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [airflow] potiuk commented on a change in pull request #8145: Drop redundant project id null checks.
potiuk commented on a change in pull request #8145: Drop redundant project id null checks. URL: https://github.com/apache/airflow/pull/8145#discussion_r406384391 ## File path: airflow/providers/google/cloud/hooks/stackdriver.py ## @@ -500,8 +500,8 @@ def enable_notification_channels( @GoogleBaseHook.fallback_to_default_project_id def disable_notification_channels( self, +project_id: str, Review comment: UPDATING.md entry needed here as well. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [airflow] potiuk commented on a change in pull request #8145: Drop redundant project id null checks.
potiuk commented on a change in pull request #8145: Drop redundant project id null checks. URL: https://github.com/apache/airflow/pull/8145#discussion_r406384068 ## File path: airflow/providers/google/cloud/hooks/stackdriver.py ## @@ -186,10 +186,10 @@ def enable_alert_policies( @GoogleBaseHook.fallback_to_default_project_id def disable_alert_policies( self, -project_id: Optional[str] = None, +project_id: str, filter_: Optional[str] = None, retry: Optional[str] = DEFAULT, -timeout: Optional[str] = DEFAULT, +timeout: Optional[float] = DEFAULT, Review comment: Nice! This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [airflow] potiuk commented on a change in pull request #8145: Drop redundant project id null checks.
potiuk commented on a change in pull request #8145: Drop redundant project id null checks. URL: https://github.com/apache/airflow/pull/8145#discussion_r406383572 ## File path: airflow/providers/google/cloud/hooks/dataflow.py ## @@ -730,10 +718,10 @@ def is_job_dataflow_running(self, name: str, variables: Dict, project_id: Option @GoogleBaseHook.fallback_to_default_project_id def cancel_job( self, +project_id: str, Review comment: Same here -> I think it needs UPDATING.md entry This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
