[GitHub] XD-DENG commented on a change in pull request #4415: [AIRFLOW-3606] Fix Flake8 test & fix the Flake8 errors introduced since Flake8 test was broken
XD-DENG commented on a change in pull request #4415: [AIRFLOW-3606] Fix Flake8 test & fix the Flake8 errors introduced since Flake8 test was broken URL: https://github.com/apache/incubator-airflow/pull/4415#discussion_r244628797 ## File path: .flake8 ## @@ -1,4 +1,3 @@ [flake8] max-line-length = 110 ignore = E731 -exclude = .git, env, venv, .venv, VENV, .tox, .eggs, .* Review comment: I'm not completely sure, but I believe it's the `.*`. But before we trouble-shoot thoroughly, I would suggest to revert the whole line to ensure that Travis CI works as expected. This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] holdenk commented on a change in pull request #4415: [AIRFLOW-3606] Fix Flake8 test & fix the Flake8 errors introduced since Flake8 test was broken
holdenk commented on a change in pull request #4415: [AIRFLOW-3606] Fix Flake8 test & fix the Flake8 errors introduced since Flake8 test was broken URL: https://github.com/apache/incubator-airflow/pull/4415#discussion_r244628726 ## File path: .flake8 ## @@ -1,4 +1,3 @@ [flake8] max-line-length = 110 ignore = E731 -exclude = .git, env, venv, .venv, VENV, .tox, .eggs, .* Review comment: Is the problem from the ".*" or does any exclude cause this problem? This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] codecov-io commented on issue #4415: [AIRFLOW-3606] Fix Flake8 test & fix the Flake8 errors introduced since Flake8 test was broken
codecov-io commented on issue #4415: [AIRFLOW-3606] Fix Flake8 test & fix the Flake8 errors introduced since Flake8 test was broken URL: https://github.com/apache/incubator-airflow/pull/4415#issuecomment-450711430 # [Codecov](https://codecov.io/gh/apache/incubator-airflow/pull/4415?src=pr=h1) Report > Merging [#4415](https://codecov.io/gh/apache/incubator-airflow/pull/4415?src=pr=desc) into [master](https://codecov.io/gh/apache/incubator-airflow/commit/8938d2c727b16762529ef9f1e257ba5fa39f4bea?src=pr=desc) will **decrease** coverage by `<.01%`. > The diff coverage is `100%`. [](https://codecov.io/gh/apache/incubator-airflow/pull/4415?src=pr=tree) ```diff @@Coverage Diff @@ ## master#4415 +/- ## == - Coverage 78.38% 78.37% -0.01% == Files 204 204 Lines 1644516443 -2 == - Hits1289012888 -2 Misses 3555 3555 ``` | [Impacted Files](https://codecov.io/gh/apache/incubator-airflow/pull/4415?src=pr=tree) | Coverage Δ | | |---|---|---| | [airflow/bin/cli.py](https://codecov.io/gh/apache/incubator-airflow/pull/4415/diff?src=pr=tree#diff-YWlyZmxvdy9iaW4vY2xpLnB5) | `65.02% <ø> (-0.05%)` | :arrow_down: | | [airflow/models/\_\_init\_\_.py](https://codecov.io/gh/apache/incubator-airflow/pull/4415/diff?src=pr=tree#diff-YWlyZmxvdy9tb2RlbHMvX19pbml0X18ucHk=) | `92.6% <ø> (-0.01%)` | :arrow_down: | | [airflow/www\_rbac/views.py](https://codecov.io/gh/apache/incubator-airflow/pull/4415/diff?src=pr=tree#diff-YWlyZmxvdy93d3dfcmJhYy92aWV3cy5weQ==) | `73.71% <100%> (ø)` | :arrow_up: | | [airflow/www/views.py](https://codecov.io/gh/apache/incubator-airflow/pull/4415/diff?src=pr=tree#diff-YWlyZmxvdy93d3cvdmlld3MucHk=) | `69.69% <100%> (ø)` | :arrow_up: | -- [Continue to review full report at Codecov](https://codecov.io/gh/apache/incubator-airflow/pull/4415?src=pr=continue). > **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta) > `Δ = absolute (impact)`, `ø = not affected`, `? = missing data` > Powered by [Codecov](https://codecov.io/gh/apache/incubator-airflow/pull/4415?src=pr=footer). Last update [8938d2c...abfb8ff](https://codecov.io/gh/apache/incubator-airflow/pull/4415?src=pr=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments). This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] XD-DENG commented on issue #4415: [AIRFLOW-3606] Fix Flake8 test & fix the Flake8 errors introduced since Flake8 test was broken
XD-DENG commented on issue #4415: [AIRFLOW-3606] Fix Flake8 test & fix the Flake8 errors introduced since Flake8 test was broken URL: https://github.com/apache/incubator-airflow/pull/4415#issuecomment-450710430 Hi @ashb , @Fokko , @kaxil , @feng-tao , may any of you have a quick check on this? **The flake8 test in the Travis CI was broken since https://github.com/apache/incubator-airflow/pull/4361** . You can refer to this sample commit https://github.com/XD-DENG/incubator-airflow/commit/ba046e3901ae, in which I purposely made some Flake8 errors, while this test still passed with green light (https://travis-ci.org/XD-DENG/incubator-airflow/jobs/474002074). https://github.com/apache/incubator-airflow/pull/4361 actually "turned off" the Flake8 test. It would be good if we can fix this issue quickly to avoid any potential Flake8 errors in new PRs to be introduced into master branch. CC @holdenk who authored https://github.com/apache/incubator-airflow/pull/4361. Please let me know if I misunderstood or missed anything. Thanks! This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Commented] (AIRFLOW-3606) Fix Flake8 test & Fix the Flake8 errors introduced since Flake8 test was broken
[ https://issues.apache.org/jira/browse/AIRFLOW-3606?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16731523#comment-16731523 ] ASF GitHub Bot commented on AIRFLOW-3606: - XD-DENG commented on pull request #4415: [AIRFLOW-3606] Fix Flake8 test & fix the Flake8 errors introduced since Flake8 test was broken URL: https://github.com/apache/incubator-airflow/pull/4415 ### Jira - https://issues.apache.org/jira/browse/AIRFLOW-3606 ### Description - [x] Here are some details about my PR, including screenshots of any UI changes: 1. **The `flake8 test` in the Travis CI was broken since https://github.com/apache/incubator-airflow/pull/4361 (https://github.com/apache/incubator-airflow/commit/7a6acbf5b343e4a6895d1cc8af75ecc02b4fd0e8 )** 2. Some Flake8 errors (code style/quality issues. found in 10 files) were introduce since flake8 test was broken. In this PR, I reverted https://github.com/apache/incubator-airflow/pull/4361 and fixed all the Flake8 errors introduce into master branch since the `flake8 test` was broken (in the past 9 days). This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org > Fix Flake8 test & Fix the Flake8 errors introduced since Flake8 test was > broken > > > Key: AIRFLOW-3606 > URL: https://issues.apache.org/jira/browse/AIRFLOW-3606 > Project: Apache Airflow > Issue Type: Bug >Reporter: Xiaodong DENG >Assignee: Xiaodong DENG >Priority: Critical > > The flake8 test in the Travis CI was broken since > [https://github.com/apache/incubator-airflow/pull/4361] > ([https://github.com/apache/incubator-airflow/commit/7a6acbf5b343e4a6895d1cc8af75ecc02b4fd0e8] > ) > > And some Flake8 errors (code style/quality issues. found in 10 files) were > introduce since flake8 test was broken. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[GitHub] XD-DENG opened a new pull request #4415: [AIRFLOW-3606] Fix Flake8 test & fix the Flake8 errors introduced since Flake8 test was broken
XD-DENG opened a new pull request #4415: [AIRFLOW-3606] Fix Flake8 test & fix the Flake8 errors introduced since Flake8 test was broken URL: https://github.com/apache/incubator-airflow/pull/4415 ### Jira - https://issues.apache.org/jira/browse/AIRFLOW-3606 ### Description - [x] Here are some details about my PR, including screenshots of any UI changes: 1. **The `flake8 test` in the Travis CI was broken since https://github.com/apache/incubator-airflow/pull/4361 (https://github.com/apache/incubator-airflow/commit/7a6acbf5b343e4a6895d1cc8af75ecc02b4fd0e8 )** 2. Some Flake8 errors (code style/quality issues. found in 10 files) were introduce since flake8 test was broken. In this PR, I reverted https://github.com/apache/incubator-airflow/pull/4361 and fixed all the Flake8 errors introduce into master branch since the `flake8 test` was broken (in the past 9 days). This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Created] (AIRFLOW-3606) Fix Flake8 test & Fix the Flake8 errors introduced since Flake8 test was broken
Xiaodong DENG created AIRFLOW-3606: -- Summary: Fix Flake8 test & Fix the Flake8 errors introduced since Flake8 test was broken Key: AIRFLOW-3606 URL: https://issues.apache.org/jira/browse/AIRFLOW-3606 Project: Apache Airflow Issue Type: Bug Reporter: Xiaodong DENG Assignee: Xiaodong DENG The flake8 test in the Travis CI was broken since [https://github.com/apache/incubator-airflow/pull/4361] ([https://github.com/apache/incubator-airflow/commit/7a6acbf5b343e4a6895d1cc8af75ecc02b4fd0e8] ) And some Flake8 errors (code style/quality issues. found in 10 files) were introduce since flake8 test was broken. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[GitHub] codecov-io edited a comment on issue #4409: WIP [AIRFLOW-3601] add location support to BigQuery operators
codecov-io edited a comment on issue #4409: WIP [AIRFLOW-3601] add location support to BigQuery operators URL: https://github.com/apache/incubator-airflow/pull/4409#issuecomment-450601800 # [Codecov](https://codecov.io/gh/apache/incubator-airflow/pull/4409?src=pr=h1) Report > Merging [#4409](https://codecov.io/gh/apache/incubator-airflow/pull/4409?src=pr=desc) into [master](https://codecov.io/gh/apache/incubator-airflow/commit/b50ffa091dd4cdf1dabc8ed4ee701cac9d8318c6?src=pr=desc) will **increase** coverage by `0.13%`. > The diff coverage is `n/a`. [](https://codecov.io/gh/apache/incubator-airflow/pull/4409?src=pr=tree) ```diff @@Coverage Diff @@ ## master#4409 +/- ## == + Coverage 78.25% 78.38% +0.13% == Files 204 204 Lines 1643316445 +12 == + Hits1285912890 +31 + Misses 3574 3555 -19 ``` | [Impacted Files](https://codecov.io/gh/apache/incubator-airflow/pull/4409?src=pr=tree) | Coverage Δ | | |---|---|---| | [airflow/www/api/experimental/endpoints.py](https://codecov.io/gh/apache/incubator-airflow/pull/4409/diff?src=pr=tree#diff-YWlyZmxvdy93d3cvYXBpL2V4cGVyaW1lbnRhbC9lbmRwb2ludHMucHk=) | `71.12% <0%> (-19.02%)` | :arrow_down: | | [airflow/api/common/experimental/delete\_dag.py](https://codecov.io/gh/apache/incubator-airflow/pull/4409/diff?src=pr=tree#diff-YWlyZmxvdy9hcGkvY29tbW9uL2V4cGVyaW1lbnRhbC9kZWxldGVfZGFnLnB5) | `84% <0%> (-4%)` | :arrow_down: | | [airflow/models/\_\_init\_\_.py](https://codecov.io/gh/apache/incubator-airflow/pull/4409/diff?src=pr=tree#diff-YWlyZmxvdy9tb2RlbHMvX19pbml0X18ucHk=) | `92.6% <0%> (+0.07%)` | :arrow_up: | | [airflow/www/views.py](https://codecov.io/gh/apache/incubator-airflow/pull/4409/diff?src=pr=tree#diff-YWlyZmxvdy93d3cvdmlld3MucHk=) | `69.69% <0%> (+0.22%)` | :arrow_up: | | [airflow/www\_rbac/views.py](https://codecov.io/gh/apache/incubator-airflow/pull/4409/diff?src=pr=tree#diff-YWlyZmxvdy93d3dfcmJhYy92aWV3cy5weQ==) | `73.71% <0%> (+0.93%)` | :arrow_up: | | [airflow/hooks/samba\_hook.py](https://codecov.io/gh/apache/incubator-airflow/pull/4409/diff?src=pr=tree#diff-YWlyZmxvdy9ob29rcy9zYW1iYV9ob29rLnB5) | `38.88% <0%> (+38.88%)` | :arrow_up: | | [airflow/operators/hive\_to\_samba\_operator.py](https://codecov.io/gh/apache/incubator-airflow/pull/4409/diff?src=pr=tree#diff-YWlyZmxvdy9vcGVyYXRvcnMvaGl2ZV90b19zYW1iYV9vcGVyYXRvci5weQ==) | `100% <0%> (+100%)` | :arrow_up: | -- [Continue to review full report at Codecov](https://codecov.io/gh/apache/incubator-airflow/pull/4409?src=pr=continue). > **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta) > `Δ = absolute (impact)`, `ø = not affected`, `? = missing data` > Powered by [Codecov](https://codecov.io/gh/apache/incubator-airflow/pull/4409?src=pr=footer). Last update [b50ffa0...5b2280c](https://codecov.io/gh/apache/incubator-airflow/pull/4409?src=pr=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments). This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] codecov-io edited a comment on issue #4409: WIP [AIRFLOW-3601] add location support to BigQuery operators
codecov-io edited a comment on issue #4409: WIP [AIRFLOW-3601] add location support to BigQuery operators URL: https://github.com/apache/incubator-airflow/pull/4409#issuecomment-450601800 # [Codecov](https://codecov.io/gh/apache/incubator-airflow/pull/4409?src=pr=h1) Report > Merging [#4409](https://codecov.io/gh/apache/incubator-airflow/pull/4409?src=pr=desc) into [master](https://codecov.io/gh/apache/incubator-airflow/commit/b50ffa091dd4cdf1dabc8ed4ee701cac9d8318c6?src=pr=desc) will **increase** coverage by `0.13%`. > The diff coverage is `n/a`. [](https://codecov.io/gh/apache/incubator-airflow/pull/4409?src=pr=tree) ```diff @@Coverage Diff @@ ## master#4409 +/- ## == + Coverage 78.25% 78.38% +0.13% == Files 204 204 Lines 1643316445 +12 == + Hits1285912890 +31 + Misses 3574 3555 -19 ``` | [Impacted Files](https://codecov.io/gh/apache/incubator-airflow/pull/4409?src=pr=tree) | Coverage Δ | | |---|---|---| | [airflow/www/api/experimental/endpoints.py](https://codecov.io/gh/apache/incubator-airflow/pull/4409/diff?src=pr=tree#diff-YWlyZmxvdy93d3cvYXBpL2V4cGVyaW1lbnRhbC9lbmRwb2ludHMucHk=) | `71.12% <0%> (-19.02%)` | :arrow_down: | | [airflow/api/common/experimental/delete\_dag.py](https://codecov.io/gh/apache/incubator-airflow/pull/4409/diff?src=pr=tree#diff-YWlyZmxvdy9hcGkvY29tbW9uL2V4cGVyaW1lbnRhbC9kZWxldGVfZGFnLnB5) | `84% <0%> (-4%)` | :arrow_down: | | [airflow/models/\_\_init\_\_.py](https://codecov.io/gh/apache/incubator-airflow/pull/4409/diff?src=pr=tree#diff-YWlyZmxvdy9tb2RlbHMvX19pbml0X18ucHk=) | `92.6% <0%> (+0.07%)` | :arrow_up: | | [airflow/www/views.py](https://codecov.io/gh/apache/incubator-airflow/pull/4409/diff?src=pr=tree#diff-YWlyZmxvdy93d3cvdmlld3MucHk=) | `69.69% <0%> (+0.22%)` | :arrow_up: | | [airflow/www\_rbac/views.py](https://codecov.io/gh/apache/incubator-airflow/pull/4409/diff?src=pr=tree#diff-YWlyZmxvdy93d3dfcmJhYy92aWV3cy5weQ==) | `73.71% <0%> (+0.93%)` | :arrow_up: | | [airflow/hooks/samba\_hook.py](https://codecov.io/gh/apache/incubator-airflow/pull/4409/diff?src=pr=tree#diff-YWlyZmxvdy9ob29rcy9zYW1iYV9ob29rLnB5) | `38.88% <0%> (+38.88%)` | :arrow_up: | | [airflow/operators/hive\_to\_samba\_operator.py](https://codecov.io/gh/apache/incubator-airflow/pull/4409/diff?src=pr=tree#diff-YWlyZmxvdy9vcGVyYXRvcnMvaGl2ZV90b19zYW1iYV9vcGVyYXRvci5weQ==) | `100% <0%> (+100%)` | :arrow_up: | -- [Continue to review full report at Codecov](https://codecov.io/gh/apache/incubator-airflow/pull/4409?src=pr=continue). > **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta) > `Δ = absolute (impact)`, `ø = not affected`, `? = missing data` > Powered by [Codecov](https://codecov.io/gh/apache/incubator-airflow/pull/4409?src=pr=footer). Last update [b50ffa0...5b2280c](https://codecov.io/gh/apache/incubator-airflow/pull/4409?src=pr=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments). This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] codecov-io edited a comment on issue #4409: WIP [AIRFLOW-3601] add location support to BigQuery operators
codecov-io edited a comment on issue #4409: WIP [AIRFLOW-3601] add location support to BigQuery operators URL: https://github.com/apache/incubator-airflow/pull/4409#issuecomment-450601800 # [Codecov](https://codecov.io/gh/apache/incubator-airflow/pull/4409?src=pr=h1) Report > Merging [#4409](https://codecov.io/gh/apache/incubator-airflow/pull/4409?src=pr=desc) into [master](https://codecov.io/gh/apache/incubator-airflow/commit/b50ffa091dd4cdf1dabc8ed4ee701cac9d8318c6?src=pr=desc) will **decrease** coverage by `0.05%`. > The diff coverage is `n/a`. [](https://codecov.io/gh/apache/incubator-airflow/pull/4409?src=pr=tree) ```diff @@Coverage Diff @@ ## master#4409 +/- ## == - Coverage 78.25% 78.19% -0.06% == Files 204 204 Lines 1643316445 +12 == Hits1285912859 - Misses 3574 3586 +12 ``` | [Impacted Files](https://codecov.io/gh/apache/incubator-airflow/pull/4409?src=pr=tree) | Coverage Δ | | |---|---|---| | [airflow/executors/sequential\_executor.py](https://codecov.io/gh/apache/incubator-airflow/pull/4409/diff?src=pr=tree#diff-YWlyZmxvdy9leGVjdXRvcnMvc2VxdWVudGlhbF9leGVjdXRvci5weQ==) | `50% <0%> (-50%)` | :arrow_down: | | [airflow/www/api/experimental/endpoints.py](https://codecov.io/gh/apache/incubator-airflow/pull/4409/diff?src=pr=tree#diff-YWlyZmxvdy93d3cvYXBpL2V4cGVyaW1lbnRhbC9lbmRwb2ludHMucHk=) | `71.12% <0%> (-19.02%)` | :arrow_down: | | [airflow/utils/sqlalchemy.py](https://codecov.io/gh/apache/incubator-airflow/pull/4409/diff?src=pr=tree#diff-YWlyZmxvdy91dGlscy9zcWxhbGNoZW15LnB5) | `72.85% <0%> (-5.72%)` | :arrow_down: | | [airflow/api/common/experimental/delete\_dag.py](https://codecov.io/gh/apache/incubator-airflow/pull/4409/diff?src=pr=tree#diff-YWlyZmxvdy9hcGkvY29tbW9uL2V4cGVyaW1lbnRhbC9kZWxldGVfZGFnLnB5) | `84% <0%> (-4%)` | :arrow_down: | | [airflow/executors/\_\_init\_\_.py](https://codecov.io/gh/apache/incubator-airflow/pull/4409/diff?src=pr=tree#diff-YWlyZmxvdy9leGVjdXRvcnMvX19pbml0X18ucHk=) | `59.61% <0%> (-3.85%)` | :arrow_down: | | [airflow/utils/dag\_processing.py](https://codecov.io/gh/apache/incubator-airflow/pull/4409/diff?src=pr=tree#diff-YWlyZmxvdy91dGlscy9kYWdfcHJvY2Vzc2luZy5weQ==) | `58.6% <0%> (-1.08%)` | :arrow_down: | | [airflow/jobs.py](https://codecov.io/gh/apache/incubator-airflow/pull/4409/diff?src=pr=tree#diff-YWlyZmxvdy9qb2JzLnB5) | `76.71% <0%> (-0.74%)` | :arrow_down: | | [airflow/models/\_\_init\_\_.py](https://codecov.io/gh/apache/incubator-airflow/pull/4409/diff?src=pr=tree#diff-YWlyZmxvdy9tb2RlbHMvX19pbml0X18ucHk=) | `92.6% <0%> (+0.07%)` | :arrow_up: | | [airflow/www/views.py](https://codecov.io/gh/apache/incubator-airflow/pull/4409/diff?src=pr=tree#diff-YWlyZmxvdy93d3cvdmlld3MucHk=) | `69.69% <0%> (+0.22%)` | :arrow_up: | | [airflow/www\_rbac/views.py](https://codecov.io/gh/apache/incubator-airflow/pull/4409/diff?src=pr=tree#diff-YWlyZmxvdy93d3dfcmJhYy92aWV3cy5weQ==) | `73.71% <0%> (+0.93%)` | :arrow_up: | | ... and [2 more](https://codecov.io/gh/apache/incubator-airflow/pull/4409/diff?src=pr=tree-more) | | -- [Continue to review full report at Codecov](https://codecov.io/gh/apache/incubator-airflow/pull/4409?src=pr=continue). > **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta) > `Δ = absolute (impact)`, `ø = not affected`, `? = missing data` > Powered by [Codecov](https://codecov.io/gh/apache/incubator-airflow/pull/4409?src=pr=footer). Last update [b50ffa0...5b2280c](https://codecov.io/gh/apache/incubator-airflow/pull/4409?src=pr=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments). This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Commented] (AIRFLOW-1409) SparkSubmitHook doesn't generate a valid master URL
[ https://issues.apache.org/jira/browse/AIRFLOW-1409?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16731475#comment-16731475 ] ASF GitHub Bot commented on AIRFLOW-1409: - stale[bot] commented on pull request #2451: [AIRFLOW-1409] Parse ENV connections differently in SparkSubmitHook URL: https://github.com/apache/incubator-airflow/pull/2451 This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org > SparkSubmitHook doesn't generate a valid master URL > --- > > Key: AIRFLOW-1409 > URL: https://issues.apache.org/jira/browse/AIRFLOW-1409 > Project: Apache Airflow > Issue Type: Bug >Reporter: Luis Angel Vicente Sanchez >Priority: Major > > If you configure the connection using an ENV variable (and possibly through > the UI), SparkSubmitHook is ignoring the scheme and doesn't produce valid > master URL. Example, this ENV variable, > AIRFLOW_CONN_SPARK_CLUSTER=spark://spark-master:7077 > will produce the following master URL: > spark-master:7077 > This URL is not valid and the job submission fails. The issue is how the url > is parsed to create a connection, I have a partial fix working and I will > submit a PR to github soon. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[GitHub] stale[bot] closed pull request #2451: [AIRFLOW-1409] Parse ENV connections differently in SparkSubmitHook
stale[bot] closed pull request #2451: [AIRFLOW-1409] Parse ENV connections
differently in SparkSubmitHook
URL: https://github.com/apache/incubator-airflow/pull/2451
This is a PR merged from a forked repository.
As GitHub hides the original diff on merge, it is displayed below for
the sake of provenance:
As this is a foreign pull request (from a fork), the diff is supplied
below (as it won't show otherwise due to GitHub magic):
diff --git a/airflow/contrib/hooks/spark_submit_hook.py
b/airflow/contrib/hooks/spark_submit_hook.py
index a667753470..0494b8e0d2 100644
--- a/airflow/contrib/hooks/spark_submit_hook.py
+++ b/airflow/contrib/hooks/spark_submit_hook.py
@@ -14,15 +14,15 @@
#
import logging
import os
-import subprocess
+import random
import re
+import subprocess
from airflow.hooks.base_hook import BaseHook
from airflow.exceptions import AirflowException
log = logging.getLogger(__name__)
-
class SparkSubmitHook(BaseHook):
"""
This hook is a wrapper around the spark-submit binary to kick off a
spark-submit job.
@@ -103,6 +103,15 @@ def __init__(self,
self._connection = self._resolve_connection()
self._is_yarn = 'yarn' in self._connection['master']
+@staticmethod
+def _get_master_url(host, port = None, scheme = None):
+master_url = host
+if port:
+master_url = "{}:{}".format(master_url, port)
+if scheme:
+master_url = "{}://{}".format(scheme, master_url)
+return master_url
+
def _resolve_connection(self):
# Build from connection master or default to yarn if not available
conn_data = {'master': 'yarn',
@@ -112,12 +121,16 @@ def _resolve_connection(self):
'spark_binary': 'spark-submit'}
try:
-# Master can be local, yarn, spark://HOST:PORT or mesos://HOST:PORT
-conn = self.get_connection(self._conn_id)
-if conn.port:
-conn_data['master'] = "{}:{}".format(conn.host, conn.port)
+# If we use an ENV to define the connection, master can be //local,
+# //yarn, yarn://HOST:PORT, spark://HOST:PORT or mesos://HOST:PORT
+conn = self._get_connection_from_env(self._conn_id)
+if conn:
+conn_data['master'] = self._get_master_url(conn.host,
conn.port, conn.conn_type)
+# If we the connection comes frmo the database, master can be
local,
+# yarn, yarn://HOST:PORT, spark://HOST:PORT or mesos://HOST:PORT
else:
-conn_data['master'] = conn.host
+conn =
random.choice(self._get_connections_from_db(self._conn_id))
+conn_data['master'] = self._get_master_url(conn.host,
conn.port)
# Determine optional yarn queue from the extra field
extra = conn.extra_dejson
diff --git a/tests/contrib/hooks/test_spark_submit_hook.py
b/tests/contrib/hooks/test_spark_submit_hook.py
index 826576f0b8..57670473cf 100644
--- a/tests/contrib/hooks/test_spark_submit_hook.py
+++ b/tests/contrib/hooks/test_spark_submit_hook.py
@@ -12,7 +12,10 @@
# See the License for the specific language governing permissions and
# limitations under the License.
#
+import os
+import random
import six
+import string
import sys
import unittest
@@ -61,6 +64,10 @@ def cmd_args_to_dict(list_cmd):
return_dict[arg] = list_cmd[pos+1]
return return_dict
+@staticmethod
+def gen_conn_name(length):
+return ''.join(random.choice(string.ascii_lowercase + string.digits)
for _ in range(length))
+
def setUp(self):
configuration.load_test_config()
@@ -288,6 +295,106 @@ def
test_resolve_connection_spark_binary_and_home_set_connection(self):
self.assertEqual(connection, expected_spark_connection)
self.assertEqual(cmd[0], '/path/to/spark_home/bin/custom-spark-submit')
+def test_resolve_connection_yarn_env_connection(self):
+# Given
+conn_name = self.gen_conn_name(10)
+os.environ["AIRFLOW_CONN_SPARK_{}".format(conn_name.upper())] =
"//yarn"
+hook = SparkSubmitHook(conn_id='spark_{}'.format(conn_name))
+
+# When
+connection = hook._resolve_connection()
+cmd = hook._build_command(self._spark_job_file)
+
+# Then
+dict_cmd = self.cmd_args_to_dict(cmd)
+expected_spark_connection = {"master": "yarn",
+ "spark_binary": "spark-submit",
+ "deploy_mode": None,
+ "queue": None,
+ "spark_home": None}
+self.assertEqual(connection, expected_spark_connection)
+self.assertEqual(dict_cmd["--master"], "yarn")
+
+def test_resolve_connection_yarn_cluster_env_connection(self):
+# Given
+conn_name = self.gen_conn_name(10)
+
[GitHub] sarmadali20 opened a new pull request #4414: Support for new version of python jenkins
sarmadali20 opened a new pull request #4414: Support for new version of python
jenkins
URL: https://github.com/apache/incubator-airflow/pull/4414
Make sure you have checked _all_ steps below.
### Jira
- [ ] My PR addresses the following [Airflow
Jira](https://issues.apache.org/jira/browse/AIRFLOW/) issues and references
them in the PR title. For example, "\[AIRFLOW-XXX\] My Airflow PR"
- https://issues.apache.org/jira/browse/AIRFLOW-XXX
- In case you are fixing a typo in the documentation you can prepend your
commit with \[AIRFLOW-XXX\], code changes always need a Jira issue.
### Description
- [ ] Here are some details about my PR, including screenshots of any UI
changes:
### Tests
- [ ] My PR adds the following unit tests __OR__ does not need testing for
this extremely good reason:
### Commits
- [ ] My commits all reference Jira issues in their subject lines, and I
have squashed multiple commits if they address the same issue. In addition, my
commits follow the guidelines from "[How to write a good git commit
message](http://chris.beams.io/posts/git-commit/)":
1. Subject is separated from body by a blank line
1. Subject is limited to 50 characters (not including Jira issue reference)
1. Subject does not end with a period
1. Subject uses the imperative mood ("add", not "adding")
1. Body wraps at 72 characters
1. Body explains "what" and "why", not "how"
### Documentation
- [ ] In case of new functionality, my PR adds documentation that describes
how to use it.
- When adding new operators/hooks/sensors, the autoclass documentation
generation needs to be added.
- All the public functions and the classes in the PR contain docstrings
that explain what it does
### Code Quality
- [ ] Passes `flake8`
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] codecov-io edited a comment on issue #4331: [AIRFLOW-3531] Add gcs to gcs transfer operator.
codecov-io edited a comment on issue #4331: [AIRFLOW-3531] Add gcs to gcs transfer operator. URL: https://github.com/apache/incubator-airflow/pull/4331#issuecomment-447721611 # [Codecov](https://codecov.io/gh/apache/incubator-airflow/pull/4331?src=pr=h1) Report > Merging [#4331](https://codecov.io/gh/apache/incubator-airflow/pull/4331?src=pr=desc) into [master](https://codecov.io/gh/apache/incubator-airflow/commit/8938d2c727b16762529ef9f1e257ba5fa39f4bea?src=pr=desc) will **decrease** coverage by `<.01%`. > The diff coverage is `n/a`. [](https://codecov.io/gh/apache/incubator-airflow/pull/4331?src=pr=tree) ```diff @@Coverage Diff @@ ## master#4331 +/- ## == - Coverage 78.38% 78.37% -0.01% == Files 204 204 Lines 1644516445 == - Hits1289012889 -1 - Misses 3555 3556 +1 ``` | [Impacted Files](https://codecov.io/gh/apache/incubator-airflow/pull/4331?src=pr=tree) | Coverage Δ | | |---|---|---| | [airflow/models/\_\_init\_\_.py](https://codecov.io/gh/apache/incubator-airflow/pull/4331/diff?src=pr=tree#diff-YWlyZmxvdy9tb2RlbHMvX19pbml0X18ucHk=) | `92.55% <0%> (-0.05%)` | :arrow_down: | -- [Continue to review full report at Codecov](https://codecov.io/gh/apache/incubator-airflow/pull/4331?src=pr=continue). > **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta) > `Δ = absolute (impact)`, `ø = not affected`, `? = missing data` > Powered by [Codecov](https://codecov.io/gh/apache/incubator-airflow/pull/4331?src=pr=footer). Last update [8938d2c...97d883b](https://codecov.io/gh/apache/incubator-airflow/pull/4331?src=pr=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments). This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] codecov-io edited a comment on issue #4413: [AIRFLOW-XXX] Adds image code comment
codecov-io edited a comment on issue #4413: [AIRFLOW-XXX] Adds image code comment URL: https://github.com/apache/incubator-airflow/pull/4413#issuecomment-450678802 # [Codecov](https://codecov.io/gh/apache/incubator-airflow/pull/4413?src=pr=h1) Report > Merging [#4413](https://codecov.io/gh/apache/incubator-airflow/pull/4413?src=pr=desc) into [master](https://codecov.io/gh/apache/incubator-airflow/commit/8938d2c727b16762529ef9f1e257ba5fa39f4bea?src=pr=desc) will **increase** coverage by `0.01%`. > The diff coverage is `n/a`. [](https://codecov.io/gh/apache/incubator-airflow/pull/4413?src=pr=tree) ```diff @@Coverage Diff@@ ## master #4413 +/- ## = + Coverage 78.38% 78.4% +0.01% = Files 204 204 Lines 16445 16445 = + Hits12890 12893 +3 + Misses 35553552 -3 ``` | [Impacted Files](https://codecov.io/gh/apache/incubator-airflow/pull/4413?src=pr=tree) | Coverage Δ | | |---|---|---| | [airflow/jobs.py](https://codecov.io/gh/apache/incubator-airflow/pull/4413/diff?src=pr=tree#diff-YWlyZmxvdy9qb2JzLnB5) | `77.71% <0%> (+0.27%)` | :arrow_up: | -- [Continue to review full report at Codecov](https://codecov.io/gh/apache/incubator-airflow/pull/4413?src=pr=continue). > **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta) > `Δ = absolute (impact)`, `ø = not affected`, `? = missing data` > Powered by [Codecov](https://codecov.io/gh/apache/incubator-airflow/pull/4413?src=pr=footer). Last update [8938d2c...68c43c9](https://codecov.io/gh/apache/incubator-airflow/pull/4413?src=pr=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments). This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] codecov-io edited a comment on issue #4413: [AIRFLOW-XXX] Adds image code comment
codecov-io edited a comment on issue #4413: [AIRFLOW-XXX] Adds image code comment URL: https://github.com/apache/incubator-airflow/pull/4413#issuecomment-450678802 # [Codecov](https://codecov.io/gh/apache/incubator-airflow/pull/4413?src=pr=h1) Report > Merging [#4413](https://codecov.io/gh/apache/incubator-airflow/pull/4413?src=pr=desc) into [master](https://codecov.io/gh/apache/incubator-airflow/commit/8938d2c727b16762529ef9f1e257ba5fa39f4bea?src=pr=desc) will **increase** coverage by `0.01%`. > The diff coverage is `n/a`. [](https://codecov.io/gh/apache/incubator-airflow/pull/4413?src=pr=tree) ```diff @@Coverage Diff@@ ## master #4413 +/- ## = + Coverage 78.38% 78.4% +0.01% = Files 204 204 Lines 16445 16445 = + Hits12890 12893 +3 + Misses 35553552 -3 ``` | [Impacted Files](https://codecov.io/gh/apache/incubator-airflow/pull/4413?src=pr=tree) | Coverage Δ | | |---|---|---| | [airflow/jobs.py](https://codecov.io/gh/apache/incubator-airflow/pull/4413/diff?src=pr=tree#diff-YWlyZmxvdy9qb2JzLnB5) | `77.71% <0%> (+0.27%)` | :arrow_up: | -- [Continue to review full report at Codecov](https://codecov.io/gh/apache/incubator-airflow/pull/4413?src=pr=continue). > **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta) > `Δ = absolute (impact)`, `ø = not affected`, `? = missing data` > Powered by [Codecov](https://codecov.io/gh/apache/incubator-airflow/pull/4413?src=pr=footer). Last update [8938d2c...68c43c9](https://codecov.io/gh/apache/incubator-airflow/pull/4413?src=pr=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments). This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] codecov-io edited a comment on issue #4413: [AIRFLOW-XXX] Adds image code comment
codecov-io edited a comment on issue #4413: [AIRFLOW-XXX] Adds image code comment URL: https://github.com/apache/incubator-airflow/pull/4413#issuecomment-450678802 # [Codecov](https://codecov.io/gh/apache/incubator-airflow/pull/4413?src=pr=h1) Report > Merging [#4413](https://codecov.io/gh/apache/incubator-airflow/pull/4413?src=pr=desc) into [master](https://codecov.io/gh/apache/incubator-airflow/commit/8938d2c727b16762529ef9f1e257ba5fa39f4bea?src=pr=desc) will **increase** coverage by `0.01%`. > The diff coverage is `n/a`. [](https://codecov.io/gh/apache/incubator-airflow/pull/4413?src=pr=tree) ```diff @@Coverage Diff@@ ## master #4413 +/- ## = + Coverage 78.38% 78.4% +0.01% = Files 204 204 Lines 16445 16445 = + Hits12890 12893 +3 + Misses 35553552 -3 ``` | [Impacted Files](https://codecov.io/gh/apache/incubator-airflow/pull/4413?src=pr=tree) | Coverage Δ | | |---|---|---| | [airflow/jobs.py](https://codecov.io/gh/apache/incubator-airflow/pull/4413/diff?src=pr=tree#diff-YWlyZmxvdy9qb2JzLnB5) | `77.71% <0%> (+0.27%)` | :arrow_up: | -- [Continue to review full report at Codecov](https://codecov.io/gh/apache/incubator-airflow/pull/4413?src=pr=continue). > **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta) > `Δ = absolute (impact)`, `ø = not affected`, `? = missing data` > Powered by [Codecov](https://codecov.io/gh/apache/incubator-airflow/pull/4413?src=pr=footer). Last update [8938d2c...68c43c9](https://codecov.io/gh/apache/incubator-airflow/pull/4413?src=pr=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments). This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] seelmann commented on a change in pull request #4147: [AIRFLOW-3307] Upgrade rbac node deps via `npm audit fix`.
seelmann commented on a change in pull request #4147: [AIRFLOW-3307] Upgrade rbac node deps via `npm audit fix`. URL: https://github.com/apache/incubator-airflow/pull/4147#discussion_r244609571 ## File path: airflow/www_rbac/package.json ## @@ -57,11 +56,11 @@ "bootstrap-toggle": "^2.2.2", "d3": "^3.4.4", "d3-tip": "^0.9.1", -"dagre-d3": "^0.4.18", +"dagre-d3": "^0.6.1", Review comment: Unfortunately this breaks the www_rbac graph view, see JS error below. I cleaned `airflow/www_rbac/node_modules/` and run `airflow/www_rbac/compile_assets.sh`. We still use d3 version 3, so dagre-d3 version 0.4.18 is the last that can be used, see also https://github.com/dagrejs/dagre-d3/issues/309 If I revert back to dagre-d3 0.4.18 the graph view works. Should I create a PR to revert back? But I only have limited npm knowledge, not sure how to prevent the audit warnings. Or is there a plan to update d3? ``` dagre-d3.min.js:24 Uncaught TypeError: svgEdgeLabels.exit is not a function at createEdgeLabels (dagre-d3.min.js:24) at Array.fn (dagre-d3.min.js:90) at Array.Co.call (d3.min.js:3) at graph?dag_id=demo:1131 ``` This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] codecov-io edited a comment on issue #4413: [AIRFLOW-XXX] Adds image code comment
codecov-io edited a comment on issue #4413: [AIRFLOW-XXX] Adds image code comment URL: https://github.com/apache/incubator-airflow/pull/4413#issuecomment-450678802 # [Codecov](https://codecov.io/gh/apache/incubator-airflow/pull/4413?src=pr=h1) Report > Merging [#4413](https://codecov.io/gh/apache/incubator-airflow/pull/4413?src=pr=desc) into [master](https://codecov.io/gh/apache/incubator-airflow/commit/8938d2c727b16762529ef9f1e257ba5fa39f4bea?src=pr=desc) will **decrease** coverage by `<.01%`. > The diff coverage is `n/a`. [](https://codecov.io/gh/apache/incubator-airflow/pull/4413?src=pr=tree) ```diff @@Coverage Diff @@ ## master#4413 +/- ## == - Coverage 78.38% 78.37% -0.01% == Files 204 204 Lines 1644516445 == - Hits1289012889 -1 - Misses 3555 3556 +1 ``` | [Impacted Files](https://codecov.io/gh/apache/incubator-airflow/pull/4413?src=pr=tree) | Coverage Δ | | |---|---|---| | [airflow/models/\_\_init\_\_.py](https://codecov.io/gh/apache/incubator-airflow/pull/4413/diff?src=pr=tree#diff-YWlyZmxvdy9tb2RlbHMvX19pbml0X18ucHk=) | `92.55% <0%> (-0.05%)` | :arrow_down: | -- [Continue to review full report at Codecov](https://codecov.io/gh/apache/incubator-airflow/pull/4413?src=pr=continue). > **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta) > `Δ = absolute (impact)`, `ø = not affected`, `? = missing data` > Powered by [Codecov](https://codecov.io/gh/apache/incubator-airflow/pull/4413?src=pr=footer). Last update [8938d2c...68c43c9](https://codecov.io/gh/apache/incubator-airflow/pull/4413?src=pr=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments). This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] codecov-io commented on issue #4413: [AIRFLOW-XXX] Adds image code comment
codecov-io commented on issue #4413: [AIRFLOW-XXX] Adds image code comment URL: https://github.com/apache/incubator-airflow/pull/4413#issuecomment-450678802 # [Codecov](https://codecov.io/gh/apache/incubator-airflow/pull/4413?src=pr=h1) Report > Merging [#4413](https://codecov.io/gh/apache/incubator-airflow/pull/4413?src=pr=desc) into [master](https://codecov.io/gh/apache/incubator-airflow/commit/8938d2c727b16762529ef9f1e257ba5fa39f4bea?src=pr=desc) will **decrease** coverage by `<.01%`. > The diff coverage is `n/a`. [](https://codecov.io/gh/apache/incubator-airflow/pull/4413?src=pr=tree) ```diff @@Coverage Diff @@ ## master#4413 +/- ## == - Coverage 78.38% 78.37% -0.01% == Files 204 204 Lines 1644516445 == - Hits1289012889 -1 - Misses 3555 3556 +1 ``` | [Impacted Files](https://codecov.io/gh/apache/incubator-airflow/pull/4413?src=pr=tree) | Coverage Δ | | |---|---|---| | [airflow/models/\_\_init\_\_.py](https://codecov.io/gh/apache/incubator-airflow/pull/4413/diff?src=pr=tree#diff-YWlyZmxvdy9tb2RlbHMvX19pbml0X18ucHk=) | `92.55% <0%> (-0.05%)` | :arrow_down: | -- [Continue to review full report at Codecov](https://codecov.io/gh/apache/incubator-airflow/pull/4413?src=pr=continue). > **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta) > `Δ = absolute (impact)`, `ø = not affected`, `? = missing data` > Powered by [Codecov](https://codecov.io/gh/apache/incubator-airflow/pull/4413?src=pr=footer). Last update [8938d2c...68c43c9](https://codecov.io/gh/apache/incubator-airflow/pull/4413?src=pr=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments). This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] jmcarp commented on issue #4331: [AIRFLOW-3531] Add gcs to gcs transfer operator.
jmcarp commented on issue #4331: [AIRFLOW-3531] Add gcs to gcs transfer operator. URL: https://github.com/apache/incubator-airflow/pull/4331#issuecomment-450678345 Updated. This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] bwplotka commented on a change in pull request #4413: [AIRFLOW-XXX] Adds image code comment
bwplotka commented on a change in pull request #4413: [AIRFLOW-XXX] Adds image code comment URL: https://github.com/apache/incubator-airflow/pull/4413#discussion_r244606697 ## File path: scripts/ci/kubernetes/kube/templates/airflow.template.yaml ## @@ -15,6 +15,8 @@ # specific language governing permissions and limitations * # under the License. * +# Note: The airflow image used in this example is obtained by building the docker subdirectory Review comment: missing trailing period ;p This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] stale[bot] commented on issue #3850: [AIRFLOW-3010] added support for date format to postgres_to_gcs_operator
stale[bot] commented on issue #3850: [AIRFLOW-3010] added support for date format to postgres_to_gcs_operator URL: https://github.com/apache/incubator-airflow/pull/3850#issuecomment-450673761 This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions. This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] codecov-io edited a comment on issue #4409: [AIRFLOW-3601] add location support to BigQuery operators
codecov-io edited a comment on issue #4409: [AIRFLOW-3601] add location support to BigQuery operators URL: https://github.com/apache/incubator-airflow/pull/4409#issuecomment-450601800 # [Codecov](https://codecov.io/gh/apache/incubator-airflow/pull/4409?src=pr=h1) Report > Merging [#4409](https://codecov.io/gh/apache/incubator-airflow/pull/4409?src=pr=desc) into [master](https://codecov.io/gh/apache/incubator-airflow/commit/b50ffa091dd4cdf1dabc8ed4ee701cac9d8318c6?src=pr=desc) will **increase** coverage by `0.13%`. > The diff coverage is `n/a`. [](https://codecov.io/gh/apache/incubator-airflow/pull/4409?src=pr=tree) ```diff @@Coverage Diff @@ ## master#4409 +/- ## == + Coverage 78.25% 78.38% +0.13% == Files 204 204 Lines 1643316445 +12 == + Hits1285912890 +31 + Misses 3574 3555 -19 ``` | [Impacted Files](https://codecov.io/gh/apache/incubator-airflow/pull/4409?src=pr=tree) | Coverage Δ | | |---|---|---| | [airflow/www/api/experimental/endpoints.py](https://codecov.io/gh/apache/incubator-airflow/pull/4409/diff?src=pr=tree#diff-YWlyZmxvdy93d3cvYXBpL2V4cGVyaW1lbnRhbC9lbmRwb2ludHMucHk=) | `71.12% <0%> (-19.02%)` | :arrow_down: | | [airflow/api/common/experimental/delete\_dag.py](https://codecov.io/gh/apache/incubator-airflow/pull/4409/diff?src=pr=tree#diff-YWlyZmxvdy9hcGkvY29tbW9uL2V4cGVyaW1lbnRhbC9kZWxldGVfZGFnLnB5) | `84% <0%> (-4%)` | :arrow_down: | | [airflow/models/\_\_init\_\_.py](https://codecov.io/gh/apache/incubator-airflow/pull/4409/diff?src=pr=tree#diff-YWlyZmxvdy9tb2RlbHMvX19pbml0X18ucHk=) | `92.6% <0%> (+0.07%)` | :arrow_up: | | [airflow/www/views.py](https://codecov.io/gh/apache/incubator-airflow/pull/4409/diff?src=pr=tree#diff-YWlyZmxvdy93d3cvdmlld3MucHk=) | `69.69% <0%> (+0.22%)` | :arrow_up: | | [airflow/www\_rbac/views.py](https://codecov.io/gh/apache/incubator-airflow/pull/4409/diff?src=pr=tree#diff-YWlyZmxvdy93d3dfcmJhYy92aWV3cy5weQ==) | `73.71% <0%> (+0.93%)` | :arrow_up: | | [airflow/hooks/samba\_hook.py](https://codecov.io/gh/apache/incubator-airflow/pull/4409/diff?src=pr=tree#diff-YWlyZmxvdy9ob29rcy9zYW1iYV9ob29rLnB5) | `38.88% <0%> (+38.88%)` | :arrow_up: | | [airflow/operators/hive\_to\_samba\_operator.py](https://codecov.io/gh/apache/incubator-airflow/pull/4409/diff?src=pr=tree#diff-YWlyZmxvdy9vcGVyYXRvcnMvaGl2ZV90b19zYW1iYV9vcGVyYXRvci5weQ==) | `100% <0%> (+100%)` | :arrow_up: | -- [Continue to review full report at Codecov](https://codecov.io/gh/apache/incubator-airflow/pull/4409?src=pr=continue). > **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta) > `Δ = absolute (impact)`, `ø = not affected`, `? = missing data` > Powered by [Codecov](https://codecov.io/gh/apache/incubator-airflow/pull/4409?src=pr=footer). Last update [b50ffa0...e90995a](https://codecov.io/gh/apache/incubator-airflow/pull/4409?src=pr=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments). This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] codecov-io edited a comment on issue #4412: [AIRFLOW-3605] Load plugins from entry_points
codecov-io edited a comment on issue #4412: [AIRFLOW-3605] Load plugins from entry_points URL: https://github.com/apache/incubator-airflow/pull/4412#issuecomment-450662547 # [Codecov](https://codecov.io/gh/apache/incubator-airflow/pull/4412?src=pr=h1) Report > Merging [#4412](https://codecov.io/gh/apache/incubator-airflow/pull/4412?src=pr=desc) into [master](https://codecov.io/gh/apache/incubator-airflow/commit/8938d2c727b16762529ef9f1e257ba5fa39f4bea?src=pr=desc) will **decrease** coverage by `0.02%`. > The diff coverage is `66.66%`. [](https://codecov.io/gh/apache/incubator-airflow/pull/4412?src=pr=tree) ```diff @@Coverage Diff @@ ## master#4412 +/- ## == - Coverage 78.38% 78.36% -0.03% == Files 204 204 Lines 1644516462 +17 == + Hits1289012900 +10 - Misses 3555 3562 +7 ``` | [Impacted Files](https://codecov.io/gh/apache/incubator-airflow/pull/4412?src=pr=tree) | Coverage Δ | | |---|---|---| | [airflow/plugins\_manager.py](https://codecov.io/gh/apache/incubator-airflow/pull/4412/diff?src=pr=tree#diff-YWlyZmxvdy9wbHVnaW5zX21hbmFnZXIucHk=) | `87.5% <66.66%> (-5.61%)` | :arrow_down: | -- [Continue to review full report at Codecov](https://codecov.io/gh/apache/incubator-airflow/pull/4412?src=pr=continue). > **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta) > `Δ = absolute (impact)`, `ø = not affected`, `? = missing data` > Powered by [Codecov](https://codecov.io/gh/apache/incubator-airflow/pull/4412?src=pr=footer). Last update [8938d2c...7130eb2](https://codecov.io/gh/apache/incubator-airflow/pull/4412?src=pr=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments). This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] codecov-io edited a comment on issue #4412: [AIRFLOW-3605] Load plugins from entry_points
codecov-io edited a comment on issue #4412: [AIRFLOW-3605] Load plugins from entry_points URL: https://github.com/apache/incubator-airflow/pull/4412#issuecomment-450662547 # [Codecov](https://codecov.io/gh/apache/incubator-airflow/pull/4412?src=pr=h1) Report > Merging [#4412](https://codecov.io/gh/apache/incubator-airflow/pull/4412?src=pr=desc) into [master](https://codecov.io/gh/apache/incubator-airflow/commit/8938d2c727b16762529ef9f1e257ba5fa39f4bea?src=pr=desc) will **decrease** coverage by `0.02%`. > The diff coverage is `66.66%`. [](https://codecov.io/gh/apache/incubator-airflow/pull/4412?src=pr=tree) ```diff @@Coverage Diff @@ ## master#4412 +/- ## == - Coverage 78.38% 78.36% -0.03% == Files 204 204 Lines 1644516462 +17 == + Hits1289012900 +10 - Misses 3555 3562 +7 ``` | [Impacted Files](https://codecov.io/gh/apache/incubator-airflow/pull/4412?src=pr=tree) | Coverage Δ | | |---|---|---| | [airflow/plugins\_manager.py](https://codecov.io/gh/apache/incubator-airflow/pull/4412/diff?src=pr=tree#diff-YWlyZmxvdy9wbHVnaW5zX21hbmFnZXIucHk=) | `87.5% <66.66%> (-5.61%)` | :arrow_down: | -- [Continue to review full report at Codecov](https://codecov.io/gh/apache/incubator-airflow/pull/4412?src=pr=continue). > **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta) > `Δ = absolute (impact)`, `ø = not affected`, `? = missing data` > Powered by [Codecov](https://codecov.io/gh/apache/incubator-airflow/pull/4412?src=pr=footer). Last update [8938d2c...7130eb2](https://codecov.io/gh/apache/incubator-airflow/pull/4412?src=pr=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments). This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] codecov-io edited a comment on issue #4412: [AIRFLOW-3605] Load plugins from entry_points
codecov-io edited a comment on issue #4412: [AIRFLOW-3605] Load plugins from entry_points URL: https://github.com/apache/incubator-airflow/pull/4412#issuecomment-450662547 # [Codecov](https://codecov.io/gh/apache/incubator-airflow/pull/4412?src=pr=h1) Report > Merging [#4412](https://codecov.io/gh/apache/incubator-airflow/pull/4412?src=pr=desc) into [master](https://codecov.io/gh/apache/incubator-airflow/commit/8938d2c727b16762529ef9f1e257ba5fa39f4bea?src=pr=desc) will **decrease** coverage by `62.12%`. > The diff coverage is `66.66%`. [](https://codecov.io/gh/apache/incubator-airflow/pull/4412?src=pr=tree) ```diff @@ Coverage Diff @@ ## master#4412 +/- ## === - Coverage 78.38% 16.25% -62.13% === Files 204 204 Lines 1644516462 +17 === - Hits12890 2676-10214 - Misses 355513786+10231 ``` | [Impacted Files](https://codecov.io/gh/apache/incubator-airflow/pull/4412?src=pr=tree) | Coverage Δ | | |---|---|---| | [airflow/plugins\_manager.py](https://codecov.io/gh/apache/incubator-airflow/pull/4412/diff?src=pr=tree#diff-YWlyZmxvdy9wbHVnaW5zX21hbmFnZXIucHk=) | `86.53% <66.66%> (-6.57%)` | :arrow_down: | | [airflow/www\_rbac/validators.py](https://codecov.io/gh/apache/incubator-airflow/pull/4412/diff?src=pr=tree#diff-YWlyZmxvdy93d3dfcmJhYy92YWxpZGF0b3JzLnB5) | `0% <0%> (-100%)` | :arrow_down: | | [airflow/utils/log/es\_task\_handler.py](https://codecov.io/gh/apache/incubator-airflow/pull/4412/diff?src=pr=tree#diff-YWlyZmxvdy91dGlscy9sb2cvZXNfdGFza19oYW5kbGVyLnB5) | `0% <0%> (-100%)` | :arrow_down: | | [airflow/example\_dags/subdags/subdag.py](https://codecov.io/gh/apache/incubator-airflow/pull/4412/diff?src=pr=tree#diff-YWlyZmxvdy9leGFtcGxlX2RhZ3Mvc3ViZGFncy9zdWJkYWcucHk=) | `0% <0%> (-100%)` | :arrow_down: | | [airflow/www/validators.py](https://codecov.io/gh/apache/incubator-airflow/pull/4412/diff?src=pr=tree#diff-YWlyZmxvdy93d3cvdmFsaWRhdG9ycy5weQ==) | `0% <0%> (-100%)` | :arrow_down: | | [airflow/example\_dags/example\_docker\_operator.py](https://codecov.io/gh/apache/incubator-airflow/pull/4412/diff?src=pr=tree#diff-YWlyZmxvdy9leGFtcGxlX2RhZ3MvZXhhbXBsZV9kb2NrZXJfb3BlcmF0b3IucHk=) | `0% <0%> (-100%)` | :arrow_down: | | [airflow/example\_dags/example\_subdag\_operator.py](https://codecov.io/gh/apache/incubator-airflow/pull/4412/diff?src=pr=tree#diff-YWlyZmxvdy9leGFtcGxlX2RhZ3MvZXhhbXBsZV9zdWJkYWdfb3BlcmF0b3IucHk=) | `0% <0%> (-100%)` | :arrow_down: | | [airflow/api/common/experimental/get\_dag\_runs.py](https://codecov.io/gh/apache/incubator-airflow/pull/4412/diff?src=pr=tree#diff-YWlyZmxvdy9hcGkvY29tbW9uL2V4cGVyaW1lbnRhbC9nZXRfZGFnX3J1bnMucHk=) | `0% <0%> (-100%)` | :arrow_down: | | [airflow/www/forms.py](https://codecov.io/gh/apache/incubator-airflow/pull/4412/diff?src=pr=tree#diff-YWlyZmxvdy93d3cvZm9ybXMucHk=) | `0% <0%> (-100%)` | :arrow_down: | | [airflow/operators/postgres\_operator.py](https://codecov.io/gh/apache/incubator-airflow/pull/4412/diff?src=pr=tree#diff-YWlyZmxvdy9vcGVyYXRvcnMvcG9zdGdyZXNfb3BlcmF0b3IucHk=) | `0% <0%> (-100%)` | :arrow_down: | | ... and [166 more](https://codecov.io/gh/apache/incubator-airflow/pull/4412/diff?src=pr=tree-more) | | -- [Continue to review full report at Codecov](https://codecov.io/gh/apache/incubator-airflow/pull/4412?src=pr=continue). > **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta) > `Δ = absolute (impact)`, `ø = not affected`, `? = missing data` > Powered by [Codecov](https://codecov.io/gh/apache/incubator-airflow/pull/4412?src=pr=footer). Last update [8938d2c...7130eb2](https://codecov.io/gh/apache/incubator-airflow/pull/4412?src=pr=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments). This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] codecov-io commented on issue #4412: [AIRFLOW-3605] Load plugins from entry_points
codecov-io commented on issue #4412: [AIRFLOW-3605] Load plugins from entry_points URL: https://github.com/apache/incubator-airflow/pull/4412#issuecomment-450662547 # [Codecov](https://codecov.io/gh/apache/incubator-airflow/pull/4412?src=pr=h1) Report > Merging [#4412](https://codecov.io/gh/apache/incubator-airflow/pull/4412?src=pr=desc) into [master](https://codecov.io/gh/apache/incubator-airflow/commit/8938d2c727b16762529ef9f1e257ba5fa39f4bea?src=pr=desc) will **decrease** coverage by `62.12%`. > The diff coverage is `66.66%`. [](https://codecov.io/gh/apache/incubator-airflow/pull/4412?src=pr=tree) ```diff @@ Coverage Diff @@ ## master#4412 +/- ## === - Coverage 78.38% 16.25% -62.13% === Files 204 204 Lines 1644516462 +17 === - Hits12890 2676-10214 - Misses 355513786+10231 ``` | [Impacted Files](https://codecov.io/gh/apache/incubator-airflow/pull/4412?src=pr=tree) | Coverage Δ | | |---|---|---| | [airflow/plugins\_manager.py](https://codecov.io/gh/apache/incubator-airflow/pull/4412/diff?src=pr=tree#diff-YWlyZmxvdy9wbHVnaW5zX21hbmFnZXIucHk=) | `86.53% <66.66%> (-6.57%)` | :arrow_down: | | [airflow/www\_rbac/validators.py](https://codecov.io/gh/apache/incubator-airflow/pull/4412/diff?src=pr=tree#diff-YWlyZmxvdy93d3dfcmJhYy92YWxpZGF0b3JzLnB5) | `0% <0%> (-100%)` | :arrow_down: | | [airflow/utils/log/es\_task\_handler.py](https://codecov.io/gh/apache/incubator-airflow/pull/4412/diff?src=pr=tree#diff-YWlyZmxvdy91dGlscy9sb2cvZXNfdGFza19oYW5kbGVyLnB5) | `0% <0%> (-100%)` | :arrow_down: | | [airflow/example\_dags/subdags/subdag.py](https://codecov.io/gh/apache/incubator-airflow/pull/4412/diff?src=pr=tree#diff-YWlyZmxvdy9leGFtcGxlX2RhZ3Mvc3ViZGFncy9zdWJkYWcucHk=) | `0% <0%> (-100%)` | :arrow_down: | | [airflow/www/validators.py](https://codecov.io/gh/apache/incubator-airflow/pull/4412/diff?src=pr=tree#diff-YWlyZmxvdy93d3cvdmFsaWRhdG9ycy5weQ==) | `0% <0%> (-100%)` | :arrow_down: | | [airflow/example\_dags/example\_docker\_operator.py](https://codecov.io/gh/apache/incubator-airflow/pull/4412/diff?src=pr=tree#diff-YWlyZmxvdy9leGFtcGxlX2RhZ3MvZXhhbXBsZV9kb2NrZXJfb3BlcmF0b3IucHk=) | `0% <0%> (-100%)` | :arrow_down: | | [airflow/example\_dags/example\_subdag\_operator.py](https://codecov.io/gh/apache/incubator-airflow/pull/4412/diff?src=pr=tree#diff-YWlyZmxvdy9leGFtcGxlX2RhZ3MvZXhhbXBsZV9zdWJkYWdfb3BlcmF0b3IucHk=) | `0% <0%> (-100%)` | :arrow_down: | | [airflow/api/common/experimental/get\_dag\_runs.py](https://codecov.io/gh/apache/incubator-airflow/pull/4412/diff?src=pr=tree#diff-YWlyZmxvdy9hcGkvY29tbW9uL2V4cGVyaW1lbnRhbC9nZXRfZGFnX3J1bnMucHk=) | `0% <0%> (-100%)` | :arrow_down: | | [airflow/www/forms.py](https://codecov.io/gh/apache/incubator-airflow/pull/4412/diff?src=pr=tree#diff-YWlyZmxvdy93d3cvZm9ybXMucHk=) | `0% <0%> (-100%)` | :arrow_down: | | [airflow/operators/postgres\_operator.py](https://codecov.io/gh/apache/incubator-airflow/pull/4412/diff?src=pr=tree#diff-YWlyZmxvdy9vcGVyYXRvcnMvcG9zdGdyZXNfb3BlcmF0b3IucHk=) | `0% <0%> (-100%)` | :arrow_down: | | ... and [166 more](https://codecov.io/gh/apache/incubator-airflow/pull/4412/diff?src=pr=tree-more) | | -- [Continue to review full report at Codecov](https://codecov.io/gh/apache/incubator-airflow/pull/4412?src=pr=continue). > **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta) > `Δ = absolute (impact)`, `ø = not affected`, `? = missing data` > Powered by [Codecov](https://codecov.io/gh/apache/incubator-airflow/pull/4412?src=pr=footer). Last update [8938d2c...7130eb2](https://codecov.io/gh/apache/incubator-airflow/pull/4412?src=pr=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments). This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Created] (AIRFLOW-3605) Load Plugins via entry_points
Drew Sonne created AIRFLOW-3605:
---
Summary: Load Plugins via entry_points
Key: AIRFLOW-3605
URL: https://issues.apache.org/jira/browse/AIRFLOW-3605
Project: Apache Airflow
Issue Type: Bug
Components: plugins
Affects Versions: 1.10.1
Reporter: Drew Sonne
Assignee: Drew Sonne
Fix For: 1.10.2
Added logic to load AirflowPlugins from the
[entry_points|https://setuptools.readthedocs.io/en/latest/pkg_resources.html#id16].
Rather than moving files on the OS, this allows plugins to be installed and
distributed via {{pip}}.
Also added a callback to execute business logic from the plugin when the plugin
is loaded.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[GitHub] yohei1126 commented on a change in pull request #4324: [AIRFLOW-3327] Add support for location in BigQueryHook
yohei1126 commented on a change in pull request #4324: [AIRFLOW-3327] Add support for location in BigQueryHook URL: https://github.com/apache/incubator-airflow/pull/4324#discussion_r244590540 ## File path: tests/contrib/hooks/test_bigquery_hook.py ## @@ -729,5 +702,21 @@ def test_legacy_sql_override_propagates_properly(self, run_with_config): self.assertIs(args[0]['query']['useLegacySql'], False) +class TestBigQueryHookLocation(unittest.TestCase): +@mock.patch.object(hook.BigQueryBaseCursor, 'run_with_configuration') +def test_location_propagates_properly(self, run_with_config): +with mock.patch.object(hook.BigQueryHook, 'get_service'): +bq_hook = hook.BigQueryHook(location=None) +self.assertIsNone(bq_hook.location) + +bq_cursor = hook.BigQueryBaseCursor(mock.Mock(), +'test-project', +location=None) +self.assertIsNone(bq_cursor.location) +bq_cursor.run_query(sql='select 1', location='US') Review comment: Why do you need to specify `location` on `run_query` after initialization of `BigQueryHook`? This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] kaxil commented on a change in pull request #4324: [AIRFLOW-3327] Add support for location in BigQueryHook
kaxil commented on a change in pull request #4324: [AIRFLOW-3327] Add support for location in BigQueryHook URL: https://github.com/apache/incubator-airflow/pull/4324#discussion_r244590850 ## File path: tests/contrib/hooks/test_bigquery_hook.py ## @@ -729,5 +702,21 @@ def test_legacy_sql_override_propagates_properly(self, run_with_config): self.assertIs(args[0]['query']['useLegacySql'], False) +class TestBigQueryHookLocation(unittest.TestCase): +@mock.patch.object(hook.BigQueryBaseCursor, 'run_with_configuration') +def test_location_propagates_properly(self, run_with_config): +with mock.patch.object(hook.BigQueryHook, 'get_service'): +bq_hook = hook.BigQueryHook(location=None) +self.assertIsNone(bq_hook.location) + +bq_cursor = hook.BigQueryBaseCursor(mock.Mock(), +'test-project', +location=None) +self.assertIsNone(bq_cursor.location) +bq_cursor.run_query(sql='select 1', location='US') Review comment: It gives an option to override defaults This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] kaxil commented on a change in pull request #4324: [AIRFLOW-3327] Add support for location in BigQueryHook
kaxil commented on a change in pull request #4324: [AIRFLOW-3327] Add support for location in BigQueryHook URL: https://github.com/apache/incubator-airflow/pull/4324#discussion_r244590850 ## File path: tests/contrib/hooks/test_bigquery_hook.py ## @@ -729,5 +702,21 @@ def test_legacy_sql_override_propagates_properly(self, run_with_config): self.assertIs(args[0]['query']['useLegacySql'], False) +class TestBigQueryHookLocation(unittest.TestCase): +@mock.patch.object(hook.BigQueryBaseCursor, 'run_with_configuration') +def test_location_propagates_properly(self, run_with_config): +with mock.patch.object(hook.BigQueryHook, 'get_service'): +bq_hook = hook.BigQueryHook(location=None) +self.assertIsNone(bq_hook.location) + +bq_cursor = hook.BigQueryBaseCursor(mock.Mock(), +'test-project', +location=None) +self.assertIsNone(bq_cursor.location) +bq_cursor.run_query(sql='select 1', location='US') Review comment: It gives an option to override defaults This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] codecov-io edited a comment on issue #4302: [AIRFLOW-2641] Fix MySqlToHiveTransfer to handle MySQL DECIMAL correctly
codecov-io edited a comment on issue #4302: [AIRFLOW-2641] Fix MySqlToHiveTransfer to handle MySQL DECIMAL correctly URL: https://github.com/apache/incubator-airflow/pull/4302#issuecomment-450644788 # [Codecov](https://codecov.io/gh/apache/incubator-airflow/pull/4302?src=pr=h1) Report > Merging [#4302](https://codecov.io/gh/apache/incubator-airflow/pull/4302?src=pr=desc) into [master](https://codecov.io/gh/apache/incubator-airflow/commit/8938d2c727b16762529ef9f1e257ba5fa39f4bea?src=pr=desc) will **not change** coverage. > The diff coverage is `100%`. [](https://codecov.io/gh/apache/incubator-airflow/pull/4302?src=pr=tree) ```diff @@ Coverage Diff @@ ## master#4302 +/- ## === Coverage 78.38% 78.38% === Files 204 204 Lines 1644516445 === Hits1289012890 Misses 3555 3555 ``` | [Impacted Files](https://codecov.io/gh/apache/incubator-airflow/pull/4302?src=pr=tree) | Coverage Δ | | |---|---|---| | [airflow/operators/mysql\_to\_hive.py](https://codecov.io/gh/apache/incubator-airflow/pull/4302/diff?src=pr=tree#diff-YWlyZmxvdy9vcGVyYXRvcnMvbXlzcWxfdG9faGl2ZS5weQ==) | `100% <100%> (ø)` | :arrow_up: | -- [Continue to review full report at Codecov](https://codecov.io/gh/apache/incubator-airflow/pull/4302?src=pr=continue). > **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta) > `Δ = absolute (impact)`, `ø = not affected`, `? = missing data` > Powered by [Codecov](https://codecov.io/gh/apache/incubator-airflow/pull/4302?src=pr=footer). Last update [8938d2c...402dff9](https://codecov.io/gh/apache/incubator-airflow/pull/4302?src=pr=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments). This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] yohei1126 commented on a change in pull request #4324: [AIRFLOW-3327] Add support for location in BigQueryHook
yohei1126 commented on a change in pull request #4324: [AIRFLOW-3327] Add support for location in BigQueryHook URL: https://github.com/apache/incubator-airflow/pull/4324#discussion_r244590540 ## File path: tests/contrib/hooks/test_bigquery_hook.py ## @@ -729,5 +702,21 @@ def test_legacy_sql_override_propagates_properly(self, run_with_config): self.assertIs(args[0]['query']['useLegacySql'], False) +class TestBigQueryHookLocation(unittest.TestCase): +@mock.patch.object(hook.BigQueryBaseCursor, 'run_with_configuration') +def test_location_propagates_properly(self, run_with_config): +with mock.patch.object(hook.BigQueryHook, 'get_service'): +bq_hook = hook.BigQueryHook(location=None) +self.assertIsNone(bq_hook.location) + +bq_cursor = hook.BigQueryBaseCursor(mock.Mock(), +'test-project', +location=None) +self.assertIsNone(bq_cursor.location) +bq_cursor.run_query(sql='select 1', location='US') Review comment: Why do you need to specify `location` on `run_query` after initialization of `BigQueryHook`? This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] codecov-io commented on issue #4403: [AIRFLOW-XXX] - Fix missing type in docstring of MySqlToGoogleCloudStorageOperator
codecov-io commented on issue #4403: [AIRFLOW-XXX] - Fix missing type in docstring of MySqlToGoogleCloudStorageOperator URL: https://github.com/apache/incubator-airflow/pull/4403#issuecomment-450649560 # [Codecov](https://codecov.io/gh/apache/incubator-airflow/pull/4403?src=pr=h1) Report > Merging [#4403](https://codecov.io/gh/apache/incubator-airflow/pull/4403?src=pr=desc) into [master](https://codecov.io/gh/apache/incubator-airflow/commit/53e707428826d603f43aebb73e18fe077881c4b3?src=pr=desc) will **increase** coverage by `1.45%`. > The diff coverage is `n/a`. [](https://codecov.io/gh/apache/incubator-airflow/pull/4403?src=pr=tree) ```diff @@Coverage Diff @@ ## master#4403 +/- ## == + Coverage 78.14% 79.59% +1.45% == Files 204 204 Lines 1643317288 +855 == + Hits1284213761 +919 + Misses 3591 3527 -64 ``` | [Impacted Files](https://codecov.io/gh/apache/incubator-airflow/pull/4403?src=pr=tree) | Coverage Δ | | |---|---|---| | [airflow/api/common/experimental/delete\_dag.py](https://codecov.io/gh/apache/incubator-airflow/pull/4403/diff?src=pr=tree#diff-YWlyZmxvdy9hcGkvY29tbW9uL2V4cGVyaW1lbnRhbC9kZWxldGVfZGFnLnB5) | `88.57% <0%> (+0.57%)` | :arrow_up: | | [airflow/www/views.py](https://codecov.io/gh/apache/incubator-airflow/pull/4403/diff?src=pr=tree#diff-YWlyZmxvdy93d3cvdmlld3MucHk=) | `70.24% <0%> (+0.77%)` | :arrow_up: | | [airflow/www\_rbac/views.py](https://codecov.io/gh/apache/incubator-airflow/pull/4403/diff?src=pr=tree#diff-YWlyZmxvdy93d3dfcmJhYy92aWV3cy5weQ==) | `73.71% <0%> (+0.93%)` | :arrow_up: | | [airflow/models/\_\_init\_\_.py](https://codecov.io/gh/apache/incubator-airflow/pull/4403/diff?src=pr=tree#diff-YWlyZmxvdy9tb2RlbHMvX19pbml0X18ucHk=) | `94.58% <0%> (+2%)` | :arrow_up: | | [airflow/hooks/samba\_hook.py](https://codecov.io/gh/apache/incubator-airflow/pull/4403/diff?src=pr=tree#diff-YWlyZmxvdy9ob29rcy9zYW1iYV9ob29rLnB5) | `38.88% <0%> (+38.88%)` | :arrow_up: | | [airflow/operators/hive\_to\_samba\_operator.py](https://codecov.io/gh/apache/incubator-airflow/pull/4403/diff?src=pr=tree#diff-YWlyZmxvdy9vcGVyYXRvcnMvaGl2ZV90b19zYW1iYV9vcGVyYXRvci5weQ==) | `82.75% <0%> (+82.75%)` | :arrow_up: | | [airflow/operators/jdbc\_operator.py](https://codecov.io/gh/apache/incubator-airflow/pull/4403/diff?src=pr=tree#diff-YWlyZmxvdy9vcGVyYXRvcnMvamRiY19vcGVyYXRvci5weQ==) | `85.71% <0%> (+85.71%)` | :arrow_up: | -- [Continue to review full report at Codecov](https://codecov.io/gh/apache/incubator-airflow/pull/4403?src=pr=continue). > **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta) > `Δ = absolute (impact)`, `ø = not affected`, `? = missing data` > Powered by [Codecov](https://codecov.io/gh/apache/incubator-airflow/pull/4403?src=pr=footer). Last update [53e7074...6d01610](https://codecov.io/gh/apache/incubator-airflow/pull/4403?src=pr=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments). This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] OmerJog commented on issue #4302: [AIRFLOW-2641] Fix MySqlToHiveTransfer to handle MySQL DECIMAL correctly
OmerJog commented on issue #4302: [AIRFLOW-2641] Fix MySqlToHiveTransfer to handle MySQL DECIMAL correctly URL: https://github.com/apache/incubator-airflow/pull/4302#issuecomment-450649164 @kaxil done This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] yohei1126 commented on issue #4409: [AIRFLOW-3601] add location support to BigQuery operators
yohei1126 commented on issue #4409: [AIRFLOW-3601] add location support to BigQuery operators URL: https://github.com/apache/incubator-airflow/pull/4409#issuecomment-450648807 yes working on it This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] feluelle commented on issue #4400: [AIRFLOW-3595] Add tests for Hive2SambaOperator
feluelle commented on issue #4400: [AIRFLOW-3595] Add tests for Hive2SambaOperator URL: https://github.com/apache/incubator-airflow/pull/4400#issuecomment-450648555 @feng-tao I am wondering why the jira ticket is not automatically being closed after you have merged this PR. Did I do sth wrong? This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Updated] (AIRFLOW-3604) Azure Integration - Hdinsight-operators
[ https://issues.apache.org/jira/browse/AIRFLOW-3604?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alikemal Ocalan updated AIRFLOW-3604: - Description: I writo Hdinsight operators for creating cluster and submitting jobs to Azure Hdinsight on [my repo|https://github.com/alikemalocalan/airflow-hdinsight-operators] and i want to contribute official project. How i can add azure depends on airflow setup ? was: I writo Hdinsight operators for creating cluster and submitting jobs to Azure Hdinsight on [my repo|https://github.com/alikemalocalan/airflow-hdinsight-operators] and İ want to contribute official project. How i can add azure depends on airflow setup ? > Azure Integration - Hdinsight-operators > --- > > Key: AIRFLOW-3604 > URL: https://issues.apache.org/jira/browse/AIRFLOW-3604 > Project: Apache Airflow > Issue Type: Task > Components: operators >Reporter: Alikemal Ocalan >Assignee: Alikemal Ocalan >Priority: Minor > Labels: features > > I writo Hdinsight operators for creating cluster and submitting jobs to Azure > Hdinsight on [my > repo|https://github.com/alikemalocalan/airflow-hdinsight-operators] and i > want to contribute official project. > How i can add azure depends on airflow setup ? -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (AIRFLOW-3604) Azure Integration - Hdinsight-operators
[ https://issues.apache.org/jira/browse/AIRFLOW-3604?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alikemal Ocalan updated AIRFLOW-3604: - Description: I writo Hdinsight operators for creating cluster and submitting jobs to Azure Hdinsight on [my repo|https://github.com/alikemalocalan/airflow-hdinsight-operators] and İ want to contribute official project. How i can add azure depends on airflow setup ? was: I writo Hdinsight operators for creating cluster and submitting jobs to Azure Hdinsight on [my repo|https://github.com/alikemalocalan/airflow-hdinsight-operators]. How i can add azure depends on airflow setup ? > Azure Integration - Hdinsight-operators > --- > > Key: AIRFLOW-3604 > URL: https://issues.apache.org/jira/browse/AIRFLOW-3604 > Project: Apache Airflow > Issue Type: Task > Components: operators >Reporter: Alikemal Ocalan >Assignee: Alikemal Ocalan >Priority: Minor > Labels: features > > I writo Hdinsight operators for creating cluster and submitting jobs to Azure > Hdinsight on [my > repo|https://github.com/alikemalocalan/airflow-hdinsight-operators] and İ > want to contribute official project. > How i can add azure depends on airflow setup ? -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[GitHub] codecov-io edited a comment on issue #4411: [AIRFLOW-3603]QuboleOperator: remove SQLCommand from SparkCmd Doc
codecov-io edited a comment on issue #4411: [AIRFLOW-3603]QuboleOperator: remove SQLCommand from SparkCmd Doc URL: https://github.com/apache/incubator-airflow/pull/4411#issuecomment-450646638 # [Codecov](https://codecov.io/gh/apache/incubator-airflow/pull/4411?src=pr=h1) Report > Merging [#4411](https://codecov.io/gh/apache/incubator-airflow/pull/4411?src=pr=desc) into [master](https://codecov.io/gh/apache/incubator-airflow/commit/8938d2c727b16762529ef9f1e257ba5fa39f4bea?src=pr=desc) will **not change** coverage. > The diff coverage is `n/a`. [](https://codecov.io/gh/apache/incubator-airflow/pull/4411?src=pr=tree) ```diff @@ Coverage Diff @@ ## master#4411 +/- ## === Coverage 78.38% 78.38% === Files 204 204 Lines 1644516445 === Hits1289012890 Misses 3555 3555 ``` -- [Continue to review full report at Codecov](https://codecov.io/gh/apache/incubator-airflow/pull/4411?src=pr=continue). > **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta) > `Δ = absolute (impact)`, `ø = not affected`, `? = missing data` > Powered by [Codecov](https://codecov.io/gh/apache/incubator-airflow/pull/4411?src=pr=footer). Last update [8938d2c...6fcd789](https://codecov.io/gh/apache/incubator-airflow/pull/4411?src=pr=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments). This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] codecov-io commented on issue #4411: [AIRFLOW-3603]QuboleOperator: remove SQLCommand from SparkCmd Doc
codecov-io commented on issue #4411: [AIRFLOW-3603]QuboleOperator: remove SQLCommand from SparkCmd Doc URL: https://github.com/apache/incubator-airflow/pull/4411#issuecomment-450646638 # [Codecov](https://codecov.io/gh/apache/incubator-airflow/pull/4411?src=pr=h1) Report > Merging [#4411](https://codecov.io/gh/apache/incubator-airflow/pull/4411?src=pr=desc) into [master](https://codecov.io/gh/apache/incubator-airflow/commit/8938d2c727b16762529ef9f1e257ba5fa39f4bea?src=pr=desc) will **not change** coverage. > The diff coverage is `n/a`. [](https://codecov.io/gh/apache/incubator-airflow/pull/4411?src=pr=tree) ```diff @@ Coverage Diff @@ ## master#4411 +/- ## === Coverage 78.38% 78.38% === Files 204 204 Lines 1644516445 === Hits1289012890 Misses 3555 3555 ``` -- [Continue to review full report at Codecov](https://codecov.io/gh/apache/incubator-airflow/pull/4411?src=pr=continue). > **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta) > `Δ = absolute (impact)`, `ø = not affected`, `? = missing data` > Powered by [Codecov](https://codecov.io/gh/apache/incubator-airflow/pull/4411?src=pr=footer). Last update [8938d2c...6fcd789](https://codecov.io/gh/apache/incubator-airflow/pull/4411?src=pr=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments). This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Updated] (AIRFLOW-3604) Azure Integration - Hdinsight-operators
[ https://issues.apache.org/jira/browse/AIRFLOW-3604?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alikemal Ocalan updated AIRFLOW-3604: - Description: I writo Hdinsight operators for creating cluster and submitting jobs to Azure Hdinsight on [my repo|https://github.com/alikemalocalan/airflow-hdinsight-operators]. How i can add azure depends on airflow setup ? was:I writo Hdinsight operators for creating cluster and submitting jobs to Azure Hdinsight on [my repo|https://github.com/alikemalocalan/airflow-hdinsight-operators]. > Azure Integration - Hdinsight-operators > --- > > Key: AIRFLOW-3604 > URL: https://issues.apache.org/jira/browse/AIRFLOW-3604 > Project: Apache Airflow > Issue Type: Task > Components: operators >Reporter: Alikemal Ocalan >Assignee: Alikemal Ocalan >Priority: Minor > Labels: features > > I writo Hdinsight operators for creating cluster and submitting jobs to Azure > Hdinsight on [my > repo|https://github.com/alikemalocalan/airflow-hdinsight-operators]. > How i can add azure depends on airflow setup ? -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (AIRFLOW-3604) Azure Integration - Hdinsight-operators
Alikemal Ocalan created AIRFLOW-3604: Summary: Azure Integration - Hdinsight-operators Key: AIRFLOW-3604 URL: https://issues.apache.org/jira/browse/AIRFLOW-3604 Project: Apache Airflow Issue Type: Task Components: operators Reporter: Alikemal Ocalan Assignee: Alikemal Ocalan I writo Hdinsight operators for creating cluster and submitting jobs to Azure Hdinsight on [my repo|https://github.com/alikemalocalan/airflow-hdinsight-operators]. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[GitHub] kaxil commented on a change in pull request #4331: [AIRFLOW-3531] Add gcs to gcs transfer operator.
kaxil commented on a change in pull request #4331: [AIRFLOW-3531] Add gcs to gcs transfer operator. URL: https://github.com/apache/incubator-airflow/pull/4331#discussion_r244583971 ## File path: docs/integration.rst ## @@ -1184,6 +1184,7 @@ Storage Operators - :ref:`GoogleCloudStorageObjectCreateAclEntryOperator` : Creates a new ACL entry on the specified object. - :ref:`GoogleCloudStorageToBigQueryOperator` : Loads files from Google cloud storage into BigQuery. - :ref:`GoogleCloudStorageToGoogleCloudStorageOperator` : Copies objects from a bucket to another, with renaming if requested. +- :ref:`GoogleCloudStorageToGoogleCloudStorageTransferOperator` : Copies objects from a bucket to another, with renaming if requested using the GCP Storage Transfer Service. Review comment: It doesn't allow renaming, does it? This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] kaxil commented on a change in pull request #4331: [AIRFLOW-3531] Add gcs to gcs transfer operator.
kaxil commented on a change in pull request #4331: [AIRFLOW-3531] Add gcs to
gcs transfer operator.
URL: https://github.com/apache/incubator-airflow/pull/4331#discussion_r244583929
##
File path: airflow/contrib/operators/gcs_to_gcs_transfer_operator.py
##
@@ -0,0 +1,127 @@
+# -*- coding: utf-8 -*-
+#
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+
+from airflow.models import BaseOperator
+from airflow.contrib.hooks.gcp_transfer_hook import GCPTransferServiceHook
+from airflow.utils.decorators import apply_defaults
+
+
+class GoogleCloudStorageToGoogleCloudStorageTransferOperator(BaseOperator):
+"""
+Copies objects from a bucket to another using the GCP Storage Transfer
+Service.
+
+:param source_bucket: The source Google cloud storage bucket where the
+ object is. (templated)
+:type source_bucket: str
+:param destination_bucket: The destination Google cloud storage bucket
+where the object should be. (templated)
+:type destination_bucket: str
+:param project_id: The ID of the Google Cloud Platform Console project that
+owns the job
+:type project_id: str
+:param gcp_conn_id: The connection ID to use when connecting to Google
Cloud
+Storage.
+:type gcp_conn_id: str
+:param delegate_to: The account to impersonate, if any.
+For this to work, the service account making the request must have
+domain-wide delegation enabled.
+:type delegate_to: str
+:param description: Optional transfer service job description
+:type description: str
+:param schedule: Optional transfer service schedule; see
+
https://cloud.google.com/storage-transfer/docs/reference/rest/v1/transferJobs.
+If not set, run transfer job once as soon as the operator runs
+:type schedule: dict
+:param object_conditions: Transfer service object conditions; see
+
https://cloud.google.com/storage-transfer/docs/reference/rest/v1/TransferSpec#ObjectConditions
+:type object_conditions: dict
+:param transfer_options: Transfer service transfer options; see
+
https://cloud.google.com/storage-transfer/docs/reference/rest/v1/TransferSpec#TransferOptions
+:type transfer_options: dict
+:param wait: Wait for transfer to finish; defaults to `True`
+:type wait: bool
+
+**Example**:
+
+.. code-block:: python
+
+ gcs_to_gcs_transfer_op =
GoogleCloudStorageToGoogleCloudStorageTransferOperator(
+task_id='gcs_to_gcs_transfer_example',
+source_bucket='my-source-bucket',
+destination_bucket='my-destination-bucket',
+project_id='my-gcp-project',
+dag=my_dag)
+"""
+
+template_fields = ('source_bucket', 'destination_bucket', 'description',
'object_conditions')
+ui_color = '#e09411'
+
+@apply_defaults
+def __init__(self,
+ source_bucket,
+ destination_bucket,
+ project_id,
Review comment:
`project_id ` should be optional as we have it from the Connections
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] codecov-io edited a comment on issue #4396: [AIRFLOW-3585] - Adding edges to database
codecov-io edited a comment on issue #4396: [AIRFLOW-3585] - Adding edges to database URL: https://github.com/apache/incubator-airflow/pull/4396#issuecomment-450633941 # [Codecov](https://codecov.io/gh/apache/incubator-airflow/pull/4396?src=pr=h1) Report > Merging [#4396](https://codecov.io/gh/apache/incubator-airflow/pull/4396?src=pr=desc) into [master](https://codecov.io/gh/apache/incubator-airflow/commit/8938d2c727b16762529ef9f1e257ba5fa39f4bea?src=pr=desc) will **increase** coverage by `62.31%`. > The diff coverage is `87.14%`. [](https://codecov.io/gh/apache/incubator-airflow/pull/4396?src=pr=tree) ```diff @@ Coverage Diff @@ ## master#4396 +/- ## === + Coverage 16.21% 78.52% +62.31% === Files 204 205+1 Lines 1644516461 +16 === + Hits 266612926+10260 + Misses 13779 3535-10244 ``` | [Impacted Files](https://codecov.io/gh/apache/incubator-airflow/pull/4396?src=pr=tree) | Coverage Δ | | |---|---|---| | [airflow/models/dag\_edge.py](https://codecov.io/gh/apache/incubator-airflow/pull/4396/diff?src=pr=tree#diff-YWlyZmxvdy9tb2RlbHMvZGFnX2VkZ2UucHk=) | `100% <100%> (ø)` | | | [airflow/models/\_\_init\_\_.py](https://codecov.io/gh/apache/incubator-airflow/pull/4396/diff?src=pr=tree#diff-YWlyZmxvdy9tb2RlbHMvX19pbml0X18ucHk=) | `92.42% <81.25%> (+64.39%)` | :arrow_up: | | [airflow/www/views.py](https://codecov.io/gh/apache/incubator-airflow/pull/4396/diff?src=pr=tree#diff-YWlyZmxvdy93d3cvdmlld3MucHk=) | `70.05% <84.21%> (+70.05%)` | :arrow_up: | | [airflow/www\_rbac/views.py](https://codecov.io/gh/apache/incubator-airflow/pull/4396/diff?src=pr=tree#diff-YWlyZmxvdy93d3dfcmJhYy92aWV3cy5weQ==) | `72.7% <85%> (+72.7%)` | :arrow_up: | | [airflow/plugins\_manager.py](https://codecov.io/gh/apache/incubator-airflow/pull/4396/diff?src=pr=tree#diff-YWlyZmxvdy9wbHVnaW5zX21hbmFnZXIucHk=) | `93.1% <0%> (+1.14%)` | :arrow_up: | | [airflow/executors/dask\_executor.py](https://codecov.io/gh/apache/incubator-airflow/pull/4396/diff?src=pr=tree#diff-YWlyZmxvdy9leGVjdXRvcnMvZGFza19leGVjdXRvci5weQ==) | `2% <0%> (+2%)` | :arrow_up: | | [airflow/exceptions.py](https://codecov.io/gh/apache/incubator-airflow/pull/4396/diff?src=pr=tree#diff-YWlyZmxvdy9leGNlcHRpb25zLnB5) | `100% <0%> (+2.85%)` | :arrow_up: | | [airflow/utils/operator\_resources.py](https://codecov.io/gh/apache/incubator-airflow/pull/4396/diff?src=pr=tree#diff-YWlyZmxvdy91dGlscy9vcGVyYXRvcl9yZXNvdXJjZXMucHk=) | `86.95% <0%> (+4.34%)` | :arrow_up: | | [airflow/executors/\_\_init\_\_.py](https://codecov.io/gh/apache/incubator-airflow/pull/4396/diff?src=pr=tree#diff-YWlyZmxvdy9leGVjdXRvcnMvX19pbml0X18ucHk=) | `63.46% <0%> (+5.76%)` | :arrow_up: | | ... and [170 more](https://codecov.io/gh/apache/incubator-airflow/pull/4396/diff?src=pr=tree-more) | | -- [Continue to review full report at Codecov](https://codecov.io/gh/apache/incubator-airflow/pull/4396?src=pr=continue). > **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta) > `Δ = absolute (impact)`, `ø = not affected`, `? = missing data` > Powered by [Codecov](https://codecov.io/gh/apache/incubator-airflow/pull/4396?src=pr=footer). Last update [8938d2c...fdff0fb](https://codecov.io/gh/apache/incubator-airflow/pull/4396?src=pr=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments). This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] XD-DENG commented on issue #4373: [AIRFLOW-3569] Add "Trigger DAG" button in DAG page
XD-DENG commented on issue #4373: [AIRFLOW-3569] Add "Trigger DAG" button in DAG page URL: https://github.com/apache/incubator-airflow/pull/4373#issuecomment-450638155 Thanks @feng-tao . Fully understand your point/concern. However, I think from UI perspective it may still be worth adding it, in order to be consistent with the **`Link`** column in the homepage (screenshot below).  But I also agree the value added by this change is not that big. I would like to leave the decision to you & other committers. Thanks! And happy new year! This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Closed] (AIRFLOW-3600) Remove dagBag from trigger call
[ https://issues.apache.org/jira/browse/AIRFLOW-3600?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Fokko Driesprong closed AIRFLOW-3600. - Resolution: Fixed > Remove dagBag from trigger call > --- > > Key: AIRFLOW-3600 > URL: https://issues.apache.org/jira/browse/AIRFLOW-3600 > Project: Apache Airflow > Issue Type: Improvement > Components: webserver >Affects Versions: 1.10.1 >Reporter: Peter van 't Hof >Assignee: Peter van 't Hof >Priority: Minor > Fix For: 2.0.0 > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Resolved] (AIRFLOW-3599) Remove DagBag from delete dag
[ https://issues.apache.org/jira/browse/AIRFLOW-3599?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Fokko Driesprong resolved AIRFLOW-3599. --- Resolution: Fixed > Remove DagBag from delete dag > - > > Key: AIRFLOW-3599 > URL: https://issues.apache.org/jira/browse/AIRFLOW-3599 > Project: Apache Airflow > Issue Type: Improvement > Components: webserver >Affects Versions: 1.10.1 >Reporter: Peter van 't Hof >Assignee: Peter van 't Hof >Priority: Minor > Fix For: 2.0.0 > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[GitHub] msumit closed pull request #4410: [AIRFLOW-XXX] Reorder Airflow Users
msumit closed pull request #4410: [AIRFLOW-XXX] Reorder Airflow Users URL: https://github.com/apache/incubator-airflow/pull/4410 This is a PR merged from a forked repository. As GitHub hides the original diff on merge, it is displayed below for the sake of provenance: As this is a foreign pull request (from a fork), the diff is supplied below (as it won't show otherwise due to GitHub magic): diff --git a/README.md b/README.md index c58d8faf1a..0f7c36fdc7 100644 --- a/README.md +++ b/README.md @@ -107,6 +107,7 @@ Currently **officially** using Airflow: 1. [6play](https://www.6play.fr) [[@lemourA](https://github.com/lemoura), [@achaussende](https://github.com/achaussende), [@d-nguyen](https://github.com/d-nguyen), [@julien-gm](https://github.com/julien-gm)] 1. [8fit](https://8fit.com/) [[@nicor88](https://github.com/nicor88), [@frnzska](https://github.com/frnzska)] 1. [90 Seconds](https://90seconds.tv/) [[@aaronmak](https://github.com/aaronmak)] +1. [99](https://99taxis.com) [[@fbenevides](https://github.com/fbenevides), [@gustavoamigo](https://github.com/gustavoamigo) & [@mmmaia](https://github.com/mmmaia)] 1. [AdBOOST](https://www.adboost.sk) [[AdBOOST](https://github.com/AdBOOST)] 1. [Agari](https://github.com/agaridata) [[@r39132](https://github.com/r39132)] 1. [Airbnb](http://airbnb.io/) [[@mistercrunch](https://github.com/mistercrunch), [@artwr](https://github.com/artwr)] @@ -148,6 +149,7 @@ Currently **officially** using Airflow: 1. [Celect](http://www.celect.com) [[@superdosh](https://github.com/superdosh) & [@chadcelect](https://github.com/chadcelect)] 1. [Censys](https://censys.io) [[@zakird](https://github.com/zakird), [@dadrian](https://github.com/dadrian), & [@andrewsardone](https://github.com/andrewsardone)] 1. [Change.org](https://www.change.org) [[@change](https://github.com/change), [@vijaykramesh](https://github.com/vijaykramesh)] +1. [Chartboost](https://www.chartboost.com) [[@cgelman](https://github.com/cgelman) & [@dclubb](https://github.com/dclubb)] 1. [Checkr](https://checkr.com) [[@tongboh](https://github.com/tongboh)] 1. [Children's Hospital of Philadelphia Division of Genomic Diagnostics](http://www.chop.edu/centers-programs/division-genomic-diagnostics) [[@genomics-geek]](https://github.com/genomics-geek/) 1. [Cinimex DataLab](http://cinimex.ru) [[@kdubovikov](https://github.com/kdubovikov)] @@ -155,15 +157,14 @@ Currently **officially** using Airflow: 1. [Civey](https://civey.com/) [[@WesleyBatista](https://github.com/WesleyBatista)] 1. [Clairvoyant](https://clairvoyantsoft.com) [@shekharv](https://github.com/shekharv) 1. [Clover Health](https://www.cloverhealth.com) [[@gwax](https://github.com/gwax) & [@vansivallab](https://github.com/vansivallab)] -1. [Chartboost](https://www.chartboost.com) [[@cgelman](https://github.com/cgelman) & [@dclubb](https://github.com/dclubb)] 1. [Collectivehealth Inc.](https://www.collectivehealth.com) [@retornam](https://github.com/retornam) 1. [Compass](https://www.compass.com) [[@wdhorton](https://github.com/wdhorton)] 1. [ContaAzul](https://www.contaazul.com) [[@bern4rdelli](https://github.com/bern4rdelli), [@renanleme](https://github.com/renanleme) & [@sabino](https://github.com/sabino)] 1. [Cotap](https://github.com/cotap/) [[@maraca](https://github.com/maraca) & [@richardchew](https://github.com/richardchew)] 1. [Craig@Work](https://www.craigatwork.com) 1. [Credit Karma](https://www.creditkarma.com/) [[@preete-dixit-ck](https://github.com/preete-dixit-ck) & [@harish-gaggar-ck](https://github.com/harish-gaggar-ck) & [@greg-finley-ck](https://github.com/greg-finley-ck)] -1. [CreditCards.com](https://www.creditcards.com/)[[@vmAggies](https://github.com/vmAggies) & [@jay-wallaby](https://github.com/jay-wallaby)] 1. [Creditas](https://www.creditas.com.br) [[@dcassiano](https://github.com/dcassiano)] +1. [CreditCards.com](https://www.creditcards.com/)[[@vmAggies](https://github.com/vmAggies) & [@jay-wallaby](https://github.com/jay-wallaby)] 1. [Custom Ink](https://www.customink.com/) [[@david-dalisay](https://github.com/david-dalisay), [@dmartin11](https://github.com/dmartin11) & [@mpeteuil](https://github.com/mpeteuil)] 1. [Dailymotion](http://www.dailymotion.com/fr) [[@germaintanguy](https://github.com/germaintanguy) & [@hc](https://github.com/hc)] 1. [Danamica](https://www.danamica.dk) [[@testvinder](https://github.com/testvinder)] @@ -291,12 +292,13 @@ Currently **officially** using Airflow: 1. [Société générale](https://www.societegenerale.fr/) [[@medmrgh](https://github.com/medmrgh) & [@s83](https://github.com/s83)] 1. [Spotahome](https://www.spotahome.com/) [[@spotahome](https://github.com/spotahome)] 1. [Spotify](https://github.com/spotify) [[@znichols](https://github.com/znichols)] +1. [Square](https://squareup.com/) 1. [Stackspace](https://beta.stackspace.io/) 1. [Strava](https://strava.com) [[@strava](https://github.com/strava),
[jira] [Commented] (AIRFLOW-3600) Remove dagBag from trigger call
[ https://issues.apache.org/jira/browse/AIRFLOW-3600?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16731286#comment-16731286 ] ASF GitHub Bot commented on AIRFLOW-3600: - Fokko commented on pull request #4407: [AIRFLOW-3600] Remove dagbag from trigger URL: https://github.com/apache/incubator-airflow/pull/4407 This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org > Remove dagBag from trigger call > --- > > Key: AIRFLOW-3600 > URL: https://issues.apache.org/jira/browse/AIRFLOW-3600 > Project: Apache Airflow > Issue Type: Improvement > Components: webserver >Affects Versions: 1.10.1 >Reporter: Peter van 't Hof >Assignee: Peter van 't Hof >Priority: Minor > Fix For: 2.0.0 > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (AIRFLOW-3599) Remove DagBag from delete dag
[ https://issues.apache.org/jira/browse/AIRFLOW-3599?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16731284#comment-16731284 ] ASF GitHub Bot commented on AIRFLOW-3599: - Fokko commented on pull request #4406: [AIRFLOW-3599] Removed Dagbag from delete dag URL: https://github.com/apache/incubator-airflow/pull/4406 This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org > Remove DagBag from delete dag > - > > Key: AIRFLOW-3599 > URL: https://issues.apache.org/jira/browse/AIRFLOW-3599 > Project: Apache Airflow > Issue Type: Improvement > Components: webserver >Affects Versions: 1.10.1 >Reporter: Peter van 't Hof >Assignee: Peter van 't Hof >Priority: Minor > Fix For: 2.0.0 > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[GitHub] ffinfo commented on issue #4407: [AIRFLOW-3600] Remove dagbag from trigger
ffinfo commented on issue #4407: [AIRFLOW-3600] Remove dagbag from trigger URL: https://github.com/apache/incubator-airflow/pull/4407#issuecomment-450636764 @Fokko I did forgot the link in jira for this. This is part of AIRFLOW-3562 This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] Fokko closed pull request #4406: [AIRFLOW-3599] Removed Dagbag from delete dag
Fokko closed pull request #4406: [AIRFLOW-3599] Removed Dagbag from delete dag
URL: https://github.com/apache/incubator-airflow/pull/4406
This is a PR merged from a forked repository.
As GitHub hides the original diff on merge, it is displayed below for
the sake of provenance:
As this is a foreign pull request (from a fork), the diff is supplied
below (as it won't show otherwise due to GitHub magic):
diff --git a/airflow/api/common/experimental/delete_dag.py
b/airflow/api/common/experimental/delete_dag.py
index d04355f940..fc211c016a 100644
--- a/airflow/api/common/experimental/delete_dag.py
+++ b/airflow/api/common/experimental/delete_dag.py
@@ -17,6 +17,8 @@
# specific language governing permissions and limitations
# under the License.
+import os
+
from sqlalchemy import or_
from airflow import models, settings
@@ -39,10 +41,9 @@ def delete_dag(dag_id, keep_records_in_log=True):
if dag is None:
raise DagNotFound("Dag id {} not found".format(dag_id))
-dagbag = models.DagBag()
-if dag_id in dagbag.dags:
+if dag.fileloc and not os.path.exists(dag.fileloc):
raise DagFileExists("Dag id {} is still in DagBag. "
-"Remove the DAG file first.".format(dag_id))
+"Remove the DAG file first: {}".format(dag_id,
dag.fileloc))
count = 0
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] Fokko closed pull request #4407: [AIRFLOW-3600] Remove dagbag from trigger
Fokko closed pull request #4407: [AIRFLOW-3600] Remove dagbag from trigger
URL: https://github.com/apache/incubator-airflow/pull/4407
This is a PR merged from a forked repository.
As GitHub hides the original diff on merge, it is displayed below for
the sake of provenance:
As this is a foreign pull request (from a fork), the diff is supplied
below (as it won't show otherwise due to GitHub magic):
diff --git a/airflow/models/__init__.py b/airflow/models/__init__.py
index 5b24bde81c..50ac1e54fa 100755
--- a/airflow/models/__init__.py
+++ b/airflow/models/__init__.py
@@ -2992,6 +2992,44 @@ def get_default_view(self):
else:
return self.default_view
+def get_dag(self):
+return DagBag(dag_folder=self.fileloc).get_dag(self.dag_id)
+
+@provide_session
+def create_dagrun(self,
+ run_id,
+ state,
+ execution_date,
+ start_date=None,
+ external_trigger=False,
+ conf=None,
+ session=None):
+"""
+Creates a dag run from this dag including the tasks associated with
this dag.
+Returns the dag run.
+
+:param run_id: defines the the run id for this dag run
+:type run_id: str
+:param execution_date: the execution date of this dag run
+:type execution_date: datetime
+:param state: the state of the dag run
+:type state: State
+:param start_date: the date this dag run should be evaluated
+:type start_date: datetime
+:param external_trigger: whether this dag run is externally triggered
+:type external_trigger: bool
+:param session: database session
+:type session: Session
+"""
+
+return self.get_dag().create_dagrun(run_id=run_id,
+state=state,
+execution_date=execution_date,
+start_date=start_date,
+external_trigger=external_trigger,
+conf=conf,
+session=session)
+
@functools.total_ordering
class DAG(BaseDag, LoggingMixin):
diff --git a/airflow/www/views.py b/airflow/www/views.py
index 5bde8d3ee8..7746fb6de5 100644
--- a/airflow/www/views.py
+++ b/airflow/www/views.py
@@ -1069,11 +1069,11 @@ def delete(self):
@login_required
@wwwutils.action_logging
@wwwutils.notify_owner
-def trigger(self):
+@provide_session
+def trigger(self, session=None):
dag_id = request.args.get('dag_id')
origin = request.args.get('origin') or "/admin/"
-dag = dagbag.get_dag(dag_id)
-
+dag = session.query(models.DagModel).filter(models.DagModel.dag_id ==
dag_id).first()
if not dag:
flash("Cannot find dag {}".format(dag_id))
return redirect(origin)
@@ -1592,7 +1592,7 @@ class GraphForm(DateTimeWithNumRunsWithDagRunsForm):
task_instances=json.dumps(task_instances, indent=2),
tasks=json.dumps(tasks, indent=2),
nodes=json.dumps(nodes, indent=2),
-edges=json.dumps(edges, indent=2), )
+edges=json.dumps(edges, indent=2))
@expose('/duration')
@login_required
diff --git a/airflow/www_rbac/views.py b/airflow/www_rbac/views.py
index 5c4476a667..17efcdfe7c 100644
--- a/airflow/www_rbac/views.py
+++ b/airflow/www_rbac/views.py
@@ -798,11 +798,11 @@ def delete(self):
@has_dag_access(can_dag_edit=True)
@has_access
@action_logging
-def trigger(self):
+@provide_session
+def trigger(self, session=None):
dag_id = request.args.get('dag_id')
origin = request.args.get('origin') or "/"
-dag = dagbag.get_dag(dag_id)
-
+dag = session.query(models.DagModel).filter(models.DagModel.dag_id ==
dag_id).first()
if not dag:
flash("Cannot find dag {}".format(dag_id))
return redirect(origin)
diff --git a/tests/www/test_views.py b/tests/www/test_views.py
index a0d86ac54a..aeaca96333 100644
--- a/tests/www/test_views.py
+++ b/tests/www/test_views.py
@@ -30,6 +30,7 @@
from urllib.parse import quote_plus
from werkzeug.test import Client
+from sqlalchemy import func
from airflow import models, configuration
from airflow.config_templates.airflow_local_settings import
DEFAULT_LOGGING_CONFIG
@@ -821,5 +822,35 @@ def test_delete_dag_button_for_dag_on_scheduler_only(self):
session.commit()
+class TestTriggerDag(unittest.TestCase):
+
+def setUp(self):
+conf.load_test_config()
+app = application.create_app(testing=True)
+app.config['WTF_CSRF_METHODS'] = []
+self.app = app.test_client()
+self.session = Session()
+
[GitHub] codecov-io edited a comment on issue #4410: [AIRFLOW-XXX] Reorder Airflow Users
codecov-io edited a comment on issue #4410: [AIRFLOW-XXX] Reorder Airflow Users URL: https://github.com/apache/incubator-airflow/pull/4410#issuecomment-450636682 # [Codecov](https://codecov.io/gh/apache/incubator-airflow/pull/4410?src=pr=h1) Report > Merging [#4410](https://codecov.io/gh/apache/incubator-airflow/pull/4410?src=pr=desc) into [master](https://codecov.io/gh/apache/incubator-airflow/commit/ecc88a532658db249c3413f4e4c812b6800de725?src=pr=desc) will **not change** coverage. > The diff coverage is `n/a`. [](https://codecov.io/gh/apache/incubator-airflow/pull/4410?src=pr=tree) ```diff @@ Coverage Diff @@ ## master#4410 +/- ## === Coverage 78.44% 78.44% === Files 204 204 Lines 1643616436 === Hits1289412894 Misses 3542 3542 ``` -- [Continue to review full report at Codecov](https://codecov.io/gh/apache/incubator-airflow/pull/4410?src=pr=continue). > **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta) > `Δ = absolute (impact)`, `ø = not affected`, `? = missing data` > Powered by [Codecov](https://codecov.io/gh/apache/incubator-airflow/pull/4410?src=pr=footer). Last update [ecc88a5...b845643](https://codecov.io/gh/apache/incubator-airflow/pull/4410?src=pr=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments). This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] codecov-io commented on issue #4410: [AIRFLOW-XXX] Reorder Airflow Users
codecov-io commented on issue #4410: [AIRFLOW-XXX] Reorder Airflow Users URL: https://github.com/apache/incubator-airflow/pull/4410#issuecomment-450636682 # [Codecov](https://codecov.io/gh/apache/incubator-airflow/pull/4410?src=pr=h1) Report > Merging [#4410](https://codecov.io/gh/apache/incubator-airflow/pull/4410?src=pr=desc) into [master](https://codecov.io/gh/apache/incubator-airflow/commit/ecc88a532658db249c3413f4e4c812b6800de725?src=pr=desc) will **not change** coverage. > The diff coverage is `n/a`. [](https://codecov.io/gh/apache/incubator-airflow/pull/4410?src=pr=tree) ```diff @@ Coverage Diff @@ ## master#4410 +/- ## === Coverage 78.44% 78.44% === Files 204 204 Lines 1643616436 === Hits1289412894 Misses 3542 3542 ``` -- [Continue to review full report at Codecov](https://codecov.io/gh/apache/incubator-airflow/pull/4410?src=pr=continue). > **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta) > `Δ = absolute (impact)`, `ø = not affected`, `? = missing data` > Powered by [Codecov](https://codecov.io/gh/apache/incubator-airflow/pull/4410?src=pr=footer). Last update [ecc88a5...b845643](https://codecov.io/gh/apache/incubator-airflow/pull/4410?src=pr=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments). This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] codecov-io edited a comment on issue #4410: [AIRFLOW-XXX] Reorder Airflow Users
codecov-io edited a comment on issue #4410: [AIRFLOW-XXX] Reorder Airflow Users URL: https://github.com/apache/incubator-airflow/pull/4410#issuecomment-450636682 # [Codecov](https://codecov.io/gh/apache/incubator-airflow/pull/4410?src=pr=h1) Report > Merging [#4410](https://codecov.io/gh/apache/incubator-airflow/pull/4410?src=pr=desc) into [master](https://codecov.io/gh/apache/incubator-airflow/commit/ecc88a532658db249c3413f4e4c812b6800de725?src=pr=desc) will **not change** coverage. > The diff coverage is `n/a`. [](https://codecov.io/gh/apache/incubator-airflow/pull/4410?src=pr=tree) ```diff @@ Coverage Diff @@ ## master#4410 +/- ## === Coverage 78.44% 78.44% === Files 204 204 Lines 1643616436 === Hits1289412894 Misses 3542 3542 ``` -- [Continue to review full report at Codecov](https://codecov.io/gh/apache/incubator-airflow/pull/4410?src=pr=continue). > **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta) > `Δ = absolute (impact)`, `ø = not affected`, `? = missing data` > Powered by [Codecov](https://codecov.io/gh/apache/incubator-airflow/pull/4410?src=pr=footer). Last update [ecc88a5...b845643](https://codecov.io/gh/apache/incubator-airflow/pull/4410?src=pr=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments). This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] Fokko commented on issue #4407: [AIRFLOW-3600] Remove dagbag from trigger
Fokko commented on issue #4407: [AIRFLOW-3600] Remove dagbag from trigger URL: https://github.com/apache/incubator-airflow/pull/4407#issuecomment-450636519 @ffinfo can you explain the reasoning behind this PR? This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] codecov-io edited a comment on issue #4396: [WIP][AIRFLOW-3585] - Adding edges to database
codecov-io edited a comment on issue #4396: [WIP][AIRFLOW-3585] - Adding edges to database URL: https://github.com/apache/incubator-airflow/pull/4396#issuecomment-450633941 # [Codecov](https://codecov.io/gh/apache/incubator-airflow/pull/4396?src=pr=h1) Report > Merging [#4396](https://codecov.io/gh/apache/incubator-airflow/pull/4396?src=pr=desc) into [master](https://codecov.io/gh/apache/incubator-airflow/commit/53e707428826d603f43aebb73e18fe077881c4b3?src=pr=desc) will **increase** coverage by `0.37%`. > The diff coverage is `87.14%`. [](https://codecov.io/gh/apache/incubator-airflow/pull/4396?src=pr=tree) ```diff @@Coverage Diff @@ ## master#4396 +/- ## == + Coverage 78.14% 78.52% +0.37% == Files 204 205 +1 Lines 1643316461 +28 == + Hits1284212926 +84 + Misses 3591 3535 -56 ``` | [Impacted Files](https://codecov.io/gh/apache/incubator-airflow/pull/4396?src=pr=tree) | Coverage Δ | | |---|---|---| | [airflow/models/dag\_edge.py](https://codecov.io/gh/apache/incubator-airflow/pull/4396/diff?src=pr=tree#diff-YWlyZmxvdy9tb2RlbHMvZGFnX2VkZ2UucHk=) | `100% <100%> (ø)` | | | [airflow/models/\_\_init\_\_.py](https://codecov.io/gh/apache/incubator-airflow/pull/4396/diff?src=pr=tree#diff-YWlyZmxvdy9tb2RlbHMvX19pbml0X18ucHk=) | `92.42% <81.25%> (-0.15%)` | :arrow_down: | | [airflow/www/views.py](https://codecov.io/gh/apache/incubator-airflow/pull/4396/diff?src=pr=tree#diff-YWlyZmxvdy93d3cvdmlld3MucHk=) | `70.05% <84.21%> (+0.58%)` | :arrow_up: | | [airflow/www\_rbac/views.py](https://codecov.io/gh/apache/incubator-airflow/pull/4396/diff?src=pr=tree#diff-YWlyZmxvdy93d3dfcmJhYy92aWV3cy5weQ==) | `72.7% <85%> (-0.09%)` | :arrow_down: | | [airflow/www/utils.py](https://codecov.io/gh/apache/incubator-airflow/pull/4396/diff?src=pr=tree#diff-YWlyZmxvdy93d3cvdXRpbHMucHk=) | `90.86% <0%> (+0.53%)` | :arrow_up: | | [airflow/hooks/samba\_hook.py](https://codecov.io/gh/apache/incubator-airflow/pull/4396/diff?src=pr=tree#diff-YWlyZmxvdy9ob29rcy9zYW1iYV9ob29rLnB5) | `38.88% <0%> (+38.88%)` | :arrow_up: | | [airflow/operators/hive\_to\_samba\_operator.py](https://codecov.io/gh/apache/incubator-airflow/pull/4396/diff?src=pr=tree#diff-YWlyZmxvdy9vcGVyYXRvcnMvaGl2ZV90b19zYW1iYV9vcGVyYXRvci5weQ==) | `100% <0%> (+100%)` | :arrow_up: | | ... and [1 more](https://codecov.io/gh/apache/incubator-airflow/pull/4396/diff?src=pr=tree-more) | | -- [Continue to review full report at Codecov](https://codecov.io/gh/apache/incubator-airflow/pull/4396?src=pr=continue). > **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta) > `Δ = absolute (impact)`, `ø = not affected`, `? = missing data` > Powered by [Codecov](https://codecov.io/gh/apache/incubator-airflow/pull/4396?src=pr=footer). Last update [53e7074...11d64e2](https://codecov.io/gh/apache/incubator-airflow/pull/4396?src=pr=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments). This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] codecov-io commented on issue #4396: [WIP][AIRFLOW-3585] - Adding edges to database
codecov-io commented on issue #4396: [WIP][AIRFLOW-3585] - Adding edges to database URL: https://github.com/apache/incubator-airflow/pull/4396#issuecomment-450633941 # [Codecov](https://codecov.io/gh/apache/incubator-airflow/pull/4396?src=pr=h1) Report > Merging [#4396](https://codecov.io/gh/apache/incubator-airflow/pull/4396?src=pr=desc) into [master](https://codecov.io/gh/apache/incubator-airflow/commit/53e707428826d603f43aebb73e18fe077881c4b3?src=pr=desc) will **increase** coverage by `0.37%`. > The diff coverage is `87.14%`. [](https://codecov.io/gh/apache/incubator-airflow/pull/4396?src=pr=tree) ```diff @@Coverage Diff @@ ## master#4396 +/- ## == + Coverage 78.14% 78.52% +0.37% == Files 204 205 +1 Lines 1643316461 +28 == + Hits1284212926 +84 + Misses 3591 3535 -56 ``` | [Impacted Files](https://codecov.io/gh/apache/incubator-airflow/pull/4396?src=pr=tree) | Coverage Δ | | |---|---|---| | [airflow/models/dag\_edge.py](https://codecov.io/gh/apache/incubator-airflow/pull/4396/diff?src=pr=tree#diff-YWlyZmxvdy9tb2RlbHMvZGFnX2VkZ2UucHk=) | `100% <100%> (ø)` | | | [airflow/models/\_\_init\_\_.py](https://codecov.io/gh/apache/incubator-airflow/pull/4396/diff?src=pr=tree#diff-YWlyZmxvdy9tb2RlbHMvX19pbml0X18ucHk=) | `92.42% <81.25%> (-0.15%)` | :arrow_down: | | [airflow/www/views.py](https://codecov.io/gh/apache/incubator-airflow/pull/4396/diff?src=pr=tree#diff-YWlyZmxvdy93d3cvdmlld3MucHk=) | `70.05% <84.21%> (+0.58%)` | :arrow_up: | | [airflow/www\_rbac/views.py](https://codecov.io/gh/apache/incubator-airflow/pull/4396/diff?src=pr=tree#diff-YWlyZmxvdy93d3dfcmJhYy92aWV3cy5weQ==) | `72.7% <85%> (-0.09%)` | :arrow_down: | | [airflow/www/utils.py](https://codecov.io/gh/apache/incubator-airflow/pull/4396/diff?src=pr=tree#diff-YWlyZmxvdy93d3cvdXRpbHMucHk=) | `90.86% <0%> (+0.53%)` | :arrow_up: | | [airflow/hooks/samba\_hook.py](https://codecov.io/gh/apache/incubator-airflow/pull/4396/diff?src=pr=tree#diff-YWlyZmxvdy9ob29rcy9zYW1iYV9ob29rLnB5) | `38.88% <0%> (+38.88%)` | :arrow_up: | | [airflow/operators/hive\_to\_samba\_operator.py](https://codecov.io/gh/apache/incubator-airflow/pull/4396/diff?src=pr=tree#diff-YWlyZmxvdy9vcGVyYXRvcnMvaGl2ZV90b19zYW1iYV9vcGVyYXRvci5weQ==) | `100% <0%> (+100%)` | :arrow_up: | | ... and [1 more](https://codecov.io/gh/apache/incubator-airflow/pull/4396/diff?src=pr=tree-more) | | -- [Continue to review full report at Codecov](https://codecov.io/gh/apache/incubator-airflow/pull/4396?src=pr=continue). > **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta) > `Δ = absolute (impact)`, `ø = not affected`, `? = missing data` > Powered by [Codecov](https://codecov.io/gh/apache/incubator-airflow/pull/4396?src=pr=footer). Last update [53e7074...11d64e2](https://codecov.io/gh/apache/incubator-airflow/pull/4396?src=pr=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments). This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] feluelle opened a new pull request #4410: [AIRFLOW-XXX] Reorder Airflow Users
feluelle opened a new pull request #4410: [AIRFLOW-XXX] Reorder Airflow Users
URL: https://github.com/apache/incubator-airflow/pull/4410
Make sure you have checked _all_ steps below.
### Jira
- [x] My PR addresses the following [Airflow
Jira](https://issues.apache.org/jira/browse/AIRFLOW/) issues and references
them in the PR title. For example, "\[AIRFLOW-XXX\] My Airflow PR"
- https://issues.apache.org/jira/browse/AIRFLOW-XXX
- In case you are fixing a typo in the documentation you can prepend your
commit with \[AIRFLOW-XXX\], code changes always need a Jira issue.
### Description
- [x] Here are some details about my PR, including screenshots of any UI
changes:
### Tests
- [x] My PR adds the following unit tests __OR__ does not need testing for
this extremely good reason:
### Commits
- [x] My commits all reference Jira issues in their subject lines, and I
have squashed multiple commits if they address the same issue. In addition, my
commits follow the guidelines from "[How to write a good git commit
message](http://chris.beams.io/posts/git-commit/)":
1. Subject is separated from body by a blank line
1. Subject is limited to 50 characters (not including Jira issue reference)
1. Subject does not end with a period
1. Subject uses the imperative mood ("add", not "adding")
1. Body wraps at 72 characters
1. Body explains "what" and "why", not "how"
### Documentation
- [x] In case of new functionality, my PR adds documentation that describes
how to use it.
- When adding new operators/hooks/sensors, the autoclass documentation
generation needs to be added.
- All the public functions and the classes in the PR contain docstrings
that explain what it does
### Code Quality
- [x] Passes `flake8`
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] codecov-io edited a comment on issue #4406: [AIRFLOW-3599] Removed Dagbag from delete dag
codecov-io edited a comment on issue #4406: [AIRFLOW-3599] Removed Dagbag from delete dag URL: https://github.com/apache/incubator-airflow/pull/4406#issuecomment-450633307 # [Codecov](https://codecov.io/gh/apache/incubator-airflow/pull/4406?src=pr=h1) Report > Merging [#4406](https://codecov.io/gh/apache/incubator-airflow/pull/4406?src=pr=desc) into [master](https://codecov.io/gh/apache/incubator-airflow/commit/53e707428826d603f43aebb73e18fe077881c4b3?src=pr=desc) will **decrease** coverage by `0.43%`. > The diff coverage is `100%`. [](https://codecov.io/gh/apache/incubator-airflow/pull/4406?src=pr=tree) ```diff @@Coverage Diff @@ ## master#4406 +/- ## == - Coverage 78.14% 77.71% -0.44% == Files 204 204 Lines 1643316436 +3 == - Hits1284212773 -69 - Misses 3591 3663 +72 ``` | [Impacted Files](https://codecov.io/gh/apache/incubator-airflow/pull/4406?src=pr=tree) | Coverage Δ | | |---|---|---| | [airflow/api/common/experimental/delete\_dag.py](https://codecov.io/gh/apache/incubator-airflow/pull/4406/diff?src=pr=tree#diff-YWlyZmxvdy9hcGkvY29tbW9uL2V4cGVyaW1lbnRhbC9kZWxldGVfZGFnLnB5) | `84% <100%> (-4%)` | :arrow_down: | | [...rflow/api/common/experimental/get\_dag\_run\_state.py](https://codecov.io/gh/apache/incubator-airflow/pull/4406/diff?src=pr=tree#diff-YWlyZmxvdy9hcGkvY29tbW9uL2V4cGVyaW1lbnRhbC9nZXRfZGFnX3J1bl9zdGF0ZS5weQ==) | `23.07% <0%> (-76.93%)` | :arrow_down: | | [...rflow/api/common/experimental/get\_task\_instance.py](https://codecov.io/gh/apache/incubator-airflow/pull/4406/diff?src=pr=tree#diff-YWlyZmxvdy9hcGkvY29tbW9uL2V4cGVyaW1lbnRhbC9nZXRfdGFza19pbnN0YW5jZS5weQ==) | `15% <0%> (-75%)` | :arrow_down: | | [airflow/www\_rbac/api/experimental/endpoints.py](https://codecov.io/gh/apache/incubator-airflow/pull/4406/diff?src=pr=tree#diff-YWlyZmxvdy93d3dfcmJhYy9hcGkvZXhwZXJpbWVudGFsL2VuZHBvaW50cy5weQ==) | `64.94% <0%> (-27.02%)` | :arrow_down: | | [airflow/www/api/experimental/endpoints.py](https://codecov.io/gh/apache/incubator-airflow/pull/4406/diff?src=pr=tree#diff-YWlyZmxvdy93d3cvYXBpL2V4cGVyaW1lbnRhbC9lbmRwb2ludHMucHk=) | `71.12% <0%> (-19.02%)` | :arrow_down: | | [airflow/api/common/experimental/get\_dag\_runs.py](https://codecov.io/gh/apache/incubator-airflow/pull/4406/diff?src=pr=tree#diff-YWlyZmxvdy9hcGkvY29tbW9uL2V4cGVyaW1lbnRhbC9nZXRfZGFnX3J1bnMucHk=) | `92.3% <0%> (-7.7%)` | :arrow_down: | | [airflow/www\_rbac/views.py](https://codecov.io/gh/apache/incubator-airflow/pull/4406/diff?src=pr=tree#diff-YWlyZmxvdy93d3dfcmJhYy92aWV3cy5weQ==) | `72.19% <0%> (-0.6%)` | :arrow_down: | | [airflow/www/views.py](https://codecov.io/gh/apache/incubator-airflow/pull/4406/diff?src=pr=tree#diff-YWlyZmxvdy93d3cvdmlld3MucHk=) | `68.91% <0%> (-0.56%)` | :arrow_down: | | [airflow/models/\_\_init\_\_.py](https://codecov.io/gh/apache/incubator-airflow/pull/4406/diff?src=pr=tree#diff-YWlyZmxvdy9tb2RlbHMvX19pbml0X18ucHk=) | `92.45% <0%> (-0.13%)` | :arrow_down: | | [airflow/hooks/samba\_hook.py](https://codecov.io/gh/apache/incubator-airflow/pull/4406/diff?src=pr=tree#diff-YWlyZmxvdy9ob29rcy9zYW1iYV9ob29rLnB5) | `38.88% <0%> (+38.88%)` | :arrow_up: | | ... and [2 more](https://codecov.io/gh/apache/incubator-airflow/pull/4406/diff?src=pr=tree-more) | | -- [Continue to review full report at Codecov](https://codecov.io/gh/apache/incubator-airflow/pull/4406?src=pr=continue). > **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta) > `Δ = absolute (impact)`, `ø = not affected`, `? = missing data` > Powered by [Codecov](https://codecov.io/gh/apache/incubator-airflow/pull/4406?src=pr=footer). Last update [53e7074...3b19aaa](https://codecov.io/gh/apache/incubator-airflow/pull/4406?src=pr=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments). This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] codecov-io commented on issue #4406: [AIRFLOW-3599] Removed Dagbag from delete dag
codecov-io commented on issue #4406: [AIRFLOW-3599] Removed Dagbag from delete dag URL: https://github.com/apache/incubator-airflow/pull/4406#issuecomment-450633307 # [Codecov](https://codecov.io/gh/apache/incubator-airflow/pull/4406?src=pr=h1) Report > Merging [#4406](https://codecov.io/gh/apache/incubator-airflow/pull/4406?src=pr=desc) into [master](https://codecov.io/gh/apache/incubator-airflow/commit/53e707428826d603f43aebb73e18fe077881c4b3?src=pr=desc) will **decrease** coverage by `0.43%`. > The diff coverage is `100%`. [](https://codecov.io/gh/apache/incubator-airflow/pull/4406?src=pr=tree) ```diff @@Coverage Diff @@ ## master#4406 +/- ## == - Coverage 78.14% 77.71% -0.44% == Files 204 204 Lines 1643316436 +3 == - Hits1284212773 -69 - Misses 3591 3663 +72 ``` | [Impacted Files](https://codecov.io/gh/apache/incubator-airflow/pull/4406?src=pr=tree) | Coverage Δ | | |---|---|---| | [airflow/api/common/experimental/delete\_dag.py](https://codecov.io/gh/apache/incubator-airflow/pull/4406/diff?src=pr=tree#diff-YWlyZmxvdy9hcGkvY29tbW9uL2V4cGVyaW1lbnRhbC9kZWxldGVfZGFnLnB5) | `84% <100%> (-4%)` | :arrow_down: | | [...rflow/api/common/experimental/get\_dag\_run\_state.py](https://codecov.io/gh/apache/incubator-airflow/pull/4406/diff?src=pr=tree#diff-YWlyZmxvdy9hcGkvY29tbW9uL2V4cGVyaW1lbnRhbC9nZXRfZGFnX3J1bl9zdGF0ZS5weQ==) | `23.07% <0%> (-76.93%)` | :arrow_down: | | [...rflow/api/common/experimental/get\_task\_instance.py](https://codecov.io/gh/apache/incubator-airflow/pull/4406/diff?src=pr=tree#diff-YWlyZmxvdy9hcGkvY29tbW9uL2V4cGVyaW1lbnRhbC9nZXRfdGFza19pbnN0YW5jZS5weQ==) | `15% <0%> (-75%)` | :arrow_down: | | [airflow/www\_rbac/api/experimental/endpoints.py](https://codecov.io/gh/apache/incubator-airflow/pull/4406/diff?src=pr=tree#diff-YWlyZmxvdy93d3dfcmJhYy9hcGkvZXhwZXJpbWVudGFsL2VuZHBvaW50cy5weQ==) | `64.94% <0%> (-27.02%)` | :arrow_down: | | [airflow/www/api/experimental/endpoints.py](https://codecov.io/gh/apache/incubator-airflow/pull/4406/diff?src=pr=tree#diff-YWlyZmxvdy93d3cvYXBpL2V4cGVyaW1lbnRhbC9lbmRwb2ludHMucHk=) | `71.12% <0%> (-19.02%)` | :arrow_down: | | [airflow/api/common/experimental/get\_dag\_runs.py](https://codecov.io/gh/apache/incubator-airflow/pull/4406/diff?src=pr=tree#diff-YWlyZmxvdy9hcGkvY29tbW9uL2V4cGVyaW1lbnRhbC9nZXRfZGFnX3J1bnMucHk=) | `92.3% <0%> (-7.7%)` | :arrow_down: | | [airflow/www\_rbac/views.py](https://codecov.io/gh/apache/incubator-airflow/pull/4406/diff?src=pr=tree#diff-YWlyZmxvdy93d3dfcmJhYy92aWV3cy5weQ==) | `72.19% <0%> (-0.6%)` | :arrow_down: | | [airflow/www/views.py](https://codecov.io/gh/apache/incubator-airflow/pull/4406/diff?src=pr=tree#diff-YWlyZmxvdy93d3cvdmlld3MucHk=) | `68.91% <0%> (-0.56%)` | :arrow_down: | | [airflow/models/\_\_init\_\_.py](https://codecov.io/gh/apache/incubator-airflow/pull/4406/diff?src=pr=tree#diff-YWlyZmxvdy9tb2RlbHMvX19pbml0X18ucHk=) | `92.45% <0%> (-0.13%)` | :arrow_down: | | [airflow/hooks/samba\_hook.py](https://codecov.io/gh/apache/incubator-airflow/pull/4406/diff?src=pr=tree#diff-YWlyZmxvdy9ob29rcy9zYW1iYV9ob29rLnB5) | `38.88% <0%> (+38.88%)` | :arrow_up: | | ... and [2 more](https://codecov.io/gh/apache/incubator-airflow/pull/4406/diff?src=pr=tree-more) | | -- [Continue to review full report at Codecov](https://codecov.io/gh/apache/incubator-airflow/pull/4406?src=pr=continue). > **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta) > `Δ = absolute (impact)`, `ø = not affected`, `? = missing data` > Powered by [Codecov](https://codecov.io/gh/apache/incubator-airflow/pull/4406?src=pr=footer). Last update [53e7074...3b19aaa](https://codecov.io/gh/apache/incubator-airflow/pull/4406?src=pr=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments). This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] codecov-io edited a comment on issue #4407: [AIRFLOW-3600] Remove dagbag from trigger
codecov-io edited a comment on issue #4407: [AIRFLOW-3600] Remove dagbag from trigger URL: https://github.com/apache/incubator-airflow/pull/4407#issuecomment-450585018 # [Codecov](https://codecov.io/gh/apache/incubator-airflow/pull/4407?src=pr=h1) Report > Merging [#4407](https://codecov.io/gh/apache/incubator-airflow/pull/4407?src=pr=desc) into [master](https://codecov.io/gh/apache/incubator-airflow/commit/53e707428826d603f43aebb73e18fe077881c4b3?src=pr=desc) will **increase** coverage by `0.45%`. > The diff coverage is `100%`. [](https://codecov.io/gh/apache/incubator-airflow/pull/4407?src=pr=tree) ```diff @@Coverage Diff@@ ## master #4407 +/- ## = + Coverage 78.14% 78.6% +0.45% = Files 204 204 Lines 16433 16445 +12 = + Hits12842 12926 +84 + Misses 35913519 -72 ``` | [Impacted Files](https://codecov.io/gh/apache/incubator-airflow/pull/4407?src=pr=tree) | Coverage Δ | | |---|---|---| | [airflow/www/views.py](https://codecov.io/gh/apache/incubator-airflow/pull/4407/diff?src=pr=tree#diff-YWlyZmxvdy93d3cvdmlld3MucHk=) | `70.24% <100%> (+0.77%)` | :arrow_up: | | [airflow/www\_rbac/views.py](https://codecov.io/gh/apache/incubator-airflow/pull/4407/diff?src=pr=tree#diff-YWlyZmxvdy93d3dfcmJhYy92aWV3cy5weQ==) | `73.71% <100%> (+0.93%)` | :arrow_up: | | [airflow/models/\_\_init\_\_.py](https://codecov.io/gh/apache/incubator-airflow/pull/4407/diff?src=pr=tree#diff-YWlyZmxvdy9tb2RlbHMvX19pbml0X18ucHk=) | `92.55% <100%> (-0.02%)` | :arrow_down: | | [airflow/hooks/samba\_hook.py](https://codecov.io/gh/apache/incubator-airflow/pull/4407/diff?src=pr=tree#diff-YWlyZmxvdy9ob29rcy9zYW1iYV9ob29rLnB5) | `38.88% <0%> (+38.88%)` | :arrow_up: | | [airflow/operators/hive\_to\_samba\_operator.py](https://codecov.io/gh/apache/incubator-airflow/pull/4407/diff?src=pr=tree#diff-YWlyZmxvdy9vcGVyYXRvcnMvaGl2ZV90b19zYW1iYV9vcGVyYXRvci5weQ==) | `100% <0%> (+100%)` | :arrow_up: | | [airflow/operators/jdbc\_operator.py](https://codecov.io/gh/apache/incubator-airflow/pull/4407/diff?src=pr=tree#diff-YWlyZmxvdy9vcGVyYXRvcnMvamRiY19vcGVyYXRvci5weQ==) | `100% <0%> (+100%)` | :arrow_up: | -- [Continue to review full report at Codecov](https://codecov.io/gh/apache/incubator-airflow/pull/4407?src=pr=continue). > **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta) > `Δ = absolute (impact)`, `ø = not affected`, `? = missing data` > Powered by [Codecov](https://codecov.io/gh/apache/incubator-airflow/pull/4407?src=pr=footer). Last update [53e7074...0957961](https://codecov.io/gh/apache/incubator-airflow/pull/4407?src=pr=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments). This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Commented] (AIRFLOW-3235) Add list function in AzureDataLakeHook
[ https://issues.apache.org/jira/browse/AIRFLOW-3235?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16731260#comment-16731260 ] jack commented on AIRFLOW-3235: --- This was merged. ticket can be closed. > Add list function in AzureDataLakeHook > -- > > Key: AIRFLOW-3235 > URL: https://issues.apache.org/jira/browse/AIRFLOW-3235 > Project: Apache Airflow > Issue Type: Improvement > Components: hooks >Reporter: Brandon Kvarda >Assignee: Brandon Kvarda >Priority: Minor > > AzureDataLakeHook could use a list function similar to those of both the GCS > and S3 hooks. This could be the basis for an ADLS list operator (again, which > both GCS and S3 have). -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (AIRFLOW-55) Add HDFS Log Support
[ https://issues.apache.org/jira/browse/AIRFLOW-55?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16731262#comment-16731262 ] jack commented on AIRFLOW-55: - Resolved by https://github.com/apache/incubator-airflow/pull/2119 > Add HDFS Log Support > > > Key: AIRFLOW-55 > URL: https://issues.apache.org/jira/browse/AIRFLOW-55 > Project: Apache Airflow > Issue Type: New Feature > Components: hooks >Affects Versions: 1.7.0 >Reporter: Wu Xiang >Assignee: Daniel Huang >Priority: Major > Labels: features > Fix For: 1.8.0 > > > To support save task logs on HDFS. > PR: > https://github.com/apache/incubator-airflow/pull/1409 -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[GitHub] codecov-io edited a comment on issue #4407: [AIRFLOW-3600] Remove dagbag from trigger
codecov-io edited a comment on issue #4407: [AIRFLOW-3600] Remove dagbag from trigger URL: https://github.com/apache/incubator-airflow/pull/4407#issuecomment-450585018 # [Codecov](https://codecov.io/gh/apache/incubator-airflow/pull/4407?src=pr=h1) Report > Merging [#4407](https://codecov.io/gh/apache/incubator-airflow/pull/4407?src=pr=desc) into [master](https://codecov.io/gh/apache/incubator-airflow/commit/53e707428826d603f43aebb73e18fe077881c4b3?src=pr=desc) will **increase** coverage by `0.45%`. > The diff coverage is `100%`. [](https://codecov.io/gh/apache/incubator-airflow/pull/4407?src=pr=tree) ```diff @@Coverage Diff@@ ## master #4407 +/- ## = + Coverage 78.14% 78.6% +0.45% = Files 204 204 Lines 16433 16445 +12 = + Hits12842 12927 +85 + Misses 35913518 -73 ``` | [Impacted Files](https://codecov.io/gh/apache/incubator-airflow/pull/4407?src=pr=tree) | Coverage Δ | | |---|---|---| | [airflow/www/views.py](https://codecov.io/gh/apache/incubator-airflow/pull/4407/diff?src=pr=tree#diff-YWlyZmxvdy93d3cvdmlld3MucHk=) | `70.24% <100%> (+0.77%)` | :arrow_up: | | [airflow/www\_rbac/views.py](https://codecov.io/gh/apache/incubator-airflow/pull/4407/diff?src=pr=tree#diff-YWlyZmxvdy93d3dfcmJhYy92aWV3cy5weQ==) | `73.71% <100%> (+0.93%)` | :arrow_up: | | [airflow/models/\_\_init\_\_.py](https://codecov.io/gh/apache/incubator-airflow/pull/4407/diff?src=pr=tree#diff-YWlyZmxvdy9tb2RlbHMvX19pbml0X18ucHk=) | `92.6% <100%> (+0.02%)` | :arrow_up: | | [airflow/hooks/samba\_hook.py](https://codecov.io/gh/apache/incubator-airflow/pull/4407/diff?src=pr=tree#diff-YWlyZmxvdy9ob29rcy9zYW1iYV9ob29rLnB5) | `38.88% <0%> (+38.88%)` | :arrow_up: | | [airflow/operators/hive\_to\_samba\_operator.py](https://codecov.io/gh/apache/incubator-airflow/pull/4407/diff?src=pr=tree#diff-YWlyZmxvdy9vcGVyYXRvcnMvaGl2ZV90b19zYW1iYV9vcGVyYXRvci5weQ==) | `100% <0%> (+100%)` | :arrow_up: | | [airflow/operators/jdbc\_operator.py](https://codecov.io/gh/apache/incubator-airflow/pull/4407/diff?src=pr=tree#diff-YWlyZmxvdy9vcGVyYXRvcnMvamRiY19vcGVyYXRvci5weQ==) | `100% <0%> (+100%)` | :arrow_up: | -- [Continue to review full report at Codecov](https://codecov.io/gh/apache/incubator-airflow/pull/4407?src=pr=continue). > **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta) > `Δ = absolute (impact)`, `ø = not affected`, `? = missing data` > Powered by [Codecov](https://codecov.io/gh/apache/incubator-airflow/pull/4407?src=pr=footer). Last update [53e7074...0957961](https://codecov.io/gh/apache/incubator-airflow/pull/4407?src=pr=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments). This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] codecov-io edited a comment on issue #4407: [AIRFLOW-3600] Remove dagbag from trigger
codecov-io edited a comment on issue #4407: [AIRFLOW-3600] Remove dagbag from trigger URL: https://github.com/apache/incubator-airflow/pull/4407#issuecomment-450585018 # [Codecov](https://codecov.io/gh/apache/incubator-airflow/pull/4407?src=pr=h1) Report > Merging [#4407](https://codecov.io/gh/apache/incubator-airflow/pull/4407?src=pr=desc) into [master](https://codecov.io/gh/apache/incubator-airflow/commit/53e707428826d603f43aebb73e18fe077881c4b3?src=pr=desc) will **increase** coverage by `0.45%`. > The diff coverage is `100%`. [](https://codecov.io/gh/apache/incubator-airflow/pull/4407?src=pr=tree) ```diff @@Coverage Diff@@ ## master #4407 +/- ## = + Coverage 78.14% 78.6% +0.45% = Files 204 204 Lines 16433 16445 +12 = + Hits12842 12927 +85 + Misses 35913518 -73 ``` | [Impacted Files](https://codecov.io/gh/apache/incubator-airflow/pull/4407?src=pr=tree) | Coverage Δ | | |---|---|---| | [airflow/www/views.py](https://codecov.io/gh/apache/incubator-airflow/pull/4407/diff?src=pr=tree#diff-YWlyZmxvdy93d3cvdmlld3MucHk=) | `70.24% <100%> (+0.77%)` | :arrow_up: | | [airflow/www\_rbac/views.py](https://codecov.io/gh/apache/incubator-airflow/pull/4407/diff?src=pr=tree#diff-YWlyZmxvdy93d3dfcmJhYy92aWV3cy5weQ==) | `73.71% <100%> (+0.93%)` | :arrow_up: | | [airflow/models/\_\_init\_\_.py](https://codecov.io/gh/apache/incubator-airflow/pull/4407/diff?src=pr=tree#diff-YWlyZmxvdy9tb2RlbHMvX19pbml0X18ucHk=) | `92.6% <100%> (+0.02%)` | :arrow_up: | | [airflow/hooks/samba\_hook.py](https://codecov.io/gh/apache/incubator-airflow/pull/4407/diff?src=pr=tree#diff-YWlyZmxvdy9ob29rcy9zYW1iYV9ob29rLnB5) | `38.88% <0%> (+38.88%)` | :arrow_up: | | [airflow/operators/hive\_to\_samba\_operator.py](https://codecov.io/gh/apache/incubator-airflow/pull/4407/diff?src=pr=tree#diff-YWlyZmxvdy9vcGVyYXRvcnMvaGl2ZV90b19zYW1iYV9vcGVyYXRvci5weQ==) | `100% <0%> (+100%)` | :arrow_up: | | [airflow/operators/jdbc\_operator.py](https://codecov.io/gh/apache/incubator-airflow/pull/4407/diff?src=pr=tree#diff-YWlyZmxvdy9vcGVyYXRvcnMvamRiY19vcGVyYXRvci5weQ==) | `100% <0%> (+100%)` | :arrow_up: | -- [Continue to review full report at Codecov](https://codecov.io/gh/apache/incubator-airflow/pull/4407?src=pr=continue). > **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta) > `Δ = absolute (impact)`, `ø = not affected`, `? = missing data` > Powered by [Codecov](https://codecov.io/gh/apache/incubator-airflow/pull/4407?src=pr=footer). Last update [53e7074...0957961](https://codecov.io/gh/apache/incubator-airflow/pull/4407?src=pr=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments). This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Commented] (AIRFLOW-1161) HttpHook shall be configurable to non-status errors
[
https://issues.apache.org/jira/browse/AIRFLOW-1161?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16731256#comment-16731256
]
jack commented on AIRFLOW-1161:
---
But this is exactly what Airflow provides... If the task fails it will retry.
Are you requesting that the hook will have an inner retry loop?
> HttpHook shall be configurable to non-status errors
> ---
>
> Key: AIRFLOW-1161
> URL: https://issues.apache.org/jira/browse/AIRFLOW-1161
> Project: Apache Airflow
> Issue Type: Improvement
>Reporter: Deo
>Priority: Major
>
> When using HttpSensor, which underlying would use HttpHook to perform the
> request. If the target service is down, which refused the connection, the
> task would fail immediately.
> would be great if this behaviour is configurable, so the sensor would keep
> sensoring until the service is up again.
> traceback of the error:
> [2017-04-29 02:00:31,248] {base_task_runner.py:95} INFO - Subtask:
> requests.exceptions.ConnectionError: HTTPConnectionPool(host='',
> port=123): Max retries exceeded with url: / (Caused by
> NewConnectionError(' object at 0x7f94b64b44e0>: Failed to establish a new connection: [Errno 111]
> Connection refused',))
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[GitHub] Rotemlofer commented on issue #2879: [AIRFLOW-1921] Added support for presto https and user auth
Rotemlofer commented on issue #2879: [AIRFLOW-1921] Added support for presto https and user auth URL: https://github.com/apache/incubator-airflow/pull/2879#issuecomment-450626286 @Fokko is there a way to get this in? I could really use this This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] antweiss commented on issue #4404: Remove invalid parameter KeepJobFlowAliveWhenNoSteps
antweiss commented on issue #4404: Remove invalid parameter KeepJobFlowAliveWhenNoSteps URL: https://github.com/apache/incubator-airflow/pull/4404#issuecomment-450624523 @r39132 I'd love to. Can you give me a little direction as to how to create such a test? Are there tests for other DAG examples that I can learn from? This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Comment Edited] (AIRFLOW-1072) cannot import name viewkeys
[
https://issues.apache.org/jira/browse/AIRFLOW-1072?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16731232#comment-16731232
]
jack edited comment on AIRFLOW-1072 at 12/31/18 9:15 AM:
-
Airflow does not use html5lib directly.
Airflow uses bleach.
In any case [https://github.com/apache/incubator-airflow/pull/2858] upgraded
bleach to {color:#032f62}2.1.2{color} so I guess this issue is no longer
relevant.
was (Author: jackjack10):
Airflow does not use html5lib directly.
Airflow uses bleach.
In any case [https://github.com/apache/incubator-airflow/pull/2858] upgraded
bleach to 2.0.0 so I guess this issue is no longer relevant.
> cannot import name viewkeys
> ---
>
> Key: AIRFLOW-1072
> URL: https://issues.apache.org/jira/browse/AIRFLOW-1072
> Project: Apache Airflow
> Issue Type: Bug
>Reporter: Semet
>Priority: Major
>
> Here is an error when integrating Airflow (HEAD) inside a docker:
> {code}
> INFO [alembic.runtime.migration] Context impl PostgresqlImpl.
> INFO [alembic.runtime.migration] Will assume transactional DDL.
> /usr/local/lib/python2.7/dist-packages/airflow-1.9.0.dev0+apache.incubating-py2.7.egg/airflow/configuration.py:128:
> DeprecationWarning: This method will be removed in future versions. Use
> 'parser.read_file()' instead.
> self.readfp(StringIO.StringIO(string))
> /usr/local/lib/python2.7/dist-packages/flask/exthook.py:71:
> ExtDeprecationWarning: Importing flask.ext.cache is deprecated, use
> flask_cache instead.
> .format(x=modname), ExtDeprecationWarning
> Traceback (most recent call last):
> File "/usr/local/bin/airflow", line 4, in
>
> __import__('pkg_resources').run_script('airflow==1.9.0.dev0+apache.incubating',
> 'airflow')
> File "/usr/local/lib/python2.7/dist-packages/pkg_resources/__init__.py",
> line 738, in run_script
> self.require(requires)[0].run_script(script_name, ns)
> File "/usr/local/lib/python2.7/dist-packages/pkg_resources/__init__.py",
> line 1499, in run_script
> exec(code, namespace, namespace)
> File
> "/usr/local/lib/python2.7/dist-packages/airflow-1.9.0.dev0+apache.incubating-py2.7.egg/EGG-INFO/scripts/airflow",
> line 28, in
> args.func(args)
> File
> "/usr/local/lib/python2.7/dist-packages/airflow-1.9.0.dev0+apache.incubating-py2.7.egg/airflow/bin/cli.py",
> line 734, in webserver
> app = cached_app(conf)
> File
> "/usr/local/lib/python2.7/dist-packages/airflow-1.9.0.dev0+apache.incubating-py2.7.egg/airflow/www/app.py",
> line 161, in cached_app
> app = create_app(config)
> File
> "/usr/local/lib/python2.7/dist-packages/airflow-1.9.0.dev0+apache.incubating-py2.7.egg/airflow/www/app.py",
> line 60, in create_app
> from airflow.www import views
> File
> "/usr/local/lib/python2.7/dist-packages/airflow-1.9.0.dev0+apache.incubating-py2.7.egg/airflow/www/views.py",
> line 27, in
> import bleach
> File
> "/usr/local/lib/python2.7/dist-packages/bleach-2.0.0-py2.7.egg/bleach/__init__.py",
> line 5, in
> from bleach.linkifier import (
> File
> "/usr/local/lib/python2.7/dist-packages/bleach-2.0.0-py2.7.egg/bleach/linkifier.py",
> line 4, in
> import html5lib
> File "build/bdist.linux-x86_64/egg/html5lib/__init__.py", line 16, in
>
> File "build/bdist.linux-x86_64/egg/html5lib/html5parser.py", line 2, in
>
> ImportError: cannot import name viewkeys
> {code}
> Seems a bug in the [version of
> html5lib|http://stackoverflow.com/questions/39658404/python-cannot-import-name-viewkeys].
> Forcing it to {{html5lib==1.0b8}} seems to work.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (AIRFLOW-1754) Add GCP logging download for Dataflow operator
[ https://issues.apache.org/jira/browse/AIRFLOW-1754?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16731228#comment-16731228 ] jack commented on AIRFLOW-1754: --- Wasn't this already implemented? > Add GCP logging download for Dataflow operator > -- > > Key: AIRFLOW-1754 > URL: https://issues.apache.org/jira/browse/AIRFLOW-1754 > Project: Apache Airflow > Issue Type: New Feature >Reporter: Chris Riccomini >Priority: Major > Fix For: 1.10.0 > > > Based on conversation in AIRFLOW-1732 and > https://github.com/apache/incubator-airflow/pull/2702, there is useful > logging that occurs for Dataflow on the server-side (i.e. it's not visible > simply by piping client logs to the Airflow log file). > We should add a method to fetch logs from GCP logging (stack driver?), so we > can spool server side logging into the Dataflow operator for debugging > purposes. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[GitHub] codecov-io edited a comment on issue #4407: [AIRFLOW-3600] Remove dagbag from trigger
codecov-io edited a comment on issue #4407: [AIRFLOW-3600] Remove dagbag from trigger URL: https://github.com/apache/incubator-airflow/pull/4407#issuecomment-450585018 # [Codecov](https://codecov.io/gh/apache/incubator-airflow/pull/4407?src=pr=h1) Report > Merging [#4407](https://codecov.io/gh/apache/incubator-airflow/pull/4407?src=pr=desc) into [master](https://codecov.io/gh/apache/incubator-airflow/commit/53e707428826d603f43aebb73e18fe077881c4b3?src=pr=desc) will **increase** coverage by `0.27%`. > The diff coverage is `60%`. [](https://codecov.io/gh/apache/incubator-airflow/pull/4407?src=pr=tree) ```diff @@Coverage Diff @@ ## master#4407 +/- ## == + Coverage 78.14% 78.42% +0.27% == Files 204 204 Lines 1643316449 +16 == + Hits1284212900 +58 + Misses 3591 3549 -42 ``` | [Impacted Files](https://codecov.io/gh/apache/incubator-airflow/pull/4407?src=pr=tree) | Coverage Δ | | |---|---|---| | [airflow/www\_rbac/views.py](https://codecov.io/gh/apache/incubator-airflow/pull/4407/diff?src=pr=tree#diff-YWlyZmxvdy93d3dfcmJhYy92aWV3cy5weQ==) | `72.61% <25%> (-0.18%)` | :arrow_down: | | [airflow/www/views.py](https://codecov.io/gh/apache/incubator-airflow/pull/4407/diff?src=pr=tree#diff-YWlyZmxvdy93d3cvdmlld3MucHk=) | `69.5% <66.66%> (+0.03%)` | :arrow_up: | | [airflow/models/\_\_init\_\_.py](https://codecov.io/gh/apache/incubator-airflow/pull/4407/diff?src=pr=tree#diff-YWlyZmxvdy9tb2RlbHMvX19pbml0X18ucHk=) | `92.47% <88.88%> (-0.11%)` | :arrow_down: | | [airflow/hooks/samba\_hook.py](https://codecov.io/gh/apache/incubator-airflow/pull/4407/diff?src=pr=tree#diff-YWlyZmxvdy9ob29rcy9zYW1iYV9ob29rLnB5) | `38.88% <0%> (+38.88%)` | :arrow_up: | | [airflow/operators/hive\_to\_samba\_operator.py](https://codecov.io/gh/apache/incubator-airflow/pull/4407/diff?src=pr=tree#diff-YWlyZmxvdy9vcGVyYXRvcnMvaGl2ZV90b19zYW1iYV9vcGVyYXRvci5weQ==) | `100% <0%> (+100%)` | :arrow_up: | | [airflow/operators/jdbc\_operator.py](https://codecov.io/gh/apache/incubator-airflow/pull/4407/diff?src=pr=tree#diff-YWlyZmxvdy9vcGVyYXRvcnMvamRiY19vcGVyYXRvci5weQ==) | `100% <0%> (+100%)` | :arrow_up: | -- [Continue to review full report at Codecov](https://codecov.io/gh/apache/incubator-airflow/pull/4407?src=pr=continue). > **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta) > `Δ = absolute (impact)`, `ø = not affected`, `? = missing data` > Powered by [Codecov](https://codecov.io/gh/apache/incubator-airflow/pull/4407?src=pr=footer). Last update [53e7074...00e9667](https://codecov.io/gh/apache/incubator-airflow/pull/4407?src=pr=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments). This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] codecov-io edited a comment on issue #4407: [AIRFLOW-3600] Remove dagbag from trigger
codecov-io edited a comment on issue #4407: [AIRFLOW-3600] Remove dagbag from trigger URL: https://github.com/apache/incubator-airflow/pull/4407#issuecomment-450585018 # [Codecov](https://codecov.io/gh/apache/incubator-airflow/pull/4407?src=pr=h1) Report > Merging [#4407](https://codecov.io/gh/apache/incubator-airflow/pull/4407?src=pr=desc) into [master](https://codecov.io/gh/apache/incubator-airflow/commit/53e707428826d603f43aebb73e18fe077881c4b3?src=pr=desc) will **increase** coverage by `0.27%`. > The diff coverage is `60%`. [](https://codecov.io/gh/apache/incubator-airflow/pull/4407?src=pr=tree) ```diff @@Coverage Diff @@ ## master#4407 +/- ## == + Coverage 78.14% 78.42% +0.27% == Files 204 204 Lines 1643316449 +16 == + Hits1284212900 +58 + Misses 3591 3549 -42 ``` | [Impacted Files](https://codecov.io/gh/apache/incubator-airflow/pull/4407?src=pr=tree) | Coverage Δ | | |---|---|---| | [airflow/www\_rbac/views.py](https://codecov.io/gh/apache/incubator-airflow/pull/4407/diff?src=pr=tree#diff-YWlyZmxvdy93d3dfcmJhYy92aWV3cy5weQ==) | `72.61% <25%> (-0.18%)` | :arrow_down: | | [airflow/www/views.py](https://codecov.io/gh/apache/incubator-airflow/pull/4407/diff?src=pr=tree#diff-YWlyZmxvdy93d3cvdmlld3MucHk=) | `69.5% <66.66%> (+0.03%)` | :arrow_up: | | [airflow/models/\_\_init\_\_.py](https://codecov.io/gh/apache/incubator-airflow/pull/4407/diff?src=pr=tree#diff-YWlyZmxvdy9tb2RlbHMvX19pbml0X18ucHk=) | `92.47% <88.88%> (-0.11%)` | :arrow_down: | | [airflow/hooks/samba\_hook.py](https://codecov.io/gh/apache/incubator-airflow/pull/4407/diff?src=pr=tree#diff-YWlyZmxvdy9ob29rcy9zYW1iYV9ob29rLnB5) | `38.88% <0%> (+38.88%)` | :arrow_up: | | [airflow/operators/hive\_to\_samba\_operator.py](https://codecov.io/gh/apache/incubator-airflow/pull/4407/diff?src=pr=tree#diff-YWlyZmxvdy9vcGVyYXRvcnMvaGl2ZV90b19zYW1iYV9vcGVyYXRvci5weQ==) | `100% <0%> (+100%)` | :arrow_up: | | [airflow/operators/jdbc\_operator.py](https://codecov.io/gh/apache/incubator-airflow/pull/4407/diff?src=pr=tree#diff-YWlyZmxvdy9vcGVyYXRvcnMvamRiY19vcGVyYXRvci5weQ==) | `100% <0%> (+100%)` | :arrow_up: | -- [Continue to review full report at Codecov](https://codecov.io/gh/apache/incubator-airflow/pull/4407?src=pr=continue). > **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta) > `Δ = absolute (impact)`, `ø = not affected`, `? = missing data` > Powered by [Codecov](https://codecov.io/gh/apache/incubator-airflow/pull/4407?src=pr=footer). Last update [53e7074...00e9667](https://codecov.io/gh/apache/incubator-airflow/pull/4407?src=pr=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments). This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] codecov-io edited a comment on issue #4407: [AIRFLOW-3600] Remove dagbag from trigger
codecov-io edited a comment on issue #4407: [AIRFLOW-3600] Remove dagbag from trigger URL: https://github.com/apache/incubator-airflow/pull/4407#issuecomment-450585018 # [Codecov](https://codecov.io/gh/apache/incubator-airflow/pull/4407?src=pr=h1) Report > Merging [#4407](https://codecov.io/gh/apache/incubator-airflow/pull/4407?src=pr=desc) into [master](https://codecov.io/gh/apache/incubator-airflow/commit/53e707428826d603f43aebb73e18fe077881c4b3?src=pr=desc) will **increase** coverage by `0.27%`. > The diff coverage is `60%`. [](https://codecov.io/gh/apache/incubator-airflow/pull/4407?src=pr=tree) ```diff @@Coverage Diff @@ ## master#4407 +/- ## == + Coverage 78.14% 78.42% +0.27% == Files 204 204 Lines 1643316449 +16 == + Hits1284212900 +58 + Misses 3591 3549 -42 ``` | [Impacted Files](https://codecov.io/gh/apache/incubator-airflow/pull/4407?src=pr=tree) | Coverage Δ | | |---|---|---| | [airflow/www\_rbac/views.py](https://codecov.io/gh/apache/incubator-airflow/pull/4407/diff?src=pr=tree#diff-YWlyZmxvdy93d3dfcmJhYy92aWV3cy5weQ==) | `72.61% <25%> (-0.18%)` | :arrow_down: | | [airflow/www/views.py](https://codecov.io/gh/apache/incubator-airflow/pull/4407/diff?src=pr=tree#diff-YWlyZmxvdy93d3cvdmlld3MucHk=) | `69.5% <66.66%> (+0.03%)` | :arrow_up: | | [airflow/models/\_\_init\_\_.py](https://codecov.io/gh/apache/incubator-airflow/pull/4407/diff?src=pr=tree#diff-YWlyZmxvdy9tb2RlbHMvX19pbml0X18ucHk=) | `92.47% <88.88%> (-0.11%)` | :arrow_down: | | [airflow/hooks/samba\_hook.py](https://codecov.io/gh/apache/incubator-airflow/pull/4407/diff?src=pr=tree#diff-YWlyZmxvdy9ob29rcy9zYW1iYV9ob29rLnB5) | `38.88% <0%> (+38.88%)` | :arrow_up: | | [airflow/operators/hive\_to\_samba\_operator.py](https://codecov.io/gh/apache/incubator-airflow/pull/4407/diff?src=pr=tree#diff-YWlyZmxvdy9vcGVyYXRvcnMvaGl2ZV90b19zYW1iYV9vcGVyYXRvci5weQ==) | `100% <0%> (+100%)` | :arrow_up: | | [airflow/operators/jdbc\_operator.py](https://codecov.io/gh/apache/incubator-airflow/pull/4407/diff?src=pr=tree#diff-YWlyZmxvdy9vcGVyYXRvcnMvamRiY19vcGVyYXRvci5weQ==) | `100% <0%> (+100%)` | :arrow_up: | -- [Continue to review full report at Codecov](https://codecov.io/gh/apache/incubator-airflow/pull/4407?src=pr=continue). > **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta) > `Δ = absolute (impact)`, `ø = not affected`, `? = missing data` > Powered by [Codecov](https://codecov.io/gh/apache/incubator-airflow/pull/4407?src=pr=footer). Last update [53e7074...00e9667](https://codecov.io/gh/apache/incubator-airflow/pull/4407?src=pr=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments). This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] codecov-io edited a comment on issue #4309: [AIRFLOW-3504] Extend/refine the functionality of "/health" endpoint
codecov-io edited a comment on issue #4309: [AIRFLOW-3504] Extend/refine the functionality of "/health" endpoint URL: https://github.com/apache/incubator-airflow/pull/4309#issuecomment-447846696 # [Codecov](https://codecov.io/gh/apache/incubator-airflow/pull/4309?src=pr=h1) Report > Merging [#4309](https://codecov.io/gh/apache/incubator-airflow/pull/4309?src=pr=desc) into [master](https://codecov.io/gh/apache/incubator-airflow/commit/ecc88a532658db249c3413f4e4c812b6800de725?src=pr=desc) will **increase** coverage by `<.01%`. > The diff coverage is `85.36%`. [](https://codecov.io/gh/apache/incubator-airflow/pull/4309?src=pr=tree) ```diff @@Coverage Diff @@ ## master#4309 +/- ## == + Coverage 78.44% 78.45% +<.01% == Files 204 204 Lines 1643616469 +33 == + Hits1289412920 +26 - Misses 3542 3549 +7 ``` | [Impacted Files](https://codecov.io/gh/apache/incubator-airflow/pull/4309?src=pr=tree) | Coverage Δ | | |---|---|---| | [airflow/www/views.py](https://codecov.io/gh/apache/incubator-airflow/pull/4309/diff?src=pr=tree#diff-YWlyZmxvdy93d3cvdmlld3MucHk=) | `69.47% <ø> (ø)` | :arrow_up: | | [airflow/www\_rbac/blueprints.py](https://codecov.io/gh/apache/incubator-airflow/pull/4309/diff?src=pr=tree#diff-YWlyZmxvdy93d3dfcmJhYy9ibHVlcHJpbnRzLnB5) | `100% <100%> (ø)` | :arrow_up: | | [airflow/www\_rbac/views.py](https://codecov.io/gh/apache/incubator-airflow/pull/4309/diff?src=pr=tree#diff-YWlyZmxvdy93d3dfcmJhYy92aWV3cy5weQ==) | `72.94% <84.21%> (+0.15%)` | :arrow_up: | | [airflow/www/blueprints.py](https://codecov.io/gh/apache/incubator-airflow/pull/4309/diff?src=pr=tree#diff-YWlyZmxvdy93d3cvYmx1ZXByaW50cy5weQ==) | `88.46% <85.71%> (-11.54%)` | :arrow_down: | | [airflow/models/\_\_init\_\_.py](https://codecov.io/gh/apache/incubator-airflow/pull/4309/diff?src=pr=tree#diff-YWlyZmxvdy9tb2RlbHMvX19pbml0X18ucHk=) | `92.54% <0%> (-0.05%)` | :arrow_down: | -- [Continue to review full report at Codecov](https://codecov.io/gh/apache/incubator-airflow/pull/4309?src=pr=continue). > **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta) > `Δ = absolute (impact)`, `ø = not affected`, `? = missing data` > Powered by [Codecov](https://codecov.io/gh/apache/incubator-airflow/pull/4309?src=pr=footer). Last update [ecc88a5...2b365e2](https://codecov.io/gh/apache/incubator-airflow/pull/4309?src=pr=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments). This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] codecov-io edited a comment on issue #4309: [AIRFLOW-3504] Extend/refine the functionality of "/health" endpoint
codecov-io edited a comment on issue #4309: [AIRFLOW-3504] Extend/refine the functionality of "/health" endpoint URL: https://github.com/apache/incubator-airflow/pull/4309#issuecomment-447846696 # [Codecov](https://codecov.io/gh/apache/incubator-airflow/pull/4309?src=pr=h1) Report > Merging [#4309](https://codecov.io/gh/apache/incubator-airflow/pull/4309?src=pr=desc) into [master](https://codecov.io/gh/apache/incubator-airflow/commit/ecc88a532658db249c3413f4e4c812b6800de725?src=pr=desc) will **increase** coverage by `<.01%`. > The diff coverage is `85.36%`. [](https://codecov.io/gh/apache/incubator-airflow/pull/4309?src=pr=tree) ```diff @@Coverage Diff @@ ## master#4309 +/- ## == + Coverage 78.44% 78.45% +<.01% == Files 204 204 Lines 1643616469 +33 == + Hits1289412920 +26 - Misses 3542 3549 +7 ``` | [Impacted Files](https://codecov.io/gh/apache/incubator-airflow/pull/4309?src=pr=tree) | Coverage Δ | | |---|---|---| | [airflow/www/views.py](https://codecov.io/gh/apache/incubator-airflow/pull/4309/diff?src=pr=tree#diff-YWlyZmxvdy93d3cvdmlld3MucHk=) | `69.47% <ø> (ø)` | :arrow_up: | | [airflow/www\_rbac/blueprints.py](https://codecov.io/gh/apache/incubator-airflow/pull/4309/diff?src=pr=tree#diff-YWlyZmxvdy93d3dfcmJhYy9ibHVlcHJpbnRzLnB5) | `100% <100%> (ø)` | :arrow_up: | | [airflow/www\_rbac/views.py](https://codecov.io/gh/apache/incubator-airflow/pull/4309/diff?src=pr=tree#diff-YWlyZmxvdy93d3dfcmJhYy92aWV3cy5weQ==) | `72.94% <84.21%> (+0.15%)` | :arrow_up: | | [airflow/www/blueprints.py](https://codecov.io/gh/apache/incubator-airflow/pull/4309/diff?src=pr=tree#diff-YWlyZmxvdy93d3cvYmx1ZXByaW50cy5weQ==) | `88.46% <85.71%> (-11.54%)` | :arrow_down: | | [airflow/models/\_\_init\_\_.py](https://codecov.io/gh/apache/incubator-airflow/pull/4309/diff?src=pr=tree#diff-YWlyZmxvdy9tb2RlbHMvX19pbml0X18ucHk=) | `92.54% <0%> (-0.05%)` | :arrow_down: | -- [Continue to review full report at Codecov](https://codecov.io/gh/apache/incubator-airflow/pull/4309?src=pr=continue). > **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta) > `Δ = absolute (impact)`, `ø = not affected`, `? = missing data` > Powered by [Codecov](https://codecov.io/gh/apache/incubator-airflow/pull/4309?src=pr=footer). Last update [ecc88a5...2b365e2](https://codecov.io/gh/apache/incubator-airflow/pull/4309?src=pr=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments). This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
