[GitHub] [airflow] MichaelChirico opened a new pull request #7463: [AIRFLOW-XXXX] clarify dag_id parameter
MichaelChirico opened a new pull request #7463: [AIRFLOW-] clarify dag_id parameter URL: https://github.com/apache/airflow/pull/7463 Re-filing #7462 here: I spent a while chasing this just now, wondering whether `:` is valid before finding `validate_key()` in helpers. I think it's helpful to surface this here directly. Trivial change so eschewing the other formality required in the PR template This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [airflow] stale[bot] commented on issue #6501: [AIRFLOW-5831] Prod image support. Depends on [AIRFLOW-5704] [AIRFLOW-5842] [AIRFLOW-5828]
stale[bot] commented on issue #6501: [AIRFLOW-5831] Prod image support. Depends on [AIRFLOW-5704] [AIRFLOW-5842] [AIRFLOW-5828] URL: https://github.com/apache/airflow/pull/6501#issuecomment-588084237 This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [airflow] MichaelChirico closed pull request #7462: [AIRFLOW-XXXX] clarify dag_id parameter
MichaelChirico closed pull request #7462: [AIRFLOW-] clarify dag_id parameter URL: https://github.com/apache/airflow/pull/7462 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [airflow] MichaelChirico commented on issue #7462: [AIRFLOW-XXXX] clarify dag_id parameter
MichaelChirico commented on issue #7462: [AIRFLOW-] clarify dag_id parameter URL: https://github.com/apache/airflow/pull/7462#issuecomment-588083940 let's try again This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [airflow] zhongjiajie commented on issue #7462: [AIRFLOW-XXXX] clarify dag_id parameter
zhongjiajie commented on issue #7462: [AIRFLOW-] clarify dag_id parameter URL: https://github.com/apache/airflow/pull/7462#issuecomment-588071249 @MichaelChirico use `git push -f` to restart it. maybe? This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [airflow] zhongjiajie edited a comment on issue #7462: [AIRFLOW-XXXX] clarify dag_id parameter
zhongjiajie edited a comment on issue #7462: [AIRFLOW-] clarify dag_id parameter URL: https://github.com/apache/airflow/pull/7462#issuecomment-588071249 @MichaelChirico use `git push -f` to restart it. maybe? This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [airflow] MichaelChirico commented on issue #7462: [AIRFLOW-XXXX] clarify dag_id parameter
MichaelChirico commented on issue #7462: [AIRFLOW-] clarify dag_id parameter URL: https://github.com/apache/airflow/pull/7462#issuecomment-588065081 Is there a way to re-trigger the Title Validator without adding commits? This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [airflow] MichaelChirico opened a new pull request #7462: [Documentation] clarify dag_id parameter
MichaelChirico opened a new pull request #7462: [Documentation] clarify dag_id parameter URL: https://github.com/apache/airflow/pull/7462 I spent a while chasing this just now, wondering whether `:` is valid before finding `validate_key()` in helpers. I think it's helpful to surface this here directly. Trivial change so eschewing the other formality required in the PR template This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [airflow] boring-cyborg[bot] commented on issue #7462: [Documentation] clarify dag_id parameter
boring-cyborg[bot] commented on issue #7462: [Documentation] clarify dag_id parameter URL: https://github.com/apache/airflow/pull/7462#issuecomment-588064513 Congratulations on your first Pull Request and welcome to the Apache Airflow community! If you have any issues or are unsure about any anything please check our Contribution Guide (https://github.com/apache/airflow/blob/master/CONTRIBUTING.rst) Here are some useful points: - Pay attention to the quality of your code (flake8, pylint and type annotations). Our [pre-commits]( https://github.com/apache/airflow/blob/master/STATIC_CODE_CHECKS.rst#prerequisites-for-pre-commit-hooks) will help you with that. - In case of a new feature add useful documentation (in docstrings or in `docs/` directory). Adding a new operator? Check this short [guide](https://github.com/apache/airflow/blob/master/docs/howto/custom-operator.rst) Consider adding an example DAG that shows how users should use it. - Consider using [Breeze environment](https://github.com/apache/airflow/blob/master/BREEZE.rst) for testing locally, it’s a heavy docker but it ships with a working Airflow and a lot of integrations. - Be patient and persistent. It might take some time to get a review or get the final approval from Committers. Apache Airflow is a community-driven project and together we are making it better 🚀. In case of doubts contact the developers at: Mailing List: d...@airflow.apache.org Slack: https://apache-airflow-slack.herokuapp.com/ This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [airflow] codenamelxl commented on issue #7374: [AIRFLOW-6399] Serialization: DAG access_control field should be deco…
codenamelxl commented on issue #7374: [AIRFLOW-6399] Serialization: DAG access_control field should be deco… URL: https://github.com/apache/airflow/pull/7374#issuecomment-588056628 @kaxil Test run OK with Postgres. I'm not sure what wrong with the test for MySQL and SQLite. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [airflow] codecov-io commented on issue #7457: [AIRFLOW-6837] limit description len of dag in home
codecov-io commented on issue #7457: [AIRFLOW-6837] limit description len of dag in home URL: https://github.com/apache/airflow/pull/7457#issuecomment-588048473 # [Codecov](https://codecov.io/gh/apache/airflow/pull/7457?src=pr&el=h1) Report > Merging [#7457](https://codecov.io/gh/apache/airflow/pull/7457?src=pr&el=desc) into [master](https://codecov.io/gh/apache/airflow/commit/65dd28eb77d996ec8306c67d5ce1ccee2c14cc9d?src=pr&el=desc) will **decrease** coverage by `0.17%`. > The diff coverage is `n/a`. [![Impacted file tree graph](https://codecov.io/gh/apache/airflow/pull/7457/graphs/tree.svg?width=650&token=WdLKlKHOAU&height=150&src=pr)](https://codecov.io/gh/apache/airflow/pull/7457?src=pr&el=tree) ```diff @@Coverage Diff @@ ## master#7457 +/- ## == - Coverage 86.62% 86.45% -0.18% == Files 878 878 Lines 4129241292 == - Hits3577035697 -73 - Misses 5522 5595 +73 ``` | [Impacted Files](https://codecov.io/gh/apache/airflow/pull/7457?src=pr&el=tree) | Coverage Δ | | |---|---|---| | [...w/providers/apache/hive/operators/mysql\_to\_hive.py](https://codecov.io/gh/apache/airflow/pull/7457/diff?src=pr&el=tree#diff-YWlyZmxvdy9wcm92aWRlcnMvYXBhY2hlL2hpdmUvb3BlcmF0b3JzL215c3FsX3RvX2hpdmUucHk=) | `35.84% <0%> (-64.16%)` | :arrow_down: | | [airflow/security/kerberos.py](https://codecov.io/gh/apache/airflow/pull/7457/diff?src=pr&el=tree#diff-YWlyZmxvdy9zZWN1cml0eS9rZXJiZXJvcy5weQ==) | `30.43% <0%> (-45.66%)` | :arrow_down: | | [airflow/providers/mysql/operators/mysql.py](https://codecov.io/gh/apache/airflow/pull/7457/diff?src=pr&el=tree#diff-YWlyZmxvdy9wcm92aWRlcnMvbXlzcWwvb3BlcmF0b3JzL215c3FsLnB5) | `55% <0%> (-45%)` | :arrow_down: | | [airflow/providers/apache/hive/hooks/hive.py](https://codecov.io/gh/apache/airflow/pull/7457/diff?src=pr&el=tree#diff-YWlyZmxvdy9wcm92aWRlcnMvYXBhY2hlL2hpdmUvaG9va3MvaGl2ZS5weQ==) | `76.02% <0%> (-1.54%)` | :arrow_down: | | [airflow/hooks/dbapi\_hook.py](https://codecov.io/gh/apache/airflow/pull/7457/diff?src=pr&el=tree#diff-YWlyZmxvdy9ob29rcy9kYmFwaV9ob29rLnB5) | `90.9% <0%> (-0.83%)` | :arrow_down: | | [airflow/jobs/backfill\_job.py](https://codecov.io/gh/apache/airflow/pull/7457/diff?src=pr&el=tree#diff-YWlyZmxvdy9qb2JzL2JhY2tmaWxsX2pvYi5weQ==) | `91.8% <0%> (-0.29%)` | :arrow_down: | | [airflow/jobs/scheduler\_job.py](https://codecov.io/gh/apache/airflow/pull/7457/diff?src=pr&el=tree#diff-YWlyZmxvdy9qb2JzL3NjaGVkdWxlcl9qb2IucHk=) | `89.51% <0%> (-0.15%)` | :arrow_down: | -- [Continue to review full report at Codecov](https://codecov.io/gh/apache/airflow/pull/7457?src=pr&el=continue). > **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta) > `Δ = absolute (impact)`, `ø = not affected`, `? = missing data` > Powered by [Codecov](https://codecov.io/gh/apache/airflow/pull/7457?src=pr&el=footer). Last update [65dd28e...089c99d](https://codecov.io/gh/apache/airflow/pull/7457?src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments). This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [airflow] boring-cyborg[bot] commented on issue #7461: [AIRFLOW-XXXX] Update write-logs.rst, add instructions for logging to localstack S3
boring-cyborg[bot] commented on issue #7461: [AIRFLOW-] Update write-logs.rst, add instructions for logging to localstack S3 URL: https://github.com/apache/airflow/pull/7461#issuecomment-588037518 Congratulations on your first Pull Request and welcome to the Apache Airflow community! If you have any issues or are unsure about any anything please check our Contribution Guide (https://github.com/apache/airflow/blob/master/CONTRIBUTING.rst) Here are some useful points: - Pay attention to the quality of your code (flake8, pylint and type annotations). Our [pre-commits]( https://github.com/apache/airflow/blob/master/STATIC_CODE_CHECKS.rst#prerequisites-for-pre-commit-hooks) will help you with that. - In case of a new feature add useful documentation (in docstrings or in `docs/` directory). Adding a new operator? Check this short [guide](https://github.com/apache/airflow/blob/master/docs/howto/custom-operator.rst) Consider adding an example DAG that shows how users should use it. - Consider using [Breeze environment](https://github.com/apache/airflow/blob/master/BREEZE.rst) for testing locally, it’s a heavy docker but it ships with a working Airflow and a lot of integrations. - Be patient and persistent. It might take some time to get a review or get the final approval from Committers. Apache Airflow is a community-driven project and together we are making it better 🚀. In case of doubts contact the developers at: Mailing List: d...@airflow.apache.org Slack: https://apache-airflow-slack.herokuapp.com/ This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [airflow] redSlug opened a new pull request #7461: [AIRFLOW-XXXX] Update write-logs.rst, add instructions for logging to localstack S3
redSlug opened a new pull request #7461: [AIRFLOW-] Update write-logs.rst, add instructions for logging to localstack S3 URL: https://github.com/apache/airflow/pull/7461 --- Issue link: Document only change, no JIRA issue Make sure to mark the boxes below before creating PR: [x] - [ ] Description above provides context of the change - [ ] Commit message/PR title starts with `[AIRFLOW-]`. AIRFLOW- = JIRA ID* - [ ] Unit tests coverage for changes (not needed for documentation changes) - [ ] Commits follow "[How to write a good git commit message](http://chris.beams.io/posts/git-commit/)" - [ ] Relevant documentation is updated including usage instructions. - [ ] I will engage committers as explained in [Contribution Workflow Example](https://github.com/apache/airflow/blob/master/CONTRIBUTING.rst#contribution-workflow-example). * For document-only changes commit message can start with `[AIRFLOW-]`. --- In case of fundamental code change, Airflow Improvement Proposal ([AIP](https://cwiki.apache.org/confluence/display/AIRFLOW/Airflow+Improvements+Proposals)) is needed. In case of a new dependency, check compliance with the [ASF 3rd Party License Policy](https://www.apache.org/legal/resolved.html#category-x). In case of backwards incompatible changes please leave a note in [UPDATING.md](https://github.com/apache/airflow/blob/master/UPDATING.md). Read the [Pull Request Guidelines](https://github.com/apache/airflow/blob/master/CONTRIBUTING.rst#pull-request-guidelines) for more information. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [airflow] simis2626 commented on issue #7395: [AIRFLOW-5629] Implement K8s priorityClassName in KubernetesPo…
simis2626 commented on issue #7395: [AIRFLOW-5629] Implement K8s priorityClassName in KubernetesPo… URL: https://github.com/apache/airflow/pull/7395#issuecomment-588032431 Hi @dimberman Appreciate your recent return to this work, just pinging you incase it's slipped by. Thanks in advance, appreciate your support & time. Simon This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [airflow] mik-laj commented on issue #7458: [AIRFLOW-6838][WIP] Introduce real subcommands for Breeze
mik-laj commented on issue #7458: [AIRFLOW-6838][WIP] Introduce real subcommands for Breeze URL: https://github.com/apache/airflow/pull/7458#issuecomment-588015135 I updated autocompllete. This should also solve the problems with getopt. > you can do ./breeeze and it will do something instead of complaining. Thanks. I added missing check. I still have to do rebase, but I will do it tomorrow. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [airflow] potiuk commented on issue #7458: [AIRFLOW-6838][WIP] Introduce real subcommands for Breeze
potiuk commented on issue #7458: [AIRFLOW-6838][WIP] Introduce real subcommands for Breeze URL: https://github.com/apache/airflow/pull/7458#issuecomment-588009246 Even if you provide --stop, get opt will suggest that it is ambigius and will tell you that --stop-environment is one of possible options. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [airflow] potiuk commented on issue #7458: [AIRFLOW-6838][WIP] Introduce real subcommands for Breeze
potiuk commented on issue #7458: [AIRFLOW-6838][WIP] Introduce real subcommands for Breeze URL: https://github.com/apache/airflow/pull/7458#issuecomment-588009056 Sure: getopt verifies switches provided and will complain if the switch is wrong. Try --nnn . with the command specified you get no verification you can do ./breeeze and it will do something instead of complaining. If you specify --stop-environment as switch, it will pass over getopt but --stop-environments will fail in get_opt. And the list of options (available in breeze-complete) is not updated. The same list is used for autocomplete This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [airflow] codecov-io edited a comment on issue #7459: [AIRFLOW-6820] fixup! fix unbound variable
codecov-io edited a comment on issue #7459: [AIRFLOW-6820] fixup! fix unbound variable URL: https://github.com/apache/airflow/pull/7459#issuecomment-588007834 # [Codecov](https://codecov.io/gh/apache/airflow/pull/7459?src=pr&el=h1) Report > Merging [#7459](https://codecov.io/gh/apache/airflow/pull/7459?src=pr&el=desc) into [master](https://codecov.io/gh/apache/airflow/commit/24f9f11775bf7b38cced9955f6c70705ad56f61c?src=pr&el=desc) will **not change** coverage. > The diff coverage is `n/a`. [![Impacted file tree graph](https://codecov.io/gh/apache/airflow/pull/7459/graphs/tree.svg?width=650&token=WdLKlKHOAU&height=150&src=pr)](https://codecov.io/gh/apache/airflow/pull/7459?src=pr&el=tree) ```diff @@ Coverage Diff @@ ## master#7459 +/- ## === Coverage 86.62% 86.62% === Files 878 878 Lines 4129241292 === Hits3577035770 Misses 5522 5522 ``` -- [Continue to review full report at Codecov](https://codecov.io/gh/apache/airflow/pull/7459?src=pr&el=continue). > **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta) > `Δ = absolute (impact)`, `ø = not affected`, `? = missing data` > Powered by [Codecov](https://codecov.io/gh/apache/airflow/pull/7459?src=pr&el=footer). Last update [24f9f11...a54cf8b](https://codecov.io/gh/apache/airflow/pull/7459?src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments). This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [airflow] codecov-io commented on issue #7459: [AIRFLOW-6820] fixup! fix unbound variable
codecov-io commented on issue #7459: [AIRFLOW-6820] fixup! fix unbound variable URL: https://github.com/apache/airflow/pull/7459#issuecomment-588007834 # [Codecov](https://codecov.io/gh/apache/airflow/pull/7459?src=pr&el=h1) Report > Merging [#7459](https://codecov.io/gh/apache/airflow/pull/7459?src=pr&el=desc) into [master](https://codecov.io/gh/apache/airflow/commit/24f9f11775bf7b38cced9955f6c70705ad56f61c?src=pr&el=desc) will **decrease** coverage by `0.18%`. > The diff coverage is `n/a`. [![Impacted file tree graph](https://codecov.io/gh/apache/airflow/pull/7459/graphs/tree.svg?width=650&token=WdLKlKHOAU&height=150&src=pr)](https://codecov.io/gh/apache/airflow/pull/7459?src=pr&el=tree) ```diff @@Coverage Diff @@ ## master#7459 +/- ## == - Coverage 86.62% 86.44% -0.19% == Files 878 878 Lines 4129241292 == - Hits3577035694 -76 - Misses 5522 5598 +76 ``` | [Impacted Files](https://codecov.io/gh/apache/airflow/pull/7459?src=pr&el=tree) | Coverage Δ | | |---|---|---| | [...w/providers/apache/hive/operators/mysql\_to\_hive.py](https://codecov.io/gh/apache/airflow/pull/7459/diff?src=pr&el=tree#diff-YWlyZmxvdy9wcm92aWRlcnMvYXBhY2hlL2hpdmUvb3BlcmF0b3JzL215c3FsX3RvX2hpdmUucHk=) | `35.84% <0%> (-64.16%)` | :arrow_down: | | [airflow/security/kerberos.py](https://codecov.io/gh/apache/airflow/pull/7459/diff?src=pr&el=tree#diff-YWlyZmxvdy9zZWN1cml0eS9rZXJiZXJvcy5weQ==) | `30.43% <0%> (-45.66%)` | :arrow_down: | | [airflow/providers/mysql/operators/mysql.py](https://codecov.io/gh/apache/airflow/pull/7459/diff?src=pr&el=tree#diff-YWlyZmxvdy9wcm92aWRlcnMvbXlzcWwvb3BlcmF0b3JzL215c3FsLnB5) | `55% <0%> (-45%)` | :arrow_down: | | [airflow/providers/apache/hive/hooks/hive.py](https://codecov.io/gh/apache/airflow/pull/7459/diff?src=pr&el=tree#diff-YWlyZmxvdy9wcm92aWRlcnMvYXBhY2hlL2hpdmUvaG9va3MvaGl2ZS5weQ==) | `76.02% <0%> (-1.54%)` | :arrow_down: | | [airflow/jobs/backfill\_job.py](https://codecov.io/gh/apache/airflow/pull/7459/diff?src=pr&el=tree#diff-YWlyZmxvdy9qb2JzL2JhY2tmaWxsX2pvYi5weQ==) | `90.96% <0%> (-1.13%)` | :arrow_down: | | [airflow/hooks/dbapi\_hook.py](https://codecov.io/gh/apache/airflow/pull/7459/diff?src=pr&el=tree#diff-YWlyZmxvdy9ob29rcy9kYmFwaV9ob29rLnB5) | `90.9% <0%> (-0.83%)` | :arrow_down: | | [airflow/jobs/scheduler\_job.py](https://codecov.io/gh/apache/airflow/pull/7459/diff?src=pr&el=tree#diff-YWlyZmxvdy9qb2JzL3NjaGVkdWxlcl9qb2IucHk=) | `89.51% <0%> (-0.15%)` | :arrow_down: | -- [Continue to review full report at Codecov](https://codecov.io/gh/apache/airflow/pull/7459?src=pr&el=continue). > **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta) > `Δ = absolute (impact)`, `ø = not affected`, `? = missing data` > Powered by [Codecov](https://codecov.io/gh/apache/airflow/pull/7459?src=pr&el=footer). Last update [24f9f11...a54cf8b](https://codecov.io/gh/apache/airflow/pull/7459?src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments). This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Commented] (AIRFLOW-6820) Split breeze into functions
[ https://issues.apache.org/jira/browse/AIRFLOW-6820?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17039654#comment-17039654 ] ASF GitHub Bot commented on AIRFLOW-6820: - potiuk commented on pull request #7459: [AIRFLOW-6820] fixup! fix unbound variable URL: https://github.com/apache/airflow/pull/7459 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org > Split breeze into functions > --- > > Key: AIRFLOW-6820 > URL: https://issues.apache.org/jira/browse/AIRFLOW-6820 > Project: Apache Airflow > Issue Type: Improvement > Components: breeze >Affects Versions: 1.10.10 >Reporter: Jarek Potiuk >Priority: Major > Fix For: 2.0.0 > > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (AIRFLOW-6820) Split breeze into functions
[ https://issues.apache.org/jira/browse/AIRFLOW-6820?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17039655#comment-17039655 ] ASF subversion and git services commented on AIRFLOW-6820: -- Commit d2ab8b8f5f921361dc6f28e21c6e588693c0e81d in airflow's branch refs/heads/master from Jarek Potiuk [ https://gitbox.apache.org/repos/asf?p=airflow.git;h=d2ab8b8 ] [AIRFLOW-6820] fixup! fix unbound variable (#7459) > Split breeze into functions > --- > > Key: AIRFLOW-6820 > URL: https://issues.apache.org/jira/browse/AIRFLOW-6820 > Project: Apache Airflow > Issue Type: Improvement > Components: breeze >Affects Versions: 1.10.10 >Reporter: Jarek Potiuk >Priority: Major > Fix For: 2.0.0 > > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[GitHub] [airflow] potiuk merged pull request #7459: [AIRFLOW-6820] fixup! fix unbound variable
potiuk merged pull request #7459: [AIRFLOW-6820] fixup! fix unbound variable URL: https://github.com/apache/airflow/pull/7459 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Commented] (AIRFLOW-6839) Even more mypy speedup
[ https://issues.apache.org/jira/browse/AIRFLOW-6839?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17039653#comment-17039653 ] ASF GitHub Bot commented on AIRFLOW-6839: - potiuk commented on pull request #7460: [AIRFLOW-6839] even more mypy speed improvements URL: https://github.com/apache/airflow/pull/7460 Require_serial:true is better choice than pass_filename: false as it can speed-up mypy for single file changes. Significant gains can be achieved for single file changes and no cache for all other files. This is majority of cases for our users who have pre-commits installed as hooks because most people change only few files and never run check with --all-files When just one file is changed and no cache is built, the difference is drastic: require_serial: true = 4s pass_filenames: false = 13s --- Issue link: WILL BE INSERTED BY [boring-cyborg](https://github.com/kaxil/boring-cyborg) Make sure to mark the boxes below before creating PR: [x] - [x] Description above provides context of the change - [x] Commit message/PR title starts with `[AIRFLOW-]`. AIRFLOW- = JIRA ID* - [x] Unit tests coverage for changes (not needed for documentation changes) - [x] Commits follow "[How to write a good git commit message](http://chris.beams.io/posts/git-commit/)" - [x] Relevant documentation is updated including usage instructions. - [x] I will engage committers as explained in [Contribution Workflow Example](https://github.com/apache/airflow/blob/master/CONTRIBUTING.rst#contribution-workflow-example). * For document-only changes commit message can start with `[AIRFLOW-]`. --- In case of fundamental code change, Airflow Improvement Proposal ([AIP](https://cwiki.apache.org/confluence/display/AIRFLOW/Airflow+Improvements+Proposals)) is needed. In case of a new dependency, check compliance with the [ASF 3rd Party License Policy](https://www.apache.org/legal/resolved.html#category-x). In case of backwards incompatible changes please leave a note in [UPDATING.md](https://github.com/apache/airflow/blob/master/UPDATING.md). Read the [Pull Request Guidelines](https://github.com/apache/airflow/blob/master/CONTRIBUTING.rst#pull-request-guidelines) for more information. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org > Even more mypy speedup > -- > > Key: AIRFLOW-6839 > URL: https://issues.apache.org/jira/browse/AIRFLOW-6839 > Project: Apache Airflow > Issue Type: Improvement > Components: pre-commit >Affects Versions: 2.0.0, 1.10.9 >Reporter: Jarek Potiuk >Priority: Major > > require_serial:true is better choice than pass_filename: false as it can > speed-up mypy for single file changes. > Significant gains can be achieved for single file changes and no cache for > all other files. This is majority of cases for our users who have pre-commits > installed as hooks because most people change only few files and never run > check with --all-files > When just one file is changed and no cache is built, the difference is > drastic: > require_serial: true (4s) > > {code:java} > Run > mypy.PassedRun > mypy.Passed > real 0m4,080s user 0m0,283s sys 0m0,095s > {code} > pass_filenames: false: (13s) > {code:java} > time pre-commit run mypy time pre-commit run mypyRun > mypy.Passed > real 0m13,756suser 0m0,281ssys 0m0,099s{code} > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[GitHub] [airflow] potiuk commented on issue #7459: [AIRFLOW-6820] fixup! fix unbound variable
potiuk commented on issue #7459: [AIRFLOW-6820] fixup! fix unbound variable URL: https://github.com/apache/airflow/pull/7459#issuecomment-588002347 I'll merge this now - this is a fixup to just merged commit with one variable set - to prevent people to get it with breeze (breeze was failing). This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [airflow] potiuk opened a new pull request #7460: [AIRFLOW-6839] even more mypy speed improvements
potiuk opened a new pull request #7460: [AIRFLOW-6839] even more mypy speed improvements URL: https://github.com/apache/airflow/pull/7460 Require_serial:true is better choice than pass_filename: false as it can speed-up mypy for single file changes. Significant gains can be achieved for single file changes and no cache for all other files. This is majority of cases for our users who have pre-commits installed as hooks because most people change only few files and never run check with --all-files When just one file is changed and no cache is built, the difference is drastic: require_serial: true = 4s pass_filenames: false = 13s --- Issue link: WILL BE INSERTED BY [boring-cyborg](https://github.com/kaxil/boring-cyborg) Make sure to mark the boxes below before creating PR: [x] - [x] Description above provides context of the change - [x] Commit message/PR title starts with `[AIRFLOW-]`. AIRFLOW- = JIRA ID* - [x] Unit tests coverage for changes (not needed for documentation changes) - [x] Commits follow "[How to write a good git commit message](http://chris.beams.io/posts/git-commit/)" - [x] Relevant documentation is updated including usage instructions. - [x] I will engage committers as explained in [Contribution Workflow Example](https://github.com/apache/airflow/blob/master/CONTRIBUTING.rst#contribution-workflow-example). * For document-only changes commit message can start with `[AIRFLOW-]`. --- In case of fundamental code change, Airflow Improvement Proposal ([AIP](https://cwiki.apache.org/confluence/display/AIRFLOW/Airflow+Improvements+Proposals)) is needed. In case of a new dependency, check compliance with the [ASF 3rd Party License Policy](https://www.apache.org/legal/resolved.html#category-x). In case of backwards incompatible changes please leave a note in [UPDATING.md](https://github.com/apache/airflow/blob/master/UPDATING.md). Read the [Pull Request Guidelines](https://github.com/apache/airflow/blob/master/CONTRIBUTING.rst#pull-request-guidelines) for more information. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Created] (AIRFLOW-6839) Even more mypy speedup
Jarek Potiuk created AIRFLOW-6839: - Summary: Even more mypy speedup Key: AIRFLOW-6839 URL: https://issues.apache.org/jira/browse/AIRFLOW-6839 Project: Apache Airflow Issue Type: Improvement Components: pre-commit Affects Versions: 1.10.9, 2.0.0 Reporter: Jarek Potiuk require_serial:true is better choice than pass_filename: false as it can speed-up mypy for single file changes. Significant gains can be achieved for single file changes and no cache for all other files. This is majority of cases for our users who have pre-commits installed as hooks because most people change only few files and never run check with --all-files When just one file is changed and no cache is built, the difference is drastic: require_serial: true (4s) {code:java} Run mypy.PassedRun mypy.Passed real 0m4,080s user 0m0,283s sys 0m0,095s {code} pass_filenames: false: (13s) {code:java} time pre-commit run mypy time pre-commit run mypyRun mypy.Passed real 0m13,756suser 0m0,281ssys 0m0,099s{code} -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (AIRFLOW-6818) Prevent Docker cache-busting on when editing www templates
[ https://issues.apache.org/jira/browse/AIRFLOW-6818?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jarek Potiuk updated AIRFLOW-6818: -- Affects Version/s: (was: 2.0.0) 1.10.10 > Prevent Docker cache-busting on when editing www templates > -- > > Key: AIRFLOW-6818 > URL: https://issues.apache.org/jira/browse/AIRFLOW-6818 > Project: Apache Airflow > Issue Type: Improvement > Components: ci >Affects Versions: 1.10.10 >Reporter: Ash Berlin-Taylor >Assignee: Ash Berlin-Taylor >Priority: Minor > Fix For: 1.10.10 > > > Due to permission differences on files/folders between Travis and docker hub > auto build we are needlessly busting the docker layer cache -> builds are a > bit slower than they need to be -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (AIRFLOW-6823) Breeze broken on current master on Linux
[ https://issues.apache.org/jira/browse/AIRFLOW-6823?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jarek Potiuk updated AIRFLOW-6823: -- Affects Version/s: (was: 2.0.0) > Breeze broken on current master on Linux > > > Key: AIRFLOW-6823 > URL: https://issues.apache.org/jira/browse/AIRFLOW-6823 > Project: Apache Airflow > Issue Type: Bug > Components: breeze >Affects Versions: 1.10.10 >Reporter: Aleksander Nitecki >Assignee: Ash Berlin-Taylor >Priority: Minor > Fix For: 2.0.0 > > Attachments: Breeze on current master log.txt > > > See the attachment for log. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (AIRFLOW-6820) Split breeze into functions
[ https://issues.apache.org/jira/browse/AIRFLOW-6820?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jarek Potiuk updated AIRFLOW-6820: -- Affects Version/s: (was: 1.10.9) 1.10.10 > Split breeze into functions > --- > > Key: AIRFLOW-6820 > URL: https://issues.apache.org/jira/browse/AIRFLOW-6820 > Project: Apache Airflow > Issue Type: Improvement > Components: breeze >Affects Versions: 2.0.0, 1.10.10 >Reporter: Jarek Potiuk >Priority: Major > Fix For: 2.0.0 > > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (AIRFLOW-6820) Split breeze into functions
[ https://issues.apache.org/jira/browse/AIRFLOW-6820?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jarek Potiuk updated AIRFLOW-6820: -- Affects Version/s: (was: 2.0.0) > Split breeze into functions > --- > > Key: AIRFLOW-6820 > URL: https://issues.apache.org/jira/browse/AIRFLOW-6820 > Project: Apache Airflow > Issue Type: Improvement > Components: breeze >Affects Versions: 1.10.10 >Reporter: Jarek Potiuk >Priority: Major > Fix For: 2.0.0 > > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Resolved] (AIRFLOW-6816) Separate breeze's variable setup into functions
[ https://issues.apache.org/jira/browse/AIRFLOW-6816?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jarek Potiuk resolved AIRFLOW-6816. --- Fix Version/s: 1.10.10 Resolution: Fixed > Separate breeze's variable setup into functions > --- > > Key: AIRFLOW-6816 > URL: https://issues.apache.org/jira/browse/AIRFLOW-6816 > Project: Apache Airflow > Issue Type: Improvement > Components: breeze >Affects Versions: 2.0.0, 1.10.9 >Reporter: Jarek Potiuk >Assignee: Jarek Potiuk >Priority: Major > Fix For: 1.10.10 > > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (AIRFLOW-6823) Breeze broken on current master on Linux
[ https://issues.apache.org/jira/browse/AIRFLOW-6823?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17039637#comment-17039637 ] ASF subversion and git services commented on AIRFLOW-6823: -- Commit dbe0bbde74ebadb271b2f80dc0508090b69d88d1 in airflow's branch refs/heads/v1-10-test from Jarek Potiuk [ https://gitbox.apache.org/repos/asf?p=airflow.git;h=dbe0bbd ] [AIRFLOW-6823] Fix root owned files automatically in Breeze (#7449) (cherry picked from commit 03fb50aaa4bd7b00968c300781039cebf91043f2) > Breeze broken on current master on Linux > > > Key: AIRFLOW-6823 > URL: https://issues.apache.org/jira/browse/AIRFLOW-6823 > Project: Apache Airflow > Issue Type: Bug > Components: breeze >Affects Versions: 2.0.0 >Reporter: Aleksander Nitecki >Assignee: Ash Berlin-Taylor >Priority: Minor > Fix For: 2.0.0 > > Attachments: Breeze on current master log.txt > > > See the attachment for log. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (AIRFLOW-6823) Breeze broken on current master on Linux
[ https://issues.apache.org/jira/browse/AIRFLOW-6823?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jarek Potiuk updated AIRFLOW-6823: -- Affects Version/s: 1.10.10 > Breeze broken on current master on Linux > > > Key: AIRFLOW-6823 > URL: https://issues.apache.org/jira/browse/AIRFLOW-6823 > Project: Apache Airflow > Issue Type: Bug > Components: breeze >Affects Versions: 2.0.0, 1.10.10 >Reporter: Aleksander Nitecki >Assignee: Ash Berlin-Taylor >Priority: Minor > Fix For: 2.0.0 > > Attachments: Breeze on current master log.txt > > > See the attachment for log. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (AIRFLOW-6820) Split breeze into functions
[ https://issues.apache.org/jira/browse/AIRFLOW-6820?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17039639#comment-17039639 ] ASF subversion and git services commented on AIRFLOW-6820: -- Commit 453e65abcf9250685bf702b4018554e189d05ca9 in airflow's branch refs/heads/v1-10-test from Jarek Potiuk [ https://gitbox.apache.org/repos/asf?p=airflow.git;h=453e65a ] [AIRFLOW-6820] fixup! fix unbound variable (cherry picked from commit a54cf8b33030a52c45e46cf7f507f496ee00d610) > Split breeze into functions > --- > > Key: AIRFLOW-6820 > URL: https://issues.apache.org/jira/browse/AIRFLOW-6820 > Project: Apache Airflow > Issue Type: Improvement > Components: breeze >Affects Versions: 2.0.0, 1.10.9 >Reporter: Jarek Potiuk >Priority: Major > Fix For: 2.0.0 > > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (AIRFLOW-6818) Prevent Docker cache-busting on when editing www templates
[ https://issues.apache.org/jira/browse/AIRFLOW-6818?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17039635#comment-17039635 ] ASF subversion and git services commented on AIRFLOW-6818: -- Commit ee0ea017367cedc6bf2b9f17991e50d74cb78f38 in airflow's branch refs/heads/v1-10-test from Ash Berlin-Taylor [ https://gitbox.apache.org/repos/asf?p=airflow.git;h=ee0ea01 ] [AIRFLOW-6818] Prevent Docker cache-busting on when editing www templates (#7432) There is two parts to this PR: 1. Only copying www/webpack.config.js and www/static/ before running the asset pipeline 2. Making sure that _all_ files (not just the critical ones) have the same permissions. (cherry picked from commit cec9249f904bd91be9cb5a6be7e04979585da12e) > Prevent Docker cache-busting on when editing www templates > -- > > Key: AIRFLOW-6818 > URL: https://issues.apache.org/jira/browse/AIRFLOW-6818 > Project: Apache Airflow > Issue Type: Improvement > Components: ci >Affects Versions: 2.0.0 >Reporter: Ash Berlin-Taylor >Assignee: Ash Berlin-Taylor >Priority: Minor > Fix For: 1.10.10 > > > Due to permission differences on files/folders between Travis and docker hub > auto build we are needlessly busting the docker layer cache -> builds are a > bit slower than they need to be -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (AIRFLOW-6820) Split breeze into functions
[ https://issues.apache.org/jira/browse/AIRFLOW-6820?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17039640#comment-17039640 ] ASF GitHub Bot commented on AIRFLOW-6820: - potiuk commented on pull request #7459: [AIRFLOW-6820] fixup! fix unbound variable URL: https://github.com/apache/airflow/pull/7459 --- Issue link: WILL BE INSERTED BY [boring-cyborg](https://github.com/kaxil/boring-cyborg) Make sure to mark the boxes below before creating PR: [x] - [x] Description above provides context of the change - [x] Commit message/PR title starts with `[AIRFLOW-]`. AIRFLOW- = JIRA ID* - [x] Unit tests coverage for changes (not needed for documentation changes) - [x] Commits follow "[How to write a good git commit message](http://chris.beams.io/posts/git-commit/)" - [x] Relevant documentation is updated including usage instructions. - [x] I will engage committers as explained in [Contribution Workflow Example](https://github.com/apache/airflow/blob/master/CONTRIBUTING.rst#contribution-workflow-example). * For document-only changes commit message can start with `[AIRFLOW-]`. --- In case of fundamental code change, Airflow Improvement Proposal ([AIP](https://cwiki.apache.org/confluence/display/AIRFLOW/Airflow+Improvements+Proposals)) is needed. In case of a new dependency, check compliance with the [ASF 3rd Party License Policy](https://www.apache.org/legal/resolved.html#category-x). In case of backwards incompatible changes please leave a note in [UPDATING.md](https://github.com/apache/airflow/blob/master/UPDATING.md). Read the [Pull Request Guidelines](https://github.com/apache/airflow/blob/master/CONTRIBUTING.rst#pull-request-guidelines) for more information. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org > Split breeze into functions > --- > > Key: AIRFLOW-6820 > URL: https://issues.apache.org/jira/browse/AIRFLOW-6820 > Project: Apache Airflow > Issue Type: Improvement > Components: breeze >Affects Versions: 2.0.0, 1.10.9 >Reporter: Jarek Potiuk >Priority: Major > Fix For: 2.0.0 > > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (AIRFLOW-6820) Split breeze into functions
[ https://issues.apache.org/jira/browse/AIRFLOW-6820?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17039638#comment-17039638 ] ASF subversion and git services commented on AIRFLOW-6820: -- Commit ac45146dd21e64f5677c757dd4cf70ca834bbcc4 in airflow's branch refs/heads/v1-10-test from Jarek Potiuk [ https://gitbox.apache.org/repos/asf?p=airflow.git;h=ac45146 ] [AIRFLOW-6820] split breeze into functions (#7433) (cherry picked from commit 24f9f11775bf7b38cced9955f6c70705ad56f61c) > Split breeze into functions > --- > > Key: AIRFLOW-6820 > URL: https://issues.apache.org/jira/browse/AIRFLOW-6820 > Project: Apache Airflow > Issue Type: Improvement > Components: breeze >Affects Versions: 2.0.0, 1.10.9 >Reporter: Jarek Potiuk >Priority: Major > Fix For: 2.0.0 > > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[GitHub] [airflow] potiuk commented on issue #7459: [AIRFLOW-6820] fixup! fix unbound variable
potiuk commented on issue #7459: [AIRFLOW-6820] fixup! fix unbound variable URL: https://github.com/apache/airflow/pull/7459#issuecomment-587996718 Quick fix for unbound variable This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Commented] (AIRFLOW-6816) Separate breeze's variable setup into functions
[ https://issues.apache.org/jira/browse/AIRFLOW-6816?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17039636#comment-17039636 ] ASF subversion and git services commented on AIRFLOW-6816: -- Commit f6eafcefe9c61ef1dfdf29d0de1050feb3ed8fe4 in airflow's branch refs/heads/v1-10-test from Jarek Potiuk [ https://gitbox.apache.org/repos/asf?p=airflow.git;h=f6eafce ] [AIRFLOW-6816] Simplified common functions in breeze scripts (#7430) (cherry picked from commit ff7034f39ae4919ee8efbc8ef1c0feada1107d6a) > Separate breeze's variable setup into functions > --- > > Key: AIRFLOW-6816 > URL: https://issues.apache.org/jira/browse/AIRFLOW-6816 > Project: Apache Airflow > Issue Type: Improvement > Components: breeze >Affects Versions: 2.0.0, 1.10.9 >Reporter: Jarek Potiuk >Assignee: Jarek Potiuk >Priority: Major > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[GitHub] [airflow] potiuk opened a new pull request #7459: [AIRFLOW-6820] fixup! fix unbound variable
potiuk opened a new pull request #7459: [AIRFLOW-6820] fixup! fix unbound variable URL: https://github.com/apache/airflow/pull/7459 --- Issue link: WILL BE INSERTED BY [boring-cyborg](https://github.com/kaxil/boring-cyborg) Make sure to mark the boxes below before creating PR: [x] - [x] Description above provides context of the change - [x] Commit message/PR title starts with `[AIRFLOW-]`. AIRFLOW- = JIRA ID* - [x] Unit tests coverage for changes (not needed for documentation changes) - [x] Commits follow "[How to write a good git commit message](http://chris.beams.io/posts/git-commit/)" - [x] Relevant documentation is updated including usage instructions. - [x] I will engage committers as explained in [Contribution Workflow Example](https://github.com/apache/airflow/blob/master/CONTRIBUTING.rst#contribution-workflow-example). * For document-only changes commit message can start with `[AIRFLOW-]`. --- In case of fundamental code change, Airflow Improvement Proposal ([AIP](https://cwiki.apache.org/confluence/display/AIRFLOW/Airflow+Improvements+Proposals)) is needed. In case of a new dependency, check compliance with the [ASF 3rd Party License Policy](https://www.apache.org/legal/resolved.html#category-x). In case of backwards incompatible changes please leave a note in [UPDATING.md](https://github.com/apache/airflow/blob/master/UPDATING.md). Read the [Pull Request Guidelines](https://github.com/apache/airflow/blob/master/CONTRIBUTING.rst#pull-request-guidelines) for more information. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [airflow] mik-laj commented on issue #7458: [AIRFLOW-6838][WIP] Introduce real subcommands for Breeze
mik-laj commented on issue #7458: [AIRFLOW-6838][WIP] Introduce real subcommands for Breeze URL: https://github.com/apache/airflow/pull/7458#issuecomment-587988274 > also parsing options won't work either. Can you say something more about it? I checked, I did found any issues https://user-images.githubusercontent.com/12058428/74792980-3e8ed280-52bf-11ea-8b55-1745beae606d.png";> This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Resolved] (AIRFLOW-6820) Split breeze into functions
[ https://issues.apache.org/jira/browse/AIRFLOW-6820?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jarek Potiuk resolved AIRFLOW-6820. --- Fix Version/s: 2.0.0 Resolution: Fixed > Split breeze into functions > --- > > Key: AIRFLOW-6820 > URL: https://issues.apache.org/jira/browse/AIRFLOW-6820 > Project: Apache Airflow > Issue Type: Improvement > Components: breeze >Affects Versions: 2.0.0, 1.10.9 >Reporter: Jarek Potiuk >Priority: Major > Fix For: 2.0.0 > > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (AIRFLOW-6820) Split breeze into functions
[ https://issues.apache.org/jira/browse/AIRFLOW-6820?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17039601#comment-17039601 ] ASF subversion and git services commented on AIRFLOW-6820: -- Commit 24f9f11775bf7b38cced9955f6c70705ad56f61c in airflow's branch refs/heads/master from Jarek Potiuk [ https://gitbox.apache.org/repos/asf?p=airflow.git;h=24f9f11 ] [AIRFLOW-6820] split breeze into functions (#7433) > Split breeze into functions > --- > > Key: AIRFLOW-6820 > URL: https://issues.apache.org/jira/browse/AIRFLOW-6820 > Project: Apache Airflow > Issue Type: Improvement > Components: breeze >Affects Versions: 2.0.0, 1.10.9 >Reporter: Jarek Potiuk >Priority: Major > Fix For: 2.0.0 > > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (AIRFLOW-6820) Split breeze into functions
[ https://issues.apache.org/jira/browse/AIRFLOW-6820?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17039600#comment-17039600 ] ASF GitHub Bot commented on AIRFLOW-6820: - potiuk commented on pull request #7433: [AIRFLOW-6820] split breeze into functions URL: https://github.com/apache/airflow/pull/7433 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org > Split breeze into functions > --- > > Key: AIRFLOW-6820 > URL: https://issues.apache.org/jira/browse/AIRFLOW-6820 > Project: Apache Airflow > Issue Type: Improvement > Components: breeze >Affects Versions: 2.0.0, 1.10.9 >Reporter: Jarek Potiuk >Priority: Major > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[GitHub] [airflow] potiuk merged pull request #7433: [AIRFLOW-6820] split breeze into functions
potiuk merged pull request #7433: [AIRFLOW-6820] split breeze into functions URL: https://github.com/apache/airflow/pull/7433 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [airflow] codecov-io edited a comment on issue #7433: [AIRFLOW-6820] split breeze into functions
codecov-io edited a comment on issue #7433: [AIRFLOW-6820] split breeze into functions URL: https://github.com/apache/airflow/pull/7433#issuecomment-586737012 # [Codecov](https://codecov.io/gh/apache/airflow/pull/7433?src=pr&el=h1) Report > Merging [#7433](https://codecov.io/gh/apache/airflow/pull/7433?src=pr&el=desc) into [master](https://codecov.io/gh/apache/airflow/commit/9cbd7de6d115795aba8bfb8addb060bfdfbdf87b?src=pr&el=desc) will **increase** coverage by `0.25%`. > The diff coverage is `n/a`. [![Impacted file tree graph](https://codecov.io/gh/apache/airflow/pull/7433/graphs/tree.svg?width=650&token=WdLKlKHOAU&height=150&src=pr)](https://codecov.io/gh/apache/airflow/pull/7433?src=pr&el=tree) ```diff @@Coverage Diff @@ ## master#7433 +/- ## == + Coverage 86.37% 86.62% +0.25% == Files 877 878 +1 Lines 4122141292 +71 == + Hits3560335770 +167 + Misses 5618 5522 -96 ``` | [Impacted Files](https://codecov.io/gh/apache/airflow/pull/7433?src=pr&el=tree) | Coverage Δ | | |---|---|---| | [airflow/configuration.py](https://codecov.io/gh/apache/airflow/pull/7433/diff?src=pr&el=tree#diff-YWlyZmxvdy9jb25maWd1cmF0aW9uLnB5) | `87.78% <0%> (-0.23%)` | :arrow_down: | | [...viders/docker/example\_dags/example\_docker\_swarm.py](https://codecov.io/gh/apache/airflow/pull/7433/diff?src=pr&el=tree#diff-YWlyZmxvdy9wcm92aWRlcnMvZG9ja2VyL2V4YW1wbGVfZGFncy9leGFtcGxlX2RvY2tlcl9zd2FybS5weQ==) | `100% <0%> (ø)` | :arrow_up: | | [airflow/config\_templates/default\_celery.py](https://codecov.io/gh/apache/airflow/pull/7433/diff?src=pr&el=tree#diff-YWlyZmxvdy9jb25maWdfdGVtcGxhdGVzL2RlZmF1bHRfY2VsZXJ5LnB5) | `52.94% <0%> (ø)` | :arrow_up: | | [...ow/providers/docker/example\_dags/example\_docker.py](https://codecov.io/gh/apache/airflow/pull/7433/diff?src=pr&el=tree#diff-YWlyZmxvdy9wcm92aWRlcnMvZG9ja2VyL2V4YW1wbGVfZGFncy9leGFtcGxlX2RvY2tlci5weQ==) | `100% <0%> (ø)` | :arrow_up: | | [airflow/models/connection.py](https://codecov.io/gh/apache/airflow/pull/7433/diff?src=pr&el=tree#diff-YWlyZmxvdy9tb2RlbHMvY29ubmVjdGlvbi5weQ==) | `95.07% <0%> (ø)` | :arrow_up: | | [...low/providers/elasticsearch/hooks/elasticsearch.py](https://codecov.io/gh/apache/airflow/pull/7433/diff?src=pr&el=tree#diff-YWlyZmxvdy9wcm92aWRlcnMvZWxhc3RpY3NlYXJjaC9ob29rcy9lbGFzdGljc2VhcmNoLnB5) | `46.15% <0%> (ø)` | | | [airflow/utils/db.py](https://codecov.io/gh/apache/airflow/pull/7433/diff?src=pr&el=tree#diff-YWlyZmxvdy91dGlscy9kYi5weQ==) | `98.29% <0%> (+0.01%)` | :arrow_up: | | [airflow/kubernetes/refresh\_config.py](https://codecov.io/gh/apache/airflow/pull/7433/diff?src=pr&el=tree#diff-YWlyZmxvdy9rdWJlcm5ldGVzL3JlZnJlc2hfY29uZmlnLnB5) | `74.5% <0%> (+23.52%)` | :arrow_up: | | [...viders/cncf/kubernetes/operators/kubernetes\_pod.py](https://codecov.io/gh/apache/airflow/pull/7433/diff?src=pr&el=tree#diff-YWlyZmxvdy9wcm92aWRlcnMvY25jZi9rdWJlcm5ldGVzL29wZXJhdG9ycy9rdWJlcm5ldGVzX3BvZC5weQ==) | `94.89% <0%> (+25.51%)` | :arrow_up: | | [airflow/kubernetes/pod\_launcher.py](https://codecov.io/gh/apache/airflow/pull/7433/diff?src=pr&el=tree#diff-YWlyZmxvdy9rdWJlcm5ldGVzL3BvZF9sYXVuY2hlci5weQ==) | `92.25% <0%> (+45.07%)` | :arrow_up: | | ... and [2 more](https://codecov.io/gh/apache/airflow/pull/7433/diff?src=pr&el=tree-more) | | -- [Continue to review full report at Codecov](https://codecov.io/gh/apache/airflow/pull/7433?src=pr&el=continue). > **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta) > `Δ = absolute (impact)`, `ø = not affected`, `? = missing data` > Powered by [Codecov](https://codecov.io/gh/apache/airflow/pull/7433?src=pr&el=footer). Last update [9cbd7de...c892ed2](https://codecov.io/gh/apache/airflow/pull/7433?src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments). This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [airflow] potiuk commented on a change in pull request #7433: [AIRFLOW-6820] split breeze into functions
potiuk commented on a change in pull request #7433: [AIRFLOW-6820] split breeze into functions URL: https://github.com/apache/airflow/pull/7433#discussion_r381020777 ## File path: hooks/build ## @@ -401,34 +372,30 @@ git ls-files -z -- ./ ':!:airflow/www/static/docs' | filterout_deleted_files | x # and this deals with directories Review comment: done This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [airflow] potiuk commented on a change in pull request #7458: [AIRFLOW-6838][WIP] Introduce real subcommands for Breeze
potiuk commented on a change in pull request #7458: [AIRFLOW-6838][WIP] Introduce real subcommands for Breeze URL: https://github.com/apache/airflow/pull/7458#discussion_r381020442 ## File path: breeze ## @@ -290,7 +290,7 @@ Usage: ${CMDNAME} [COMMAND] [FLAGS] -- The swiss-knife-army tool for Airflow testings. It allows to perform various test tasks: - * Enter interactive environment when no command flags are specified (default behaviour) + * Run interactive terminal when no command flags are specified (default behaviour) Review comment: IT's not terminal. It's the whole environment using integrations, runtime, kubernetes, databases etc. I still think environment is better name, This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [airflow] potiuk commented on issue #7458: [AIRFLOW-6838][WIP] Introduce real subcommands for Breeze
potiuk commented on issue #7458: [AIRFLOW-6838][WIP] Introduce real subcommands for Breeze URL: https://github.com/apache/airflow/pull/7458#issuecomment-587978552 While I like the spirit of that change,. Breeze grew a lot in terms of functionality and maybe we should think of refactoring the structure of commands soon. But I think it's not the right way of doing it. First of all you are just about to have a lot of conflicts as we are splitting breeze into functions. Secondly it does not take into account getopt's behaviour so it is for sure wrong (We define all the available options in breeze-complete and getopt by default assumes -- or - params. So not only autocomplete will not work but also parsing options won't work either. I think we need to do much bolder change if we want to introduce sub-commands - we should rewrite it in another language (python or golang). But complicating parsing of parameters is not a good idea. Unless you use ready to use library - for example here http://people.tuebingen.mpg.de/maan/gsu/ then we could write it in bash (but it's not a good idea as it is GPL2 and it does not support ZSH I believe). I think we should talk about it and think if that's the right time to think of such change or maybe better to leave it for after we deliver the current important 2.0 features - prod image, API and others. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [airflow] potiuk commented on issue #7433: [AIRFLOW-6820] split breeze into functions
potiuk commented on issue #7433: [AIRFLOW-6820] split breeze into functions URL: https://github.com/apache/airflow/pull/7433#issuecomment-587970268 > Looks good other than one small tidy up. Done This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Commented] (AIRFLOW-6838) Introduce real subcommands for Breeze
[ https://issues.apache.org/jira/browse/AIRFLOW-6838?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17039549#comment-17039549 ] ASF GitHub Bot commented on AIRFLOW-6838: - mik-laj commented on pull request #7458: [AIRFLOW-6838] Introduce real subcommands for Breeze URL: https://github.com/apache/airflow/pull/7458 Breeze currently selects action using many different flags. I know from my own experience that this is very confusing. I think that subcommands will be much easier to use. I suspect that Breeze autocomplete is not working properly, but I have no experience with this component. Could someone help me with this? In the next step, I think that we should replace global variables with the function, because a large number of global variables makes it very difficult to trace the aapplication behavior. --- Issue link: WILL BE INSERTED BY [boring-cyborg](https://github.com/kaxil/boring-cyborg) Make sure to mark the boxes below before creating PR: [x] - [ ] Description above provides context of the change - [ ] Commit message/PR title starts with `[AIRFLOW-]`. AIRFLOW- = JIRA ID* - [ ] Unit tests coverage for changes (not needed for documentation changes) - [ ] Commits follow "[How to write a good git commit message](http://chris.beams.io/posts/git-commit/)" - [ ] Relevant documentation is updated including usage instructions. - [ ] I will engage committers as explained in [Contribution Workflow Example](https://github.com/apache/airflow/blob/master/CONTRIBUTING.rst#contribution-workflow-example). * For document-only changes commit message can start with `[AIRFLOW-]`. --- In case of fundamental code change, Airflow Improvement Proposal ([AIP](https://cwiki.apache.org/confluence/display/AIRFLOW/Airflow+Improvements+Proposals)) is needed. In case of a new dependency, check compliance with the [ASF 3rd Party License Policy](https://www.apache.org/legal/resolved.html#category-x). In case of backwards incompatible changes please leave a note in [UPDATING.md](https://github.com/apache/airflow/blob/master/UPDATING.md). Read the [Pull Request Guidelines](https://github.com/apache/airflow/blob/master/CONTRIBUTING.rst#pull-request-guidelines) for more information. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org > Introduce real subcommands for Breeze > - > > Key: AIRFLOW-6838 > URL: https://issues.apache.org/jira/browse/AIRFLOW-6838 > Project: Apache Airflow > Issue Type: Bug > Components: core >Affects Versions: 1.10.9 >Reporter: Kamil Bregula >Priority: Major > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[GitHub] [airflow] mik-laj opened a new pull request #7458: [AIRFLOW-6838] Introduce real subcommands for Breeze
mik-laj opened a new pull request #7458: [AIRFLOW-6838] Introduce real subcommands for Breeze URL: https://github.com/apache/airflow/pull/7458 Breeze currently selects action using many different flags. I know from my own experience that this is very confusing. I think that subcommands will be much easier to use. I suspect that Breeze autocomplete is not working properly, but I have no experience with this component. Could someone help me with this? In the next step, I think that we should replace global variables with the function, because a large number of global variables makes it very difficult to trace the aapplication behavior. --- Issue link: WILL BE INSERTED BY [boring-cyborg](https://github.com/kaxil/boring-cyborg) Make sure to mark the boxes below before creating PR: [x] - [ ] Description above provides context of the change - [ ] Commit message/PR title starts with `[AIRFLOW-]`. AIRFLOW- = JIRA ID* - [ ] Unit tests coverage for changes (not needed for documentation changes) - [ ] Commits follow "[How to write a good git commit message](http://chris.beams.io/posts/git-commit/)" - [ ] Relevant documentation is updated including usage instructions. - [ ] I will engage committers as explained in [Contribution Workflow Example](https://github.com/apache/airflow/blob/master/CONTRIBUTING.rst#contribution-workflow-example). * For document-only changes commit message can start with `[AIRFLOW-]`. --- In case of fundamental code change, Airflow Improvement Proposal ([AIP](https://cwiki.apache.org/confluence/display/AIRFLOW/Airflow+Improvements+Proposals)) is needed. In case of a new dependency, check compliance with the [ASF 3rd Party License Policy](https://www.apache.org/legal/resolved.html#category-x). In case of backwards incompatible changes please leave a note in [UPDATING.md](https://github.com/apache/airflow/blob/master/UPDATING.md). Read the [Pull Request Guidelines](https://github.com/apache/airflow/blob/master/CONTRIBUTING.rst#pull-request-guidelines) for more information. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [airflow] ashb commented on a change in pull request #7433: [AIRFLOW-6820] split breeze into functions
ashb commented on a change in pull request #7433: [AIRFLOW-6820] split breeze into functions URL: https://github.com/apache/airflow/pull/7433#discussion_r380991455 ## File path: hooks/build ## @@ -401,34 +372,30 @@ git ls-files -z -- ./ ':!:airflow/www/static/docs' | filterout_deleted_files | x # and this deals with directories Review comment: Can you remove the exclusion I added on the previous `git ls-files` lines now you've removed the symlink please? This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Assigned] (AIRFLOW-6837) limit description of dag in home page
[ https://issues.apache.org/jira/browse/AIRFLOW-6837?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ping Zhang reassigned AIRFLOW-6837: --- Assignee: Ping Zhang > limit description of dag in home page > - > > Key: AIRFLOW-6837 > URL: https://issues.apache.org/jira/browse/AIRFLOW-6837 > Project: Apache Airflow > Issue Type: Improvement > Components: webserver >Affects Versions: 1.10.9 >Reporter: Ping Zhang >Assignee: Ping Zhang >Priority: Major > > so that the tooltip will not cover the dag link -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (AIRFLOW-6838) Introduce real subcommands for Breeze
[ https://issues.apache.org/jira/browse/AIRFLOW-6838?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kamil Bregula updated AIRFLOW-6838: --- Summary: Introduce real subcommands for Breeze (was: Introdocue real subcommands for Breeze) > Introduce real subcommands for Breeze > - > > Key: AIRFLOW-6838 > URL: https://issues.apache.org/jira/browse/AIRFLOW-6838 > Project: Apache Airflow > Issue Type: Bug > Components: core >Affects Versions: 1.10.9 >Reporter: Kamil Bregula >Priority: Major > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (AIRFLOW-6837) limit description of dag in home page
[ https://issues.apache.org/jira/browse/AIRFLOW-6837?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17039542#comment-17039542 ] ASF GitHub Bot commented on AIRFLOW-6837: - pingzh commented on pull request #7457: [AIRFLOW-6837] limit description len of dag in home URL: https://github.com/apache/airflow/pull/7457 as it makes dags link not clickable The dag with long description: ![image](https://user-images.githubusercontent.com/8662365/74785686-bb8a5f80-525f-11ea-8463-af10b3c1ad40.png) Before: ![image](https://user-images.githubusercontent.com/8662365/74785702-c6dd8b00-525f-11ea-94af-9710da21baab.png) After: ![image](https://user-images.githubusercontent.com/8662365/74785710-ca711200-525f-11ea-9e12-6a77ebc9bcad.png) --- Issue link: WILL BE INSERTED BY [boring-cyborg](https://github.com/kaxil/boring-cyborg) Make sure to mark the boxes below before creating PR: [x] - [x] Description above provides context of the change - [x] Commit message/PR title starts with `[AIRFLOW-]`. AIRFLOW- = JIRA ID* - [x] Unit tests coverage for changes (not needed for documentation changes) - [x] Commits follow "[How to write a good git commit message](http://chris.beams.io/posts/git-commit/)" - [x] Relevant documentation is updated including usage instructions. - [x] I will engage committers as explained in [Contribution Workflow Example](https://github.com/apache/airflow/blob/master/CONTRIBUTING.rst#contribution-workflow-example). * For document-only changes commit message can start with `[AIRFLOW-]`. --- In case of fundamental code change, Airflow Improvement Proposal ([AIP](https://cwiki.apache.org/confluence/display/AIRFLOW/Airflow+Improvements+Proposals)) is needed. In case of a new dependency, check compliance with the [ASF 3rd Party License Policy](https://www.apache.org/legal/resolved.html#category-x). In case of backwards incompatible changes please leave a note in [UPDATING.md](https://github.com/apache/airflow/blob/master/UPDATING.md). Read the [Pull Request Guidelines](https://github.com/apache/airflow/blob/master/CONTRIBUTING.rst#pull-request-guidelines) for more information. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org > limit description of dag in home page > - > > Key: AIRFLOW-6837 > URL: https://issues.apache.org/jira/browse/AIRFLOW-6837 > Project: Apache Airflow > Issue Type: Improvement > Components: webserver >Affects Versions: 1.10.9 >Reporter: Ping Zhang >Priority: Major > > so that the tooltip will not cover the dag link -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (AIRFLOW-6838) Introdocue real subcommands for Breeze
Kamil Bregula created AIRFLOW-6838: -- Summary: Introdocue real subcommands for Breeze Key: AIRFLOW-6838 URL: https://issues.apache.org/jira/browse/AIRFLOW-6838 Project: Apache Airflow Issue Type: Bug Components: core Affects Versions: 1.10.9 Reporter: Kamil Bregula -- This message was sent by Atlassian Jira (v8.3.4#803005)
[GitHub] [airflow] pingzh opened a new pull request #7457: [AIRFLOW-6837] limit description len of dag in home
pingzh opened a new pull request #7457: [AIRFLOW-6837] limit description len of dag in home URL: https://github.com/apache/airflow/pull/7457 as it makes dags link not clickable The dag with long description: ![image](https://user-images.githubusercontent.com/8662365/74785686-bb8a5f80-525f-11ea-8463-af10b3c1ad40.png) Before: ![image](https://user-images.githubusercontent.com/8662365/74785702-c6dd8b00-525f-11ea-94af-9710da21baab.png) After: ![image](https://user-images.githubusercontent.com/8662365/74785710-ca711200-525f-11ea-9e12-6a77ebc9bcad.png) --- Issue link: WILL BE INSERTED BY [boring-cyborg](https://github.com/kaxil/boring-cyborg) Make sure to mark the boxes below before creating PR: [x] - [x] Description above provides context of the change - [x] Commit message/PR title starts with `[AIRFLOW-]`. AIRFLOW- = JIRA ID* - [x] Unit tests coverage for changes (not needed for documentation changes) - [x] Commits follow "[How to write a good git commit message](http://chris.beams.io/posts/git-commit/)" - [x] Relevant documentation is updated including usage instructions. - [x] I will engage committers as explained in [Contribution Workflow Example](https://github.com/apache/airflow/blob/master/CONTRIBUTING.rst#contribution-workflow-example). * For document-only changes commit message can start with `[AIRFLOW-]`. --- In case of fundamental code change, Airflow Improvement Proposal ([AIP](https://cwiki.apache.org/confluence/display/AIRFLOW/Airflow+Improvements+Proposals)) is needed. In case of a new dependency, check compliance with the [ASF 3rd Party License Policy](https://www.apache.org/legal/resolved.html#category-x). In case of backwards incompatible changes please leave a note in [UPDATING.md](https://github.com/apache/airflow/blob/master/UPDATING.md). Read the [Pull Request Guidelines](https://github.com/apache/airflow/blob/master/CONTRIBUTING.rst#pull-request-guidelines) for more information. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Created] (AIRFLOW-6837) limit description of dag in home page
Ping Zhang created AIRFLOW-6837: --- Summary: limit description of dag in home page Key: AIRFLOW-6837 URL: https://issues.apache.org/jira/browse/AIRFLOW-6837 Project: Apache Airflow Issue Type: Improvement Components: webserver Affects Versions: 1.10.9 Reporter: Ping Zhang so that the tooltip will not cover the dag link -- This message was sent by Atlassian Jira (v8.3.4#803005)
[GitHub] [airflow] msb217 opened a new pull request #7456: [AIRFLOW 6817] - Move airflow __init__.py imports to sub-packages
msb217 opened a new pull request #7456: [AIRFLOW 6817] - Move airflow __init__.py imports to sub-packages URL: https://github.com/apache/airflow/pull/7456 --- Issue link: WILL BE INSERTED BY [boring-cyborg](https://github.com/kaxil/boring-cyborg) Moved `airflow/__init__.py` imports to respective sub-packages with the exception of the `settings.initialize()` and `integrate_plugins()` All breeze unit and integration tests are passing locally. Make sure to mark the boxes below before creating PR: [x] - [X] Description above provides context of the change - [X] Commit message/PR title starts with `[AIRFLOW-]`. AIRFLOW- = JIRA ID* - [X] Unit tests coverage for changes (not needed for documentation changes) - [X] Commits follow "[How to write a good git commit message](http://chris.beams.io/posts/git-commit/)" - [X] Relevant documentation is updated including usage instructions. - [X] I will engage committers as explained in [Contribution Workflow Example](https://github.com/apache/airflow/blob/master/CONTRIBUTING.rst#contribution-workflow-example). * For document-only changes commit message can start with `[AIRFLOW-]`. --- In case of fundamental code change, Airflow Improvement Proposal ([AIP](https://cwiki.apache.org/confluence/display/AIRFLOW/Airflow+Improvements+Proposals)) is needed. In case of a new dependency, check compliance with the [ASF 3rd Party License Policy](https://www.apache.org/legal/resolved.html#category-x). In case of backwards incompatible changes please leave a note in [UPDATING.md](https://github.com/apache/airflow/blob/master/UPDATING.md). Read the [Pull Request Guidelines](https://github.com/apache/airflow/blob/master/CONTRIBUTING.rst#pull-request-guidelines) for more information. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [airflow] codecov-io edited a comment on issue #6090: [AIRFLOW-5470] Add Apache Livy REST operator
codecov-io edited a comment on issue #6090: [AIRFLOW-5470] Add Apache Livy REST operator URL: https://github.com/apache/airflow/pull/6090#issuecomment-572009682 # [Codecov](https://codecov.io/gh/apache/airflow/pull/6090?src=pr&el=h1) Report > Merging [#6090](https://codecov.io/gh/apache/airflow/pull/6090?src=pr&el=desc) into [master](https://codecov.io/gh/apache/airflow/commit/7c95c8144d1262a5d7e9d99ca1cec30941064c24?src=pr&el=desc) will **decrease** coverage by `0.41%`. > The diff coverage is `98.6%`. [![Impacted file tree graph](https://codecov.io/gh/apache/airflow/pull/6090/graphs/tree.svg?width=650&token=WdLKlKHOAU&height=150&src=pr)](https://codecov.io/gh/apache/airflow/pull/6090?src=pr&el=tree) ```diff @@Coverage Diff @@ ## master#6090 +/- ## == - Coverage 86.66% 86.25% -0.42% == Files 877 882 +5 Lines 4125241507 +255 == + Hits3575135800 +49 - Misses 5501 5707 +206 ``` | [Impacted Files](https://codecov.io/gh/apache/airflow/pull/6090?src=pr&el=tree) | Coverage Δ | | |---|---|---| | [airflow/models/connection.py](https://codecov.io/gh/apache/airflow/pull/6090/diff?src=pr&el=tree#diff-YWlyZmxvdy9tb2RlbHMvY29ubmVjdGlvbi5weQ==) | `95.07% <ø> (ø)` | :arrow_up: | | [airflow/providers/apache/livy/sensors/livy.py](https://codecov.io/gh/apache/airflow/pull/6090/diff?src=pr&el=tree#diff-YWlyZmxvdy9wcm92aWRlcnMvYXBhY2hlL2xpdnkvc2Vuc29ycy9saXZ5LnB5) | `100% <100%> (ø)` | | | [...providers/apache/livy/example\_dags/example\_livy.py](https://codecov.io/gh/apache/airflow/pull/6090/diff?src=pr&el=tree#diff-YWlyZmxvdy9wcm92aWRlcnMvYXBhY2hlL2xpdnkvZXhhbXBsZV9kYWdzL2V4YW1wbGVfbGl2eS5weQ==) | `100% <100%> (ø)` | | | [airflow/utils/db.py](https://codecov.io/gh/apache/airflow/pull/6090/diff?src=pr&el=tree#diff-YWlyZmxvdy91dGlscy9kYi5weQ==) | `98.3% <100%> (+0.02%)` | :arrow_up: | | [airflow/providers/apache/livy/operators/livy.py](https://codecov.io/gh/apache/airflow/pull/6090/diff?src=pr&el=tree#diff-YWlyZmxvdy9wcm92aWRlcnMvYXBhY2hlL2xpdnkvb3BlcmF0b3JzL2xpdnkucHk=) | `97.5% <97.5%> (ø)` | | | [airflow/providers/apache/livy/hooks/livy.py](https://codecov.io/gh/apache/airflow/pull/6090/diff?src=pr&el=tree#diff-YWlyZmxvdy9wcm92aWRlcnMvYXBhY2hlL2xpdnkvaG9va3MvbGl2eS5weQ==) | `98.62% <98.62%> (ø)` | | | [airflow/kubernetes/volume\_mount.py](https://codecov.io/gh/apache/airflow/pull/6090/diff?src=pr&el=tree#diff-YWlyZmxvdy9rdWJlcm5ldGVzL3ZvbHVtZV9tb3VudC5weQ==) | `44.44% <0%> (-55.56%)` | :arrow_down: | | [airflow/kubernetes/volume.py](https://codecov.io/gh/apache/airflow/pull/6090/diff?src=pr&el=tree#diff-YWlyZmxvdy9rdWJlcm5ldGVzL3ZvbHVtZS5weQ==) | `52.94% <0%> (-47.06%)` | :arrow_down: | | [airflow/kubernetes/pod\_launcher.py](https://codecov.io/gh/apache/airflow/pull/6090/diff?src=pr&el=tree#diff-YWlyZmxvdy9rdWJlcm5ldGVzL3BvZF9sYXVuY2hlci5weQ==) | `47.18% <0%> (-45.08%)` | :arrow_down: | | [airflow/executors/sequential\_executor.py](https://codecov.io/gh/apache/airflow/pull/6090/diff?src=pr&el=tree#diff-YWlyZmxvdy9leGVjdXRvcnMvc2VxdWVudGlhbF9leGVjdXRvci5weQ==) | `56% <0%> (-44%)` | :arrow_down: | | ... and [12 more](https://codecov.io/gh/apache/airflow/pull/6090/diff?src=pr&el=tree-more) | | -- [Continue to review full report at Codecov](https://codecov.io/gh/apache/airflow/pull/6090?src=pr&el=continue). > **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta) > `Δ = absolute (impact)`, `ø = not affected`, `? = missing data` > Powered by [Codecov](https://codecov.io/gh/apache/airflow/pull/6090?src=pr&el=footer). Last update [7c95c81...2b0e5e2](https://codecov.io/gh/apache/airflow/pull/6090?src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments). This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [airflow] vsoch commented on issue #7191: [AIRFLOW-4030] second attempt to add singularity to airflow
vsoch commented on issue #7191: [AIRFLOW-4030] second attempt to add singularity to airflow URL: https://github.com/apache/airflow/pull/7191#issuecomment-587936273 Actually it looks like it wasn't happy with working_dir as a kwargs, I'll try adding it to the type definitions to see if that changes anything... This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [airflow] codecov-io edited a comment on issue #6090: [AIRFLOW-5470] Add Apache Livy REST operator
codecov-io edited a comment on issue #6090: [AIRFLOW-5470] Add Apache Livy REST operator URL: https://github.com/apache/airflow/pull/6090#issuecomment-572009682 # [Codecov](https://codecov.io/gh/apache/airflow/pull/6090?src=pr&el=h1) Report > Merging [#6090](https://codecov.io/gh/apache/airflow/pull/6090?src=pr&el=desc) into [master](https://codecov.io/gh/apache/airflow/commit/7c95c8144d1262a5d7e9d99ca1cec30941064c24?src=pr&el=desc) will **decrease** coverage by `0.6%`. > The diff coverage is `98.6%`. [![Impacted file tree graph](https://codecov.io/gh/apache/airflow/pull/6090/graphs/tree.svg?width=650&token=WdLKlKHOAU&height=150&src=pr)](https://codecov.io/gh/apache/airflow/pull/6090?src=pr&el=tree) ```diff @@Coverage Diff @@ ## master#6090 +/- ## == - Coverage 86.66% 86.05% -0.61% == Files 877 882 +5 Lines 4125241507 +255 == - Hits3575135720 -31 - Misses 5501 5787 +286 ``` | [Impacted Files](https://codecov.io/gh/apache/airflow/pull/6090?src=pr&el=tree) | Coverage Δ | | |---|---|---| | [airflow/models/connection.py](https://codecov.io/gh/apache/airflow/pull/6090/diff?src=pr&el=tree#diff-YWlyZmxvdy9tb2RlbHMvY29ubmVjdGlvbi5weQ==) | `95.07% <ø> (ø)` | :arrow_up: | | [airflow/providers/apache/livy/sensors/livy.py](https://codecov.io/gh/apache/airflow/pull/6090/diff?src=pr&el=tree#diff-YWlyZmxvdy9wcm92aWRlcnMvYXBhY2hlL2xpdnkvc2Vuc29ycy9saXZ5LnB5) | `100% <100%> (ø)` | | | [...providers/apache/livy/example\_dags/example\_livy.py](https://codecov.io/gh/apache/airflow/pull/6090/diff?src=pr&el=tree#diff-YWlyZmxvdy9wcm92aWRlcnMvYXBhY2hlL2xpdnkvZXhhbXBsZV9kYWdzL2V4YW1wbGVfbGl2eS5weQ==) | `100% <100%> (ø)` | | | [airflow/utils/db.py](https://codecov.io/gh/apache/airflow/pull/6090/diff?src=pr&el=tree#diff-YWlyZmxvdy91dGlscy9kYi5weQ==) | `98.3% <100%> (+0.02%)` | :arrow_up: | | [airflow/providers/apache/livy/operators/livy.py](https://codecov.io/gh/apache/airflow/pull/6090/diff?src=pr&el=tree#diff-YWlyZmxvdy9wcm92aWRlcnMvYXBhY2hlL2xpdnkvb3BlcmF0b3JzL2xpdnkucHk=) | `97.5% <97.5%> (ø)` | | | [airflow/providers/apache/livy/hooks/livy.py](https://codecov.io/gh/apache/airflow/pull/6090/diff?src=pr&el=tree#diff-YWlyZmxvdy9wcm92aWRlcnMvYXBhY2hlL2xpdnkvaG9va3MvbGl2eS5weQ==) | `98.62% <98.62%> (ø)` | | | [...w/providers/apache/hive/operators/mysql\_to\_hive.py](https://codecov.io/gh/apache/airflow/pull/6090/diff?src=pr&el=tree#diff-YWlyZmxvdy9wcm92aWRlcnMvYXBhY2hlL2hpdmUvb3BlcmF0b3JzL215c3FsX3RvX2hpdmUucHk=) | `35.84% <0%> (-64.16%)` | :arrow_down: | | [airflow/kubernetes/volume\_mount.py](https://codecov.io/gh/apache/airflow/pull/6090/diff?src=pr&el=tree#diff-YWlyZmxvdy9rdWJlcm5ldGVzL3ZvbHVtZV9tb3VudC5weQ==) | `44.44% <0%> (-55.56%)` | :arrow_down: | | [airflow/kubernetes/volume.py](https://codecov.io/gh/apache/airflow/pull/6090/diff?src=pr&el=tree#diff-YWlyZmxvdy9rdWJlcm5ldGVzL3ZvbHVtZS5weQ==) | `52.94% <0%> (-47.06%)` | :arrow_down: | | [airflow/security/kerberos.py](https://codecov.io/gh/apache/airflow/pull/6090/diff?src=pr&el=tree#diff-YWlyZmxvdy9zZWN1cml0eS9rZXJiZXJvcy5weQ==) | `30.43% <0%> (-45.66%)` | :arrow_down: | | ... and [18 more](https://codecov.io/gh/apache/airflow/pull/6090/diff?src=pr&el=tree-more) | | -- [Continue to review full report at Codecov](https://codecov.io/gh/apache/airflow/pull/6090?src=pr&el=continue). > **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta) > `Δ = absolute (impact)`, `ø = not affected`, `? = missing data` > Powered by [Codecov](https://codecov.io/gh/apache/airflow/pull/6090?src=pr&el=footer). Last update [7c95c81...2b0e5e2](https://codecov.io/gh/apache/airflow/pull/6090?src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments). This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [airflow] vsoch commented on issue #7191: [AIRFLOW-4030] second attempt to add singularity to airflow
vsoch commented on issue #7191: [AIRFLOW-4030] second attempt to add singularity to airflow URL: https://github.com/apache/airflow/pull/7191#issuecomment-587935001 Can somebody explain what the task_id (provided as task-id?) is expected to work, be parsed, etc? @mik-laj added it to the test cases last year, and it's not clear if I need to be parsing this for the operator - just trying as it was it failed that the key word argument wasn't valid. >airflow.exceptions.AirflowException: Invalid arguments were passed to SingularityOperator (task_id: task-id). Invalid arguments were: This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Updated] (AIRFLOW-6836) DebugExecutor failing to change task state.
[ https://issues.apache.org/jira/browse/AIRFLOW-6836?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sam Wheating updated AIRFLOW-6836: -- Description: Running a DAG locally with the DebugExecutor fails with the following error: {noformat} File "/Users/samwheating/.pyenv/versions/3.7.3/lib/python3.7/site-packages/airflow/executors/debug_executor.py", line 148, in change_state self.running.remove(key) AttributeError: 'dict' object has no attribute 'remove' {noformat} This seems to be because the change_state function is expecting a set rather than a dict for `self.running`. This should be updated to use `del self.running[key]` or `self.running.pop(key)`. I'll submit a PR for this tomorrow. was: Running a DAG locally with the DebugExecutor fails with the following error: ``` File "/Users/samwheating/.pyenv/versions/3.7.3/lib/python3.7/site-packages/airflow/executors/debug_executor.py", line 148, in change_state self.running.remove(key) AttributeError: 'dict' object has no attribute 'remove' ``` This seems to be because the change_state function is expecting a set rather than a dict for `self.running`. This should be updated to use `del self.running[key]` or `self.running.pop(key)`. I'll submit a PR for this tomorrow. > DebugExecutor failing to change task state. > --- > > Key: AIRFLOW-6836 > URL: https://issues.apache.org/jira/browse/AIRFLOW-6836 > Project: Apache Airflow > Issue Type: Bug > Components: executors >Affects Versions: 1.10.9 >Reporter: Sam Wheating >Assignee: Sam Wheating >Priority: Minor > Labels: easyfix > Fix For: 1.10.10 > > Original Estimate: 48h > Remaining Estimate: 48h > > Running a DAG locally with the DebugExecutor fails with the following error: > > {noformat} > File > "/Users/samwheating/.pyenv/versions/3.7.3/lib/python3.7/site-packages/airflow/executors/debug_executor.py", > line 148, in change_state > self.running.remove(key) > AttributeError: 'dict' object has no attribute 'remove' > {noformat} > This seems to be because the change_state function is expecting a set rather > than a dict for `self.running`. This should be updated to use `del > self.running[key]` or `self.running.pop(key)`. > I'll submit a PR for this tomorrow. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (AIRFLOW-6836) DebugExecutor failing to change task state.
Sam Wheating created AIRFLOW-6836: - Summary: DebugExecutor failing to change task state. Key: AIRFLOW-6836 URL: https://issues.apache.org/jira/browse/AIRFLOW-6836 Project: Apache Airflow Issue Type: Bug Components: executors Affects Versions: 1.10.9 Reporter: Sam Wheating Assignee: Sam Wheating Fix For: 1.10.10 Running a DAG locally with the DebugExecutor fails with the following error: ``` File "/Users/samwheating/.pyenv/versions/3.7.3/lib/python3.7/site-packages/airflow/executors/debug_executor.py", line 148, in change_state self.running.remove(key) AttributeError: 'dict' object has no attribute 'remove' ``` This seems to be because the change_state function is expecting a set rather than a dict for `self.running`. This should be updated to use `del self.running[key]` or `self.running.pop(key)`. I'll submit a PR for this tomorrow. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[GitHub] [airflow] ashb commented on a change in pull request #7446: [AIRFLOW-6474] list_dag_runs cli command should allow exec_date between start/end range and print start/end times
ashb commented on a change in pull request #7446: [AIRFLOW-6474] list_dag_runs cli command should allow exec_date between start/end range and print start/end times URL: https://github.com/apache/airflow/pull/7446#discussion_r380961616 ## File path: airflow/models/dagrun.py ## @@ -135,7 +135,7 @@ def find(dag_id=None, run_id=None, execution_date=None, :param run_id: defines the run id for this dag run :type run_id: str :param execution_date: the execution date -:type execution_date: datetime.datetime +:type execution_date: datetime.datetime, list Review comment: ```suggestion :type execution_date: datetime.datetime or list[datetime.datetime] ``` Then This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [airflow] ashb commented on a change in pull request #7446: [AIRFLOW-6474] list_dag_runs cli command should allow exec_date between start/end range and print start/end times
ashb commented on a change in pull request #7446: [AIRFLOW-6474] list_dag_runs cli command should allow exec_date between start/end range and print start/end times URL: https://github.com/apache/airflow/pull/7446#discussion_r380961251 ## File path: airflow/bin/cli.py ## @@ -597,8 +608,12 @@ class CLIFactory: 'help': "List dag runs given a DAG id. If state option is given, it will only " "search for all the dagruns with the given state. " "If no_backfill option is given, it will filter out " -"all backfill dagruns for given dag id", -'args': ('dag_id', 'no_backfill', 'state', 'output',), +"all backfill dagruns for given dag id" +"If exec_date_from is given, it will filter out " +"all the dagruns that were executed before this date. " +"If exec_date_to is given, it will filter out " +"all the dagruns that were executed after this date. ", +'args': ('dag_id_opt', 'no_backfill', 'state', 'output', "exec_date_from", "exec_date_to"), Review comment: `airflow backfill` has `--start_date` and `--end_date` args that serve a similar purpose. I think you should re-used those names instead of creating new ones. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [airflow] ashb commented on a change in pull request #7446: [AIRFLOW-6474] list_dag_runs cli command should allow exec_date between start/end range and print start/end times
ashb commented on a change in pull request #7446: [AIRFLOW-6474] list_dag_runs cli command should allow exec_date between start/end range and print start/end times URL: https://github.com/apache/airflow/pull/7446#discussion_r380960677 ## File path: airflow/bin/cli.py ## @@ -597,8 +608,12 @@ class CLIFactory: 'help': "List dag runs given a DAG id. If state option is given, it will only " "search for all the dagruns with the given state. " "If no_backfill option is given, it will filter out " -"all backfill dagruns for given dag id", -'args': ('dag_id', 'no_backfill', 'state', 'output',), +"all backfill dagruns for given dag id" +"If exec_date_from is given, it will filter out " +"all the dagruns that were executed before this date. " +"If exec_date_to is given, it will filter out " +"all the dagruns that were executed after this date. ", Review comment: What does this look like when you run `airflow dag_runs --list`? I think you have formatting "bugs" in here. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [airflow] ashb commented on a change in pull request #7446: [AIRFLOW-6474] list_dag_runs cli command should allow exec_date between start/end range and print start/end times
ashb commented on a change in pull request #7446: [AIRFLOW-6474] list_dag_runs cli command should allow exec_date between start/end range and print start/end times URL: https://github.com/apache/airflow/pull/7446#discussion_r380960239 ## File path: airflow/bin/cli.py ## @@ -131,11 +131,22 @@ class CLIFactory: # list_dag_runs 'no_backfill': Arg( ("--no_backfill",), -"filter all the backfill dagruns given the dag id", "store_true"), Review comment: It's good practice to avoid whitespace/formatting only changes to code you don't touch. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Commented] (AIRFLOW-6474) list_dag_runs cli command should allow exec_date between start/end range and print start/end times
[ https://issues.apache.org/jira/browse/AIRFLOW-6474?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17039471#comment-17039471 ] alexander commented on AIRFLOW-6474: I've added printing of end_date. > list_dag_runs cli command should allow exec_date between start/end range and > print start/end times > -- > > Key: AIRFLOW-6474 > URL: https://issues.apache.org/jira/browse/AIRFLOW-6474 > Project: Apache Airflow > Issue Type: Improvement > Components: cli >Affects Versions: 1.10.7 >Reporter: t oo >Assignee: alexander >Priority: Major > Labels: gsoc, gsoc2020, mentor > > 1. accept argument exec_date_from, exec_date_to to filter execution_dates > returned, ie show dag runs with exec_date between 20190901 and 20190930 > 2. separate to that in the output print the start_date and end_date of each > dagrun (ie execdate for 20190907 had start_date 2019090804:23 and end_date > 2019090804:38 > 3. dag_id arg should be optional -- This message was sent by Atlassian Jira (v8.3.4#803005)
[GitHub] [airflow] ashb commented on a change in pull request #7391: [AIRFLOW-6663] Prepare backporting packages
ashb commented on a change in pull request #7391: [AIRFLOW-6663] Prepare backporting packages URL: https://github.com/apache/airflow/pull/7391#discussion_r380941844 ## File path: README.md ## @@ -107,6 +108,33 @@ unit of work and continuity. ![](/docs/img/code.png) +## Airflow Versions, backporting packages and future upgrade path + + +Currently stable versions of Apache Airflow are released in 1.10.* series. We are working on the +future, incompatible version of Airflow from the 2.0.* series. It is going to be released in +in 2020. However the exact time of release depends on many factors and is yet unknown. +We have already a lot of changes in the hooks/operators/sensors for many external systems +and they are not used because they are part of the master/2.0 release. + +In the Airflow 2.0 - following AIP-21 "change in import paths" all the non-core operators/hooks/sensors +of Apache Airflow have been moved to the "airflow.providers" package. This opened a possibility to +use the operators from Airflow 2.0 in Airflow 1.10 - with the constraint that those +packages can only be used in python3.6+ environment. + +Therefore we decided to prepare and release backport packages that can be installed +for older Airflow versions. Those backport packages are released more frequently. Users do not +have to upgrade their Airflow version to use those packages. There are a number of incompatibilities +between Airflow 2.0 and 1.10.* - documented in [UPDATING.md](UPDATING.md). With backported +providers package users can migrate their DAGs to the new providers package incrementally +and once they convert to the new operators/sensors/hooks they can seamlessly migrate their +environments to Airflow 2.0. + +More information about the status and releases of the back-ported packages are available +at https://cwiki.apache.org/confluence/display/AIRFLOW/Backported+providers+packages+for+Airflow+1.10.*+series + +You Review comment: ```suggestion ``` This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [airflow] ashb commented on a change in pull request #7391: [AIRFLOW-6663] Prepare backporting packages
ashb commented on a change in pull request #7391: [AIRFLOW-6663] Prepare backporting packages URL: https://github.com/apache/airflow/pull/7391#discussion_r380941290 ## File path: README.md ## @@ -107,6 +108,33 @@ unit of work and continuity. ![](/docs/img/code.png) +## Airflow Versions, backporting packages and future upgrade path Review comment: ```suggestion ## Using hooks and Operators from "master" in Airflow 1.10 ``` I think this captures what users' would want to see more clearly in the ToC This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [airflow] ashb commented on a change in pull request #7391: [AIRFLOW-6663] Prepare backporting packages
ashb commented on a change in pull request #7391: [AIRFLOW-6663] Prepare backporting packages URL: https://github.com/apache/airflow/pull/7391#discussion_r380940650 ## File path: README.md ## @@ -107,6 +108,33 @@ unit of work and continuity. ![](/docs/img/code.png) +## Airflow Versions, backporting packages and future upgrade path + + +Currently stable versions of Apache Airflow are released in 1.10.* series. We are working on the +future, incompatible version of Airflow from the 2.0.* series. It is going to be released in Review comment: Let's not say "incompatible" like this please -- that's a stronger word than we need. ``` Currently stable versions of Apache Airflow are released in 1.10.* series. We are working on the future, Airflow 2.x series which has some backwards-incompatible changes ``` This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [airflow] ashb commented on a change in pull request #7391: [AIRFLOW-6663] Prepare backporting packages
ashb commented on a change in pull request #7391: [AIRFLOW-6663] Prepare backporting packages URL: https://github.com/apache/airflow/pull/7391#discussion_r380939740 ## File path: MANIFEST-packages.in ## @@ -0,0 +1,24 @@ +# +# Licensed to the Apache Software Foundation (ASF) under one +# or more contributor license agreements. See the NOTICE file +# distributed with this work for additional information +# regarding copyright ownership. The ASF licenses this file +# to you under the Apache License, Version 2.0 (the +# "License"); you may not use this file except in compliance +# with the License. You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, +# software distributed under the License is distributed on an +# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY +# KIND, either express or implied. See the License for the +# specific language governing permissions and limitations +# under the License. + +include NOTICE +include LICENSE +include CHANGELOG.txt +include README.md +graft licenses/ Review comment: This is warning is still showing up on CI @potiuk ``` --- Preparing backporting package mongo --- warning: Failed to find the configured license file 'licenses/*' warning: no previously-included files matching '__pycache__' found anywhere in distribution warning: no previously-included files matching '*.pyc' found anywhere in distribution warning: check: missing required meta-data: url warning: build_py: byte-compiling is disabled, skipping. warning: install_lib: byte-compiling is disabled, skipping. gitpython not found: Cannot compute the git version. running clean removing 'build/lib' (and everything under it) removing 'build/bdist.linux-x86_64' (and everything under it) 'build/scripts-3.6' does not exist -- can't clean it removing 'build' ``` This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [airflow] ashb commented on a change in pull request #7391: [AIRFLOW-6663] Prepare backporting packages
ashb commented on a change in pull request #7391: [AIRFLOW-6663] Prepare backporting packages URL: https://github.com/apache/airflow/pull/7391#discussion_r380939740 ## File path: MANIFEST-packages.in ## @@ -0,0 +1,24 @@ +# +# Licensed to the Apache Software Foundation (ASF) under one +# or more contributor license agreements. See the NOTICE file +# distributed with this work for additional information +# regarding copyright ownership. The ASF licenses this file +# to you under the Apache License, Version 2.0 (the +# "License"); you may not use this file except in compliance +# with the License. You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, +# software distributed under the License is distributed on an +# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY +# KIND, either express or implied. See the License for the +# specific language governing permissions and limitations +# under the License. + +include NOTICE +include LICENSE +include CHANGELOG.txt +include README.md +graft licenses/ Review comment: This is warning is still showing up on CI @potiuk https://travis-ci.org/apache/airflow/jobs/651778738 ``` --- Preparing backporting package mongo --- warning: Failed to find the configured license file 'licenses/*' warning: no previously-included files matching '__pycache__' found anywhere in distribution warning: no previously-included files matching '*.pyc' found anywhere in distribution warning: check: missing required meta-data: url warning: build_py: byte-compiling is disabled, skipping. warning: install_lib: byte-compiling is disabled, skipping. gitpython not found: Cannot compute the git version. running clean removing 'build/lib' (and everything under it) removing 'build/bdist.linux-x86_64' (and everything under it) 'build/scripts-3.6' does not exist -- can't clean it removing 'build' ``` This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [airflow] ashb commented on issue #7391: [AIRFLOW-6663] Prepare backporting packages
ashb commented on issue #7391: [AIRFLOW-6663] Prepare backporting packages URL: https://github.com/apache/airflow/pull/7391#issuecomment-587860742 :eyes: This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Commented] (AIRFLOW-1202) Create Elasticsearch Hook
[ https://issues.apache.org/jira/browse/AIRFLOW-1202?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17039456#comment-17039456 ] ASF GitHub Bot commented on AIRFLOW-1202: - potiuk commented on pull request #7358: [AIRFLOW-1202] Create Elasticsearch Hook URL: https://github.com/apache/airflow/pull/7358 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org > Create Elasticsearch Hook > -- > > Key: AIRFLOW-1202 > URL: https://issues.apache.org/jira/browse/AIRFLOW-1202 > Project: Apache Airflow > Issue Type: Improvement > Components: hooks >Reporter: Hesen Peng >Assignee: Hesen Peng >Priority: Minor > Original Estimate: 168h > Remaining Estimate: 168h > > i would like to add an Elasticsearch hook for Airflow. The code is already > done in the branch linked. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (AIRFLOW-1202) Create Elasticsearch Hook
[ https://issues.apache.org/jira/browse/AIRFLOW-1202?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17039457#comment-17039457 ] ASF subversion and git services commented on AIRFLOW-1202: -- Commit 65dd28eb77d996ec8306c67d5ce1ccee2c14cc9d in airflow's branch refs/heads/master from Hossein Torabi [ https://gitbox.apache.org/repos/asf?p=airflow.git;h=65dd28e ] [AIRFLOW-1202] Create Elasticsearch Hook (#7358) > Create Elasticsearch Hook > -- > > Key: AIRFLOW-1202 > URL: https://issues.apache.org/jira/browse/AIRFLOW-1202 > Project: Apache Airflow > Issue Type: Improvement > Components: hooks >Reporter: Hesen Peng >Assignee: Hesen Peng >Priority: Minor > Original Estimate: 168h > Remaining Estimate: 168h > > i would like to add an Elasticsearch hook for Airflow. The code is already > done in the branch linked. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Resolved] (AIRFLOW-1202) Create Elasticsearch Hook
[ https://issues.apache.org/jira/browse/AIRFLOW-1202?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jarek Potiuk resolved AIRFLOW-1202. --- Fix Version/s: 2.0.0 Resolution: Fixed > Create Elasticsearch Hook > -- > > Key: AIRFLOW-1202 > URL: https://issues.apache.org/jira/browse/AIRFLOW-1202 > Project: Apache Airflow > Issue Type: Improvement > Components: hooks >Reporter: Hesen Peng >Assignee: Hesen Peng >Priority: Minor > Fix For: 2.0.0 > > Original Estimate: 168h > Remaining Estimate: 168h > > i would like to add an Elasticsearch hook for Airflow. The code is already > done in the branch linked. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[GitHub] [airflow] potiuk commented on issue #7358: [AIRFLOW-1202] Create Elasticsearch Hook
potiuk commented on issue #7358: [AIRFLOW-1202] Create Elasticsearch Hook URL: https://github.com/apache/airflow/pull/7358#issuecomment-587844278 Thanks @blcksrx ! This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [airflow] potiuk merged pull request #7358: [AIRFLOW-1202] Create Elasticsearch Hook
potiuk merged pull request #7358: [AIRFLOW-1202] Create Elasticsearch Hook URL: https://github.com/apache/airflow/pull/7358 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Created] (AIRFLOW-6835) GoogleCloudStorageToBigQueryOperator failed when loading empty csv file
Feng Gao created AIRFLOW-6835: - Summary: GoogleCloudStorageToBigQueryOperator failed when loading empty csv file Key: AIRFLOW-6835 URL: https://issues.apache.org/jira/browse/AIRFLOW-6835 Project: Apache Airflow Issue Type: Bug Components: gcp Affects Versions: 1.10.6 Environment: Google Composer Image version: composer-1.9.0-airflow-1.10.6 Reporter: Feng Gao GoogleCloudStorageToBigQueryOperator failed when loading empty csv file (size 0). Found error in logs : Exception: BigQuery job failed. Final error was: \{'reason': 'invalidQuery', 'location': 'query', 'message': 'There are no column descriptions provided for table _ev_5GNTR3nPyPPZfamjrtfpgieU8fg6kU3A_8e8qCoEAMI_source'}. Re-ran the task succeeded after adding one row to the csv file. Google Composer Image version: composer-1.9.0-airflow-1.10.6 The same dag works fine on Google Composer image version: composer-1.8.4-airflow-1.9.0 -- This message was sent by Atlassian Jira (v8.3.4#803005)
[GitHub] [airflow] vsoch commented on issue #7191: [AIRFLOW-4030] second attempt to add singularity to airflow
vsoch commented on issue #7191: [AIRFLOW-4030] second attempt to add singularity to airflow URL: https://github.com/apache/airflow/pull/7191#issuecomment-587823490 docs build passed! Tiny progress :) This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [airflow] potiuk commented on issue #7391: [AIRFLOW-6663] Prepare backporting packages
potiuk commented on issue #7391: [AIRFLOW-6663] Prepare backporting packages URL: https://github.com/apache/airflow/pull/7391#issuecomment-587815757 @ashb @mik-laj ? Can I merge the backporting packages? This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [airflow] potiuk commented on issue #7433: [AIRFLOW-6820] split breeze into functions
potiuk commented on issue #7433: [AIRFLOW-6820] split breeze into functions URL: https://github.com/apache/airflow/pull/7433#issuecomment-587814433 Hey @ashb. Are you ok with how it is ? This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [airflow] codecov-io edited a comment on issue #7433: [AIRFLOW-6820] split breeze into functions
codecov-io edited a comment on issue #7433: [AIRFLOW-6820] split breeze into functions URL: https://github.com/apache/airflow/pull/7433#issuecomment-586737012 # [Codecov](https://codecov.io/gh/apache/airflow/pull/7433?src=pr&el=h1) Report > Merging [#7433](https://codecov.io/gh/apache/airflow/pull/7433?src=pr&el=desc) into [master](https://codecov.io/gh/apache/airflow/commit/9cbd7de6d115795aba8bfb8addb060bfdfbdf87b?src=pr&el=desc) will **increase** coverage by `0.23%`. > The diff coverage is `n/a`. [![Impacted file tree graph](https://codecov.io/gh/apache/airflow/pull/7433/graphs/tree.svg?width=650&token=WdLKlKHOAU&height=150&src=pr)](https://codecov.io/gh/apache/airflow/pull/7433?src=pr&el=tree) ```diff @@Coverage Diff@@ ## master #7433 +/- ## = + Coverage 86.37% 86.6% +0.23% = Files 877 877 Lines 41221 41336 +115 = + Hits35603 35799 +196 + Misses 56185537 -81 ``` | [Impacted Files](https://codecov.io/gh/apache/airflow/pull/7433?src=pr&el=tree) | Coverage Δ | | |---|---|---| | [airflow/configuration.py](https://codecov.io/gh/apache/airflow/pull/7433/diff?src=pr&el=tree#diff-YWlyZmxvdy9jb25maWd1cmF0aW9uLnB5) | `82.03% <0%> (-5.98%)` | :arrow_down: | | [...viders/docker/example\_dags/example\_docker\_swarm.py](https://codecov.io/gh/apache/airflow/pull/7433/diff?src=pr&el=tree#diff-YWlyZmxvdy9wcm92aWRlcnMvZG9ja2VyL2V4YW1wbGVfZGFncy9leGFtcGxlX2RvY2tlcl9zd2FybS5weQ==) | `100% <0%> (ø)` | :arrow_up: | | [...ow/providers/docker/example\_dags/example\_docker.py](https://codecov.io/gh/apache/airflow/pull/7433/diff?src=pr&el=tree#diff-YWlyZmxvdy9wcm92aWRlcnMvZG9ja2VyL2V4YW1wbGVfZGFncy9leGFtcGxlX2RvY2tlci5weQ==) | `100% <0%> (ø)` | :arrow_up: | | [airflow/config\_templates/default\_celery.py](https://codecov.io/gh/apache/airflow/pull/7433/diff?src=pr&el=tree#diff-YWlyZmxvdy9jb25maWdfdGVtcGxhdGVzL2RlZmF1bHRfY2VsZXJ5LnB5) | `54.16% <0%> (+1.22%)` | :arrow_up: | | [airflow/kubernetes/refresh\_config.py](https://codecov.io/gh/apache/airflow/pull/7433/diff?src=pr&el=tree#diff-YWlyZmxvdy9rdWJlcm5ldGVzL3JlZnJlc2hfY29uZmlnLnB5) | `74.5% <0%> (+23.52%)` | :arrow_up: | | [...viders/cncf/kubernetes/operators/kubernetes\_pod.py](https://codecov.io/gh/apache/airflow/pull/7433/diff?src=pr&el=tree#diff-YWlyZmxvdy9wcm92aWRlcnMvY25jZi9rdWJlcm5ldGVzL29wZXJhdG9ycy9rdWJlcm5ldGVzX3BvZC5weQ==) | `94.89% <0%> (+25.51%)` | :arrow_up: | | [airflow/kubernetes/pod\_launcher.py](https://codecov.io/gh/apache/airflow/pull/7433/diff?src=pr&el=tree#diff-YWlyZmxvdy9rdWJlcm5ldGVzL3BvZF9sYXVuY2hlci5weQ==) | `92.25% <0%> (+45.07%)` | :arrow_up: | | [airflow/kubernetes/volume.py](https://codecov.io/gh/apache/airflow/pull/7433/diff?src=pr&el=tree#diff-YWlyZmxvdy9rdWJlcm5ldGVzL3ZvbHVtZS5weQ==) | `100% <0%> (+47.05%)` | :arrow_up: | | [airflow/kubernetes/volume\_mount.py](https://codecov.io/gh/apache/airflow/pull/7433/diff?src=pr&el=tree#diff-YWlyZmxvdy9rdWJlcm5ldGVzL3ZvbHVtZV9tb3VudC5weQ==) | `100% <0%> (+55.55%)` | :arrow_up: | -- [Continue to review full report at Codecov](https://codecov.io/gh/apache/airflow/pull/7433?src=pr&el=continue). > **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta) > `Δ = absolute (impact)`, `ø = not affected`, `? = missing data` > Powered by [Codecov](https://codecov.io/gh/apache/airflow/pull/7433?src=pr&el=footer). Last update [9cbd7de...84fce4b](https://codecov.io/gh/apache/airflow/pull/7433?src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments). This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [airflow] potiuk commented on issue #7455: [AIRFLOW-XXXX] Change title and add some more helpful information
potiuk commented on issue #7455: [AIRFLOW-] Change title and add some more helpful information URL: https://github.com/apache/airflow/pull/7455#issuecomment-587766510 Thanks @ryw This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [airflow] potiuk merged pull request #7455: [AIRFLOW-XXXX] Change title and add some more helpful information
potiuk merged pull request #7455: [AIRFLOW-] Change title and add some more helpful information URL: https://github.com/apache/airflow/pull/7455 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [airflow] potiuk commented on issue #7389: [AIRFLOW-6763] Make systems tests ready for backport tests
potiuk commented on issue #7389: [AIRFLOW-6763] Make systems tests ready for backport tests URL: https://github.com/apache/airflow/pull/7389#issuecomment-587763532 Applied changes from @ashb comments in #7439 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [airflow] snazzyfox commented on a change in pull request #7451: [AIRFLOW-6830] Add Subject/MessageAttributes to SNS hook and operator
snazzyfox commented on a change in pull request #7451: [AIRFLOW-6830] Add Subject/MessageAttributes to SNS hook and operator URL: https://github.com/apache/airflow/pull/7451#discussion_r380856952 ## File path: tests/providers/amazon/aws/hooks/test_sns.py ## @@ -49,7 +49,24 @@ def test_publish_to_target(self): self.assertTrue('MessageId' in response) @mock_sns -def test_publish_to_target_without_subject(self): +def test_publish_to_target_with_attributes(self): +hook = AwsSnsHook(aws_conn_id='aws_default') + +message = "Hello world" +topic_name = "test-topic" +target = hook.get_conn().create_topic(Name=topic_name).get('TopicArn') + +response = hook.publish_to_target(target, message, message_attributes={ +'test-string': 'string-value', +'test-number': 123456, +'test-array': ['first', 'second', 'third'], +'test-binary': b'binary-value', +}) + +self.assertTrue('MessageId' in response) Review comment: I used assertTrue since that's used in the rest of these tests. ~~Should I update those as well?~~ I've updated the untouched tests in this file as well. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Commented] (AIRFLOW-6255) Redis Hook - refactor & add most important methods
[ https://issues.apache.org/jira/browse/AIRFLOW-6255?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17039412#comment-17039412 ] ASF GitHub Bot commented on AIRFLOW-6255: - stale[bot] commented on pull request #6820: [AIRFLOW-6255] Redis Hook Refactor URL: https://github.com/apache/airflow/pull/6820 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org > Redis Hook - refactor & add most important methods > -- > > Key: AIRFLOW-6255 > URL: https://issues.apache.org/jira/browse/AIRFLOW-6255 > Project: Apache Airflow > Issue Type: Improvement > Components: hooks >Affects Versions: 1.10.6 >Reporter: Mariusz Górski >Assignee: Mariusz Górski >Priority: Minor > Labels: hooks, redis, refactor > > Currently Redis Hook consists of get_conn func only and every operation using > it is calling Redis client methods directly. This Issue aims to extend Redis > Hook so it provides all necessary methods wrapping Redis functionality used > so far. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[GitHub] [airflow] stale[bot] closed pull request #6820: [AIRFLOW-6255] Redis Hook Refactor
stale[bot] closed pull request #6820: [AIRFLOW-6255] Redis Hook Refactor URL: https://github.com/apache/airflow/pull/6820 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [airflow] codecov-io edited a comment on issue #7446: [AIRFLOW-6474] list_dag_runs cli command should allow exec_date between start/end range and print start/end times
codecov-io edited a comment on issue #7446: [AIRFLOW-6474] list_dag_runs cli command should allow exec_date between start/end range and print start/end times URL: https://github.com/apache/airflow/pull/7446#issuecomment-587128695 # [Codecov](https://codecov.io/gh/apache/airflow/pull/7446?src=pr&el=h1) Report > Merging [#7446](https://codecov.io/gh/apache/airflow/pull/7446?src=pr&el=desc) into [master](https://codecov.io/gh/apache/airflow/commit/d1ab3705d3d03f68abc2b900ebb35e8497dee2e7?src=pr&el=desc) will **decrease** coverage by `0.44%`. > The diff coverage is `71.42%`. [![Impacted file tree graph](https://codecov.io/gh/apache/airflow/pull/7446/graphs/tree.svg?width=650&token=WdLKlKHOAU&height=150&src=pr)](https://codecov.io/gh/apache/airflow/pull/7446?src=pr&el=tree) ```diff @@Coverage Diff @@ ## master#7446 +/- ## == - Coverage 86.64% 86.19% -0.45% == Files 878 877 -1 Lines 4121941258 +39 == - Hits3571535564 -151 - Misses 5504 5694 +190 ``` | [Impacted Files](https://codecov.io/gh/apache/airflow/pull/7446?src=pr&el=tree) | Coverage Δ | | |---|---|---| | [airflow/bin/cli.py](https://codecov.io/gh/apache/airflow/pull/7446/diff?src=pr&el=tree#diff-YWlyZmxvdy9iaW4vY2xpLnB5) | `94.73% <ø> (ø)` | :arrow_up: | | [airflow/cli/commands/dag\_command.py](https://codecov.io/gh/apache/airflow/pull/7446/diff?src=pr&el=tree#diff-YWlyZmxvdy9jbGkvY29tbWFuZHMvZGFnX2NvbW1hbmQucHk=) | `84.41% <100%> (ø)` | :arrow_up: | | [airflow/models/dagrun.py](https://codecov.io/gh/apache/airflow/pull/7446/diff?src=pr&el=tree#diff-YWlyZmxvdy9tb2RlbHMvZGFncnVuLnB5) | `95.79% <66.66%> (-0.76%)` | :arrow_down: | | [...w/providers/apache/hive/operators/mysql\_to\_hive.py](https://codecov.io/gh/apache/airflow/pull/7446/diff?src=pr&el=tree#diff-YWlyZmxvdy9wcm92aWRlcnMvYXBhY2hlL2hpdmUvb3BlcmF0b3JzL215c3FsX3RvX2hpdmUucHk=) | `35.84% <0%> (-64.16%)` | :arrow_down: | | [airflow/kubernetes/volume\_mount.py](https://codecov.io/gh/apache/airflow/pull/7446/diff?src=pr&el=tree#diff-YWlyZmxvdy9rdWJlcm5ldGVzL3ZvbHVtZV9tb3VudC5weQ==) | `44.44% <0%> (-55.56%)` | :arrow_down: | | [airflow/kubernetes/volume.py](https://codecov.io/gh/apache/airflow/pull/7446/diff?src=pr&el=tree#diff-YWlyZmxvdy9rdWJlcm5ldGVzL3ZvbHVtZS5weQ==) | `52.94% <0%> (-47.06%)` | :arrow_down: | | [airflow/security/kerberos.py](https://codecov.io/gh/apache/airflow/pull/7446/diff?src=pr&el=tree#diff-YWlyZmxvdy9zZWN1cml0eS9rZXJiZXJvcy5weQ==) | `30.43% <0%> (-45.66%)` | :arrow_down: | | [airflow/kubernetes/pod\_launcher.py](https://codecov.io/gh/apache/airflow/pull/7446/diff?src=pr&el=tree#diff-YWlyZmxvdy9rdWJlcm5ldGVzL3BvZF9sYXVuY2hlci5weQ==) | `47.18% <0%> (-45.08%)` | :arrow_down: | | [airflow/providers/mysql/operators/mysql.py](https://codecov.io/gh/apache/airflow/pull/7446/diff?src=pr&el=tree#diff-YWlyZmxvdy9wcm92aWRlcnMvbXlzcWwvb3BlcmF0b3JzL215c3FsLnB5) | `55% <0%> (-45%)` | :arrow_down: | | [...viders/cncf/kubernetes/operators/kubernetes\_pod.py](https://codecov.io/gh/apache/airflow/pull/7446/diff?src=pr&el=tree#diff-YWlyZmxvdy9wcm92aWRlcnMvY25jZi9rdWJlcm5ldGVzL29wZXJhdG9ycy9rdWJlcm5ldGVzX3BvZC5weQ==) | `69.38% <0%> (-25.52%)` | :arrow_down: | | ... and [33 more](https://codecov.io/gh/apache/airflow/pull/7446/diff?src=pr&el=tree-more) | | -- [Continue to review full report at Codecov](https://codecov.io/gh/apache/airflow/pull/7446?src=pr&el=continue). > **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta) > `Δ = absolute (impact)`, `ø = not affected`, `? = missing data` > Powered by [Codecov](https://codecov.io/gh/apache/airflow/pull/7446?src=pr&el=footer). Last update [d1ab370...886fca4](https://codecov.io/gh/apache/airflow/pull/7446?src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments). This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [airflow] potiuk commented on a change in pull request #7439: [AIRFLOW-6204][depends on 6763] Create GoogleSystemTest class
potiuk commented on a change in pull request #7439: [AIRFLOW-6204][depends on 6763] Create GoogleSystemTest class URL: https://github.com/apache/airflow/pull/7439#discussion_r380888600 ## File path: tests/conftest.py ## @@ -87,6 +88,26 @@ def pytest_addoption(parser): metavar="RUNTIME", help="only run tests matching the runtime: [kubernetes].", ) +group.addoption( +"--systems", +action="store", +metavar="SYSTEMS", +help="only run tests matching the systems specified [google.cloud, google.marketing_platform]", +) +group.addoption( +"--include-long-lasting", Review comment: Updated in my change. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Commented] (AIRFLOW-6823) Breeze broken on current master on Linux
[ https://issues.apache.org/jira/browse/AIRFLOW-6823?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17039384#comment-17039384 ] ASF subversion and git services commented on AIRFLOW-6823: -- Commit 03fb50aaa4bd7b00968c300781039cebf91043f2 in airflow's branch refs/heads/master from Jarek Potiuk [ https://gitbox.apache.org/repos/asf?p=airflow.git;h=03fb50a ] [AIRFLOW-6823] Fix root owned files automatically in Breeze (#7449) > Breeze broken on current master on Linux > > > Key: AIRFLOW-6823 > URL: https://issues.apache.org/jira/browse/AIRFLOW-6823 > Project: Apache Airflow > Issue Type: Bug > Components: breeze >Affects Versions: 2.0.0 >Reporter: Aleksander Nitecki >Assignee: Ash Berlin-Taylor >Priority: Minor > Fix For: 2.0.0 > > Attachments: Breeze on current master log.txt > > > See the attachment for log. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (AIRFLOW-6823) Breeze broken on current master on Linux
[ https://issues.apache.org/jira/browse/AIRFLOW-6823?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17039383#comment-17039383 ] ASF GitHub Bot commented on AIRFLOW-6823: - potiuk commented on pull request #7449: [AIRFLOW-6823] Fix root owned files automatically in Breeze URL: https://github.com/apache/airflow/pull/7449 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org > Breeze broken on current master on Linux > > > Key: AIRFLOW-6823 > URL: https://issues.apache.org/jira/browse/AIRFLOW-6823 > Project: Apache Airflow > Issue Type: Bug > Components: breeze >Affects Versions: 2.0.0 >Reporter: Aleksander Nitecki >Assignee: Ash Berlin-Taylor >Priority: Minor > Attachments: Breeze on current master log.txt > > > See the attachment for log. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Resolved] (AIRFLOW-6823) Breeze broken on current master on Linux
[ https://issues.apache.org/jira/browse/AIRFLOW-6823?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jarek Potiuk resolved AIRFLOW-6823. --- Fix Version/s: 2.0.0 Resolution: Fixed > Breeze broken on current master on Linux > > > Key: AIRFLOW-6823 > URL: https://issues.apache.org/jira/browse/AIRFLOW-6823 > Project: Apache Airflow > Issue Type: Bug > Components: breeze >Affects Versions: 2.0.0 >Reporter: Aleksander Nitecki >Assignee: Ash Berlin-Taylor >Priority: Minor > Fix For: 2.0.0 > > Attachments: Breeze on current master log.txt > > > See the attachment for log. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[GitHub] [airflow] potiuk merged pull request #7449: [AIRFLOW-6823] Fix root owned files automatically in Breeze
potiuk merged pull request #7449: [AIRFLOW-6823] Fix root owned files automatically in Breeze URL: https://github.com/apache/airflow/pull/7449 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [airflow] codecov-io edited a comment on issue #7451: [AIRFLOW-6830] Add Subject/MessageAttributes to SNS hook and operator
codecov-io edited a comment on issue #7451: [AIRFLOW-6830] Add Subject/MessageAttributes to SNS hook and operator URL: https://github.com/apache/airflow/pull/7451#issuecomment-587304168 # [Codecov](https://codecov.io/gh/apache/airflow/pull/7451?src=pr&el=h1) Report > Merging [#7451](https://codecov.io/gh/apache/airflow/pull/7451?src=pr&el=desc) into [master](https://codecov.io/gh/apache/airflow/commit/dbcd3d8787741fd8203b6d9bdbc5d1da4b10a15b?src=pr&el=desc) will **decrease** coverage by `0.28%`. > The diff coverage is `95%`. [![Impacted file tree graph](https://codecov.io/gh/apache/airflow/pull/7451/graphs/tree.svg?width=650&token=WdLKlKHOAU&height=150&src=pr)](https://codecov.io/gh/apache/airflow/pull/7451?src=pr&el=tree) ```diff @@Coverage Diff @@ ## master#7451 +/- ## == - Coverage 86.66% 86.37% -0.29% == Files 877 877 Lines 4124141266 +25 == - Hits3574235645 -97 - Misses 5499 5621 +122 ``` | [Impacted Files](https://codecov.io/gh/apache/airflow/pull/7451?src=pr&el=tree) | Coverage Δ | | |---|---|---| | [airflow/providers/amazon/aws/operators/sns.py](https://codecov.io/gh/apache/airflow/pull/7451/diff?src=pr&el=tree#diff-YWlyZmxvdy9wcm92aWRlcnMvYW1hem9uL2F3cy9vcGVyYXRvcnMvc25zLnB5) | `100% <100%> (ø)` | :arrow_up: | | [airflow/providers/amazon/aws/hooks/sns.py](https://codecov.io/gh/apache/airflow/pull/7451/diff?src=pr&el=tree#diff-YWlyZmxvdy9wcm92aWRlcnMvYW1hem9uL2F3cy9ob29rcy9zbnMucHk=) | `96.42% <94.11%> (-3.58%)` | :arrow_down: | | [airflow/kubernetes/volume\_mount.py](https://codecov.io/gh/apache/airflow/pull/7451/diff?src=pr&el=tree#diff-YWlyZmxvdy9rdWJlcm5ldGVzL3ZvbHVtZV9tb3VudC5weQ==) | `44.44% <0%> (-55.56%)` | :arrow_down: | | [airflow/kubernetes/volume.py](https://codecov.io/gh/apache/airflow/pull/7451/diff?src=pr&el=tree#diff-YWlyZmxvdy9rdWJlcm5ldGVzL3ZvbHVtZS5weQ==) | `52.94% <0%> (-47.06%)` | :arrow_down: | | [airflow/kubernetes/pod\_launcher.py](https://codecov.io/gh/apache/airflow/pull/7451/diff?src=pr&el=tree#diff-YWlyZmxvdy9rdWJlcm5ldGVzL3BvZF9sYXVuY2hlci5weQ==) | `47.18% <0%> (-45.08%)` | :arrow_down: | | [...viders/cncf/kubernetes/operators/kubernetes\_pod.py](https://codecov.io/gh/apache/airflow/pull/7451/diff?src=pr&el=tree#diff-YWlyZmxvdy9wcm92aWRlcnMvY25jZi9rdWJlcm5ldGVzL29wZXJhdG9ycy9rdWJlcm5ldGVzX3BvZC5weQ==) | `69.38% <0%> (-25.52%)` | :arrow_down: | | [airflow/kubernetes/refresh\_config.py](https://codecov.io/gh/apache/airflow/pull/7451/diff?src=pr&el=tree#diff-YWlyZmxvdy9rdWJlcm5ldGVzL3JlZnJlc2hfY29uZmlnLnB5) | `50.98% <0%> (-23.53%)` | :arrow_down: | | [airflow/configuration.py](https://codecov.io/gh/apache/airflow/pull/7451/diff?src=pr&el=tree#diff-YWlyZmxvdy9jb25maWd1cmF0aW9uLnB5) | `87.78% <0%> (-0.23%)` | :arrow_down: | | [airflow/config\_templates/default\_celery.py](https://codecov.io/gh/apache/airflow/pull/7451/diff?src=pr&el=tree#diff-YWlyZmxvdy9jb25maWdfdGVtcGxhdGVzL2RlZmF1bHRfY2VsZXJ5LnB5) | `52.94% <0%> (ø)` | :arrow_up: | -- [Continue to review full report at Codecov](https://codecov.io/gh/apache/airflow/pull/7451?src=pr&el=continue). > **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta) > `Δ = absolute (impact)`, `ø = not affected`, `? = missing data` > Powered by [Codecov](https://codecov.io/gh/apache/airflow/pull/7451?src=pr&el=footer). Last update [dbcd3d8...3ee2f6b](https://codecov.io/gh/apache/airflow/pull/7451?src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments). This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services