[GitHub] [airflow] feluelle commented on a change in pull request #5835: [AIRFLOW-5233] Fixed consistency in whitespace (tabs/eols) + common problems
feluelle commented on a change in pull request #5835: [AIRFLOW-5233] Fixed consistency in whitespace (tabs/eols) + common problems URL: https://github.com/apache/airflow/pull/5835#discussion_r314589917 ## File path: CONTRIBUTING.md ## @@ -701,6 +701,11 @@ lint-dockerfile Lint dockerfile mypy Run mypy pylint Run pylint flake8 Run flake8 +forbid-tabs Fails if tabs are used in the project +check-merge-conflict Checks if merge conflict is being committed +detect-private-key Detects if private key is added to the repository +end-of-file-fixerMake sure that there is an empty line at the end +mixed-line-endingDeteccts if mixed line ending is used (\r vs. \r\n) Review comment: ```suggestion mixed-line-endingDetects if mixed line ending is used (\r vs. \r\n) ``` ;) This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [airflow] feluelle commented on a change in pull request #5825: [AIRFLOW-5218] less polling for AWS Batch status
feluelle commented on a change in pull request #5825: [AIRFLOW-5218] less polling for AWS Batch status URL: https://github.com/apache/airflow/pull/5825#discussion_r314589328 ## File path: airflow/contrib/operators/awsbatch_operator.py ## @@ -105,6 +106,7 @@ def execute(self, context): self.jobId = response['jobId'] self.jobName = response['jobName'] +sleep(randint(10, 60)) Review comment: Thank you very much for the explanation. LGTM then :) This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Commented] (AIRFLOW-5234) Check for consistency and auto-add licences for .rst files
[
https://issues.apache.org/jira/browse/AIRFLOW-5234?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16908695#comment-16908695
]
ASF GitHub Bot commented on AIRFLOW-5234:
-
potiuk commented on pull request #5836: [AIRFLOW-5234] Rst files have
consistent, auto-added license
URL: https://github.com/apache/airflow/pull/5836
Make sure you have checked _all_ steps below.
### Jira
- [x] My PR addresses the following [Airflow
Jira](https://issues.apache.org/jira/browse/AIRFLOW/) issues and references
them in the PR title. For example, "\[AIRFLOW-XXX\] My Airflow PR"
- https://issues.apache.org/jira/browse/AIRFLOW-5234
### Description
- [x] Here are some details about my PR, including screenshots of any UI
changes:
### Tests
- [x] My PR adds the following unit tests __OR__ does not need testing for
this extremely good reason:
### Commits
- [x] My commits all reference Jira issues in their subject lines, and I
have squashed multiple commits if they address the same issue. In addition, my
commits follow the guidelines from "[How to write a good git commit
message](http://chris.beams.io/posts/git-commit/)":
1. Subject is separated from body by a blank line
1. Subject is limited to 50 characters (not including Jira issue reference)
1. Subject does not end with a period
1. Subject uses the imperative mood ("add", not "adding")
1. Body wraps at 72 characters
1. Body explains "what" and "why", not "how"
### Documentation
- [x] In case of new functionality, my PR adds documentation that describes
how to use it.
- All the public functions and the classes in the PR contain docstrings
that explain what it does
- If you implement backwards incompatible changes, please leave a note in
the [Updating.md](https://github.com/apache/airflow/blob/master/UPDATING.md) so
we can assign it to a appropriate release
### Code Quality
- [x] Passes `flake8`
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
> Check for consistency and auto-add licences for .rst files
> --
>
> Key: AIRFLOW-5234
> URL: https://issues.apache.org/jira/browse/AIRFLOW-5234
> Project: Apache Airflow
> Issue Type: Improvement
> Components: ci
>Affects Versions: 2.0.0
>Reporter: Jarek Potiuk
>Priority: Major
>
--
This message was sent by Atlassian JIRA
(v7.6.14#76016)
[GitHub] [airflow] potiuk opened a new pull request #5836: [AIRFLOW-5234] Rst files have consistent, auto-added license
potiuk opened a new pull request #5836: [AIRFLOW-5234] Rst files have
consistent, auto-added license
URL: https://github.com/apache/airflow/pull/5836
Make sure you have checked _all_ steps below.
### Jira
- [x] My PR addresses the following [Airflow
Jira](https://issues.apache.org/jira/browse/AIRFLOW/) issues and references
them in the PR title. For example, "\[AIRFLOW-XXX\] My Airflow PR"
- https://issues.apache.org/jira/browse/AIRFLOW-5234
### Description
- [x] Here are some details about my PR, including screenshots of any UI
changes:
### Tests
- [x] My PR adds the following unit tests __OR__ does not need testing for
this extremely good reason:
### Commits
- [x] My commits all reference Jira issues in their subject lines, and I
have squashed multiple commits if they address the same issue. In addition, my
commits follow the guidelines from "[How to write a good git commit
message](http://chris.beams.io/posts/git-commit/)":
1. Subject is separated from body by a blank line
1. Subject is limited to 50 characters (not including Jira issue reference)
1. Subject does not end with a period
1. Subject uses the imperative mood ("add", not "adding")
1. Body wraps at 72 characters
1. Body explains "what" and "why", not "how"
### Documentation
- [x] In case of new functionality, my PR adds documentation that describes
how to use it.
- All the public functions and the classes in the PR contain docstrings
that explain what it does
- If you implement backwards incompatible changes, please leave a note in
the [Updating.md](https://github.com/apache/airflow/blob/master/UPDATING.md) so
we can assign it to a appropriate release
### Code Quality
- [x] Passes `flake8`
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[jira] [Created] (AIRFLOW-5234) Check for consistency and auto-add licences for .rst files
Jarek Potiuk created AIRFLOW-5234: - Summary: Check for consistency and auto-add licences for .rst files Key: AIRFLOW-5234 URL: https://issues.apache.org/jira/browse/AIRFLOW-5234 Project: Apache Airflow Issue Type: Improvement Components: ci Affects Versions: 2.0.0 Reporter: Jarek Potiuk -- This message was sent by Atlassian JIRA (v7.6.14#76016)
[jira] [Commented] (AIRFLOW-5229) Add licence to all other file types
[ https://issues.apache.org/jira/browse/AIRFLOW-5229?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16908684#comment-16908684 ] ASF subversion and git services commented on AIRFLOW-5229: -- Commit 6688e1ee6990f57d274236e041f1e78db94e4d44 in airflow's branch refs/heads/v1-10-test from Jarek Potiuk [ https://gitbox.apache.org/repos/asf?p=airflow.git;h=6688e1e ] [AIRFLOW-5229] Consistent licences to all other files (#5831) (cherry picked from commit 3fac1bd04a3672a17b86f39050fa35f60458c346) > Add licence to all other file types > --- > > Key: AIRFLOW-5229 > URL: https://issues.apache.org/jira/browse/AIRFLOW-5229 > Project: Apache Airflow > Issue Type: Improvement > Components: ci >Affects Versions: 2.0.0 >Reporter: Jarek Potiuk >Priority: Major > Fix For: 1.10.5 > > > Consistent licences to all other file types -- This message was sent by Atlassian JIRA (v7.6.14#76016)
[jira] [Commented] (AIRFLOW-5225) Consistent licences can be added automatically for all JS files
[ https://issues.apache.org/jira/browse/AIRFLOW-5225?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16908685#comment-16908685 ] ASF subversion and git services commented on AIRFLOW-5225: -- Commit 1f6cbd07d78f99c17d6f638397337894100a in airflow's branch refs/heads/v1-10-test from Jarek Potiuk [ https://gitbox.apache.org/repos/asf?p=airflow.git;h=1f6cbd0 ] [AIRFLOW-5225] Consistent licence for all JS files (#5827) (cherry picked from commit 4d491f394e93943848d26fb06414308528d2ad30) > Consistent licences can be added automatically for all JS files > --- > > Key: AIRFLOW-5225 > URL: https://issues.apache.org/jira/browse/AIRFLOW-5225 > Project: Apache Airflow > Issue Type: Improvement > Components: ci >Affects Versions: 2.0.0 >Reporter: Jarek Potiuk >Priority: Major > Fix For: 1.10.5 > > -- This message was sent by Atlassian JIRA (v7.6.14#76016)
[jira] [Commented] (AIRFLOW-5233) Check for consistency in whitespace (tabs/eols) and common problems
[
https://issues.apache.org/jira/browse/AIRFLOW-5233?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16908677#comment-16908677
]
ASF GitHub Bot commented on AIRFLOW-5233:
-
potiuk commented on pull request #5835: [AIRFLOW-5233] Fixed consistency in
whitespace (tabs/eols) + common problems
URL: https://github.com/apache/airflow/pull/5835
…roblems
Make sure you have checked _all_ steps below.
### Jira
- [ ] My PR addresses the following [Airflow
Jira](https://issues.apache.org/jira/browse/AIRFLOW/) issues and references
them in the PR title. For example, "\[AIRFLOW-XXX\] My Airflow PR"
- https://issues.apache.org/jira/browse/AIRFLOW-5233
### Description
- [x] Here are some details about my PR, including screenshots of any UI
changes:
### Tests
- [x] My PR adds the following unit tests __OR__ does not need testing for
this extremely good reason:
### Commits
- [x] My commits all reference Jira issues in their subject lines, and I
have squashed multiple commits if they address the same issue. In addition, my
commits follow the guidelines from "[How to write a good git commit
message](http://chris.beams.io/posts/git-commit/)":
1. Subject is separated from body by a blank line
1. Subject is limited to 50 characters (not including Jira issue reference)
1. Subject does not end with a period
1. Subject uses the imperative mood ("add", not "adding")
1. Body wraps at 72 characters
1. Body explains "what" and "why", not "how"
### Documentation
- [x] In case of new functionality, my PR adds documentation that describes
how to use it.
- All the public functions and the classes in the PR contain docstrings
that explain what it does
- If you implement backwards incompatible changes, please leave a note in
the [Updating.md](https://github.com/apache/airflow/blob/master/UPDATING.md) so
we can assign it to a appropriate release
### Code Quality
- [x] Passes `flake8`
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
> Check for consistency in whitespace (tabs/eols) and common problems
> ---
>
> Key: AIRFLOW-5233
> URL: https://issues.apache.org/jira/browse/AIRFLOW-5233
> Project: Apache Airflow
> Issue Type: Improvement
> Components: ci
>Affects Versions: 1.10.4, 2.0.0
>Reporter: Jarek Potiuk
>Priority: Major
>
--
This message was sent by Atlassian JIRA
(v7.6.14#76016)
[GitHub] [airflow] potiuk opened a new pull request #5835: [AIRFLOW-5233] Fixed consistency in whitespace (tabs/eols) + common problems
potiuk opened a new pull request #5835: [AIRFLOW-5233] Fixed consistency in
whitespace (tabs/eols) + common problems
URL: https://github.com/apache/airflow/pull/5835
…roblems
Make sure you have checked _all_ steps below.
### Jira
- [ ] My PR addresses the following [Airflow
Jira](https://issues.apache.org/jira/browse/AIRFLOW/) issues and references
them in the PR title. For example, "\[AIRFLOW-XXX\] My Airflow PR"
- https://issues.apache.org/jira/browse/AIRFLOW-5233
### Description
- [x] Here are some details about my PR, including screenshots of any UI
changes:
### Tests
- [x] My PR adds the following unit tests __OR__ does not need testing for
this extremely good reason:
### Commits
- [x] My commits all reference Jira issues in their subject lines, and I
have squashed multiple commits if they address the same issue. In addition, my
commits follow the guidelines from "[How to write a good git commit
message](http://chris.beams.io/posts/git-commit/)":
1. Subject is separated from body by a blank line
1. Subject is limited to 50 characters (not including Jira issue reference)
1. Subject does not end with a period
1. Subject uses the imperative mood ("add", not "adding")
1. Body wraps at 72 characters
1. Body explains "what" and "why", not "how"
### Documentation
- [x] In case of new functionality, my PR adds documentation that describes
how to use it.
- All the public functions and the classes in the PR contain docstrings
that explain what it does
- If you implement backwards incompatible changes, please leave a note in
the [Updating.md](https://github.com/apache/airflow/blob/master/UPDATING.md) so
we can assign it to a appropriate release
### Code Quality
- [x] Passes `flake8`
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[jira] [Created] (AIRFLOW-5233) Check for consistency in whitespace (tabs/eols) and common problems
Jarek Potiuk created AIRFLOW-5233: - Summary: Check for consistency in whitespace (tabs/eols) and common problems Key: AIRFLOW-5233 URL: https://issues.apache.org/jira/browse/AIRFLOW-5233 Project: Apache Airflow Issue Type: Improvement Components: ci Affects Versions: 1.10.4, 2.0.0 Reporter: Jarek Potiuk -- This message was sent by Atlassian JIRA (v7.6.14#76016)
[jira] [Updated] (AIRFLOW-5206) All .md files should have all common licence, TOC (where applicable) + TODO.md should be removed
[ https://issues.apache.org/jira/browse/AIRFLOW-5206?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jarek Potiuk updated AIRFLOW-5206: -- Summary: All .md files should have all common licence, TOC (where applicable) + TODO.md should be removed (was: .md files should have all common licence) > All .md files should have all common licence, TOC (where applicable) + > TODO.md should be removed > > > Key: AIRFLOW-5206 > URL: https://issues.apache.org/jira/browse/AIRFLOW-5206 > Project: Apache Airflow > Issue Type: Improvement > Components: ci >Affects Versions: 2.0.0 >Reporter: Jarek Potiuk >Priority: Major > -- This message was sent by Atlassian JIRA (v7.6.14#76016)
[GitHub] [airflow] potiuk commented on issue #5786: [AIRFLOW-5170] Fix encoding pragmas, consistent licences for python files and related pylint fixes
potiuk commented on issue #5786: [AIRFLOW-5170] Fix encoding pragmas, consistent licences for python files and related pylint fixes URL: https://github.com/apache/airflow/pull/5786#issuecomment-521859932 All reported so far resolved. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [airflow] potiuk commented on a change in pull request #5828: [AIRFLOW-5226] Consistent licences for all jinja templates
potiuk commented on a change in pull request #5828: [AIRFLOW-5226] Consistent
licences for all jinja templates
URL: https://github.com/apache/airflow/pull/5828#discussion_r314562597
##
File path: airflow/www/templates/appbuilder/navbar_menu.html
##
@@ -1,3 +1,22 @@
+{#
+ Licensed to the Apache Software Foundation (ASF) under one
+ or more contributor license agreements. See the NOTICE file
+ distributed with this work for additional information
+ regarding copyright ownership. The ASF licenses this file
+ to you under the Apache License, Version 2.0 (the
+ "License"); you may not use this file except in compliance
+ with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+ Unless required by applicable law or agreed to in writing,
+ software distributed under the License is distributed on an
+ "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ KIND, either express or implied. See the License for the
+ specific language governing permissions and limitations
+ under the License.
+#}
+
{#
Review comment:
Same
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [airflow] potiuk commented on a change in pull request #5828: [AIRFLOW-5226] Consistent licences for all jinja templates
potiuk commented on a change in pull request #5828: [AIRFLOW-5226] Consistent
licences for all jinja templates
URL: https://github.com/apache/airflow/pull/5828#discussion_r314562628
##
File path: airflow/www/templates/appbuilder/navbar_right.html
##
@@ -1,3 +1,22 @@
+{#
+ Licensed to the Apache Software Foundation (ASF) under one
+ or more contributor license agreements. See the NOTICE file
+ distributed with this work for additional information
+ regarding copyright ownership. The ASF licenses this file
+ to you under the Apache License, Version 2.0 (the
+ "License"); you may not use this file except in compliance
+ with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+ Unless required by applicable law or agreed to in writing,
+ software distributed under the License is distributed on an
+ "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ KIND, either express or implied. See the License for the
+ specific language governing permissions and limitations
+ under the License.
+#}
+
{#
Review comment:
Same
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [airflow] potiuk commented on a change in pull request #5828: [AIRFLOW-5226] Consistent licences for all jinja templates
potiuk commented on a change in pull request #5828: [AIRFLOW-5226] Consistent
licences for all jinja templates
URL: https://github.com/apache/airflow/pull/5828#discussion_r314562511
##
File path: airflow/www/templates/appbuilder/index.html
##
@@ -1,3 +1,22 @@
+{#
+ Licensed to the Apache Software Foundation (ASF) under one
+ or more contributor license agreements. See the NOTICE file
+ distributed with this work for additional information
+ regarding copyright ownership. The ASF licenses this file
+ to you under the Apache License, Version 2.0 (the
+ "License"); you may not use this file except in compliance
+ with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+ Unless required by applicable law or agreed to in writing,
+ software distributed under the License is distributed on an
+ "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ KIND, either express or implied. See the License for the
+ specific language governing permissions and limitations
+ under the License.
+#}
+
{#
Review comment:
aww. missed it!
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[jira] [Commented] (AIRFLOW-5225) Consistent licences can be added automatically for all JS files
[ https://issues.apache.org/jira/browse/AIRFLOW-5225?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16908614#comment-16908614 ] ASF subversion and git services commented on AIRFLOW-5225: -- Commit cd04531f25708cc175c89af0a69586f2d63ca7a7 in airflow's branch refs/heads/v1-10-test from Jarek Potiuk [ https://gitbox.apache.org/repos/asf?p=airflow.git;h=cd04531 ] [AIRFLOW-5225] Consistent licence for all JS files (#5827) (cherry picked from commit 4d491f394e93943848d26fb06414308528d2ad30) > Consistent licences can be added automatically for all JS files > --- > > Key: AIRFLOW-5225 > URL: https://issues.apache.org/jira/browse/AIRFLOW-5225 > Project: Apache Airflow > Issue Type: Improvement > Components: ci >Affects Versions: 2.0.0 >Reporter: Jarek Potiuk >Priority: Major > Fix For: 1.10.5 > > -- This message was sent by Atlassian JIRA (v7.6.14#76016)
[GitHub] [airflow] potiuk commented on a change in pull request #5786: [AIRFLOW-5170] Fix encoding pragmas, consistent licences for python files and related pylint fixes
potiuk commented on a change in pull request #5786: [AIRFLOW-5170] Fix encoding pragmas, consistent licences for python files and related pylint fixes URL: https://github.com/apache/airflow/pull/5786#discussion_r314556794 ## File path: tests/kubernetes/kubernetes_request_factory/test_pod_request_factory.py ## @@ -62,7 +63,7 @@ def setUp(self): 'fsGroup': 2000, } ) -self.maxDiff = None +self.max_diff = None Review comment: Fixed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [airflow] potiuk commented on a change in pull request #5786: [AIRFLOW-5170] Fix encoding pragmas, consistent licences for python files and related pylint fixes
potiuk commented on a change in pull request #5786: [AIRFLOW-5170] Fix encoding pragmas, consistent licences for python files and related pylint fixes URL: https://github.com/apache/airflow/pull/5786#discussion_r314556526 ## File path: airflow/contrib/operators/aws_sqs_publish_operator.py ## @@ -16,29 +15,14 @@ # KIND, either express or implied. See the License for the # specific language governing permissions and limitations # under the License. - +"""Publish message to SQS queue""" from airflow.models import BaseOperator from airflow.utils.decorators import apply_defaults from airflow.contrib.hooks.aws_sqs_hook import SQSHook class SQSPublishOperator(BaseOperator): -""" -Publish message to a SQS queue. - -:param sqs_queue: The SQS queue url (templated) -:type sqs_queue: str -:param message_content: The message content (templated) -:type message_content: str -:param message_attributes: additional attributes for the message (default: None) -For details of the attributes parameter see :py:meth:`botocore.client.SQS.send_message` -:type message_attributes: dict -:param delay_seconds: message delay (templated) (default: 1 second) -:type delay_seconds: int -:param aws_conn_id: AWS connection id (default: aws_default) -:type aws_conn_id: str -""" - +"""Operator to publish message to AWS SQS queue """ Review comment: Right. Fixed few other places as well. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [airflow] codecov-io edited a comment on issue #5088: [AIRFLOW-4330] - Change _create_or_update_secret to work with python3
codecov-io edited a comment on issue #5088: [AIRFLOW-4330] - Change _create_or_update_secret to work with python3 URL: https://github.com/apache/airflow/pull/5088#issuecomment-482559987 # [Codecov](https://codecov.io/gh/apache/airflow/pull/5088?src=pr=h1) Report > :exclamation: No coverage uploaded for pull request base (`master@3ee2dcb`). [Click here to learn what that means](https://docs.codecov.io/docs/error-reference#section-missing-base-commit). > The diff coverage is `n/a`. [](https://codecov.io/gh/apache/airflow/pull/5088?src=pr=tree) ```diff @@Coverage Diff@@ ## master#5088 +/- ## = Coverage ? 76.81% = Files ? 463 Lines ?29757 Branches ?0 = Hits ?22859 Misses? 6898 Partials ?0 ``` | [Impacted Files](https://codecov.io/gh/apache/airflow/pull/5088?src=pr=tree) | Coverage Δ | | |---|---|---| | [airflow/contrib/executors/kubernetes\_executor.py](https://codecov.io/gh/apache/airflow/pull/5088/diff?src=pr=tree#diff-YWlyZmxvdy9jb250cmliL2V4ZWN1dG9ycy9rdWJlcm5ldGVzX2V4ZWN1dG9yLnB5) | `63.27% <ø> (ø)` | | -- [Continue to review full report at Codecov](https://codecov.io/gh/apache/airflow/pull/5088?src=pr=continue). > **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta) > `Δ = absolute (impact)`, `ø = not affected`, `? = missing data` > Powered by [Codecov](https://codecov.io/gh/apache/airflow/pull/5088?src=pr=footer). Last update [3ee2dcb...d725ba1](https://codecov.io/gh/apache/airflow/pull/5088?src=pr=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments). This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [airflow] potiuk commented on a change in pull request #5786: [AIRFLOW-5170] Fix encoding pragmas, consistent licences for python files and related pylint fixes
potiuk commented on a change in pull request #5786: [AIRFLOW-5170] Fix encoding pragmas, consistent licences for python files and related pylint fixes URL: https://github.com/apache/airflow/pull/5786#discussion_r314551942 ## File path: airflow/contrib/hooks/grpc_hook.py ## @@ -58,7 +61,7 @@ def get_conn(self): if auth_type == "NO_AUTH": channel = grpc.insecure_channel(base_url) -elif auth_type == "SSL" or auth_type == "TLS": +elif auth_type in ["SSL", "TLS"]: Review comment: Sure. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [airflow] potiuk commented on a change in pull request #5786: [AIRFLOW-5170] Fix encoding pragmas, consistent licences for python files and related pylint fixes
potiuk commented on a change in pull request #5786: [AIRFLOW-5170] Fix encoding pragmas, consistent licences for python files and related pylint fixes URL: https://github.com/apache/airflow/pull/5786#discussion_r314551685 ## File path: airflow/contrib/hooks/bigquery_hook.py ## @@ -1,5 +1,4 @@ -# -*- coding: utf-8 -*- # pylint: disable=too-many-lines -# +# -*- coding: utf-8 -*- Review comment: Right. Fixed :). This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Resolved] (AIRFLOW-5225) Consistent licences can be added automatically for all JS files
[ https://issues.apache.org/jira/browse/AIRFLOW-5225?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jarek Potiuk resolved AIRFLOW-5225. --- Resolution: Fixed Fix Version/s: 1.10.5 > Consistent licences can be added automatically for all JS files > --- > > Key: AIRFLOW-5225 > URL: https://issues.apache.org/jira/browse/AIRFLOW-5225 > Project: Apache Airflow > Issue Type: Improvement > Components: ci >Affects Versions: 2.0.0 >Reporter: Jarek Potiuk >Priority: Major > Fix For: 1.10.5 > > -- This message was sent by Atlassian JIRA (v7.6.14#76016)
[jira] [Commented] (AIRFLOW-5225) Consistent licences can be added automatically for all JS files
[ https://issues.apache.org/jira/browse/AIRFLOW-5225?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16908567#comment-16908567 ] ASF subversion and git services commented on AIRFLOW-5225: -- Commit 09b0df3294242df490dca9e57edea7ccb66f63fd in airflow's branch refs/heads/v1-10-test from Jarek Potiuk [ https://gitbox.apache.org/repos/asf?p=airflow.git;h=09b0df3 ] [AIRFLOW-5225] Consistent licence for all JS files (#5827) (cherry picked from commit 4d491f394e93943848d26fb06414308528d2ad30) > Consistent licences can be added automatically for all JS files > --- > > Key: AIRFLOW-5225 > URL: https://issues.apache.org/jira/browse/AIRFLOW-5225 > Project: Apache Airflow > Issue Type: Improvement > Components: ci >Affects Versions: 2.0.0 >Reporter: Jarek Potiuk >Priority: Major > Fix For: 1.10.5 > > -- This message was sent by Atlassian JIRA (v7.6.14#76016)
[jira] [Commented] (AIRFLOW-5225) Consistent licences can be added automatically for all JS files
[ https://issues.apache.org/jira/browse/AIRFLOW-5225?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16908564#comment-16908564 ] ASF subversion and git services commented on AIRFLOW-5225: -- Commit 4d491f394e93943848d26fb06414308528d2ad30 in airflow's branch refs/heads/master from Jarek Potiuk [ https://gitbox.apache.org/repos/asf?p=airflow.git;h=4d491f3 ] [AIRFLOW-5225] Consistent licence for all JS files (#5827) > Consistent licences can be added automatically for all JS files > --- > > Key: AIRFLOW-5225 > URL: https://issues.apache.org/jira/browse/AIRFLOW-5225 > Project: Apache Airflow > Issue Type: Improvement > Components: ci >Affects Versions: 2.0.0 >Reporter: Jarek Potiuk >Priority: Major > -- This message was sent by Atlassian JIRA (v7.6.14#76016)
[jira] [Commented] (AIRFLOW-5225) Consistent licences can be added automatically for all JS files
[ https://issues.apache.org/jira/browse/AIRFLOW-5225?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16908563#comment-16908563 ] ASF GitHub Bot commented on AIRFLOW-5225: - potiuk commented on pull request #5827: [AIRFLOW-5225] Consistent licence for all JS files URL: https://github.com/apache/airflow/pull/5827 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org > Consistent licences can be added automatically for all JS files > --- > > Key: AIRFLOW-5225 > URL: https://issues.apache.org/jira/browse/AIRFLOW-5225 > Project: Apache Airflow > Issue Type: Improvement > Components: ci >Affects Versions: 2.0.0 >Reporter: Jarek Potiuk >Priority: Major > -- This message was sent by Atlassian JIRA (v7.6.14#76016)
[GitHub] [airflow] potiuk merged pull request #5827: [AIRFLOW-5225] Consistent licence for all JS files
potiuk merged pull request #5827: [AIRFLOW-5225] Consistent licence for all JS files URL: https://github.com/apache/airflow/pull/5827 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Resolved] (AIRFLOW-5229) Add licence to all other file types
[ https://issues.apache.org/jira/browse/AIRFLOW-5229?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jarek Potiuk resolved AIRFLOW-5229. --- Resolution: Fixed Fix Version/s: 1.10.5 > Add licence to all other file types > --- > > Key: AIRFLOW-5229 > URL: https://issues.apache.org/jira/browse/AIRFLOW-5229 > Project: Apache Airflow > Issue Type: Improvement > Components: ci >Affects Versions: 2.0.0 >Reporter: Jarek Potiuk >Priority: Major > Fix For: 1.10.5 > > > Consistent licences to all other file types -- This message was sent by Atlassian JIRA (v7.6.14#76016)
[jira] [Commented] (AIRFLOW-5229) Add licence to all other file types
[ https://issues.apache.org/jira/browse/AIRFLOW-5229?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16908562#comment-16908562 ] ASF subversion and git services commented on AIRFLOW-5229: -- Commit 0eacebf239e044837b8d9b7f15297d72a3d43266 in airflow's branch refs/heads/v1-10-test from Jarek Potiuk [ https://gitbox.apache.org/repos/asf?p=airflow.git;h=0eacebf ] [AIRFLOW-5229] Consistent licences to all other files (#5831) (cherry picked from commit 3fac1bd04a3672a17b86f39050fa35f60458c346) > Add licence to all other file types > --- > > Key: AIRFLOW-5229 > URL: https://issues.apache.org/jira/browse/AIRFLOW-5229 > Project: Apache Airflow > Issue Type: Improvement > Components: ci >Affects Versions: 2.0.0 >Reporter: Jarek Potiuk >Priority: Major > Fix For: 1.10.5 > > > Consistent licences to all other file types -- This message was sent by Atlassian JIRA (v7.6.14#76016)
[jira] [Commented] (AIRFLOW-5229) Add licence to all other file types
[ https://issues.apache.org/jira/browse/AIRFLOW-5229?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16908561#comment-16908561 ] ASF subversion and git services commented on AIRFLOW-5229: -- Commit 3fac1bd04a3672a17b86f39050fa35f60458c346 in airflow's branch refs/heads/master from Jarek Potiuk [ https://gitbox.apache.org/repos/asf?p=airflow.git;h=3fac1bd ] [AIRFLOW-5229] Consistent licences to all other files (#5831) > Add licence to all other file types > --- > > Key: AIRFLOW-5229 > URL: https://issues.apache.org/jira/browse/AIRFLOW-5229 > Project: Apache Airflow > Issue Type: Improvement > Components: ci >Affects Versions: 2.0.0 >Reporter: Jarek Potiuk >Priority: Major > > Consistent licences to all other file types -- This message was sent by Atlassian JIRA (v7.6.14#76016)
[GitHub] [airflow] potiuk merged pull request #5831: [AIRFLOW-5229] Consistent licences to all other files
potiuk merged pull request #5831: [AIRFLOW-5229] Consistent licences to all other files URL: https://github.com/apache/airflow/pull/5831 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Commented] (AIRFLOW-5229) Add licence to all other file types
[ https://issues.apache.org/jira/browse/AIRFLOW-5229?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16908560#comment-16908560 ] ASF GitHub Bot commented on AIRFLOW-5229: - potiuk commented on pull request #5831: [AIRFLOW-5229] Consistent licences to all other files URL: https://github.com/apache/airflow/pull/5831 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org > Add licence to all other file types > --- > > Key: AIRFLOW-5229 > URL: https://issues.apache.org/jira/browse/AIRFLOW-5229 > Project: Apache Airflow > Issue Type: Improvement > Components: ci >Affects Versions: 2.0.0 >Reporter: Jarek Potiuk >Priority: Major > > Consistent licences to all other file types -- This message was sent by Atlassian JIRA (v7.6.14#76016)
[jira] [Resolved] (AIRFLOW-5227) Consistent licences for all .sql files
[ https://issues.apache.org/jira/browse/AIRFLOW-5227?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jarek Potiuk resolved AIRFLOW-5227. --- Resolution: Fixed Fix Version/s: 1.10.5 > Consistent licences for all .sql files > -- > > Key: AIRFLOW-5227 > URL: https://issues.apache.org/jira/browse/AIRFLOW-5227 > Project: Apache Airflow > Issue Type: Improvement > Components: ci >Affects Versions: 2.0.0 >Reporter: Jarek Potiuk >Priority: Major > Fix For: 1.10.5 > > -- This message was sent by Atlassian JIRA (v7.6.14#76016)
[jira] [Commented] (AIRFLOW-5227) Consistent licences for all .sql files
[ https://issues.apache.org/jira/browse/AIRFLOW-5227?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16908559#comment-16908559 ] ASF subversion and git services commented on AIRFLOW-5227: -- Commit 8218e769ec372aa6793e23990732c21af3ff4f42 in airflow's branch refs/heads/v1-10-test from Jarek Potiuk [ https://gitbox.apache.org/repos/asf?p=airflow.git;h=8218e76 ] [AIRFLOW-5227] Consistent licence for .sql files (#5829) (cherry picked from commit 2ea2c539d8b2caf395bc1e0adfa2ce715f1617f7) > Consistent licences for all .sql files > -- > > Key: AIRFLOW-5227 > URL: https://issues.apache.org/jira/browse/AIRFLOW-5227 > Project: Apache Airflow > Issue Type: Improvement > Components: ci >Affects Versions: 2.0.0 >Reporter: Jarek Potiuk >Priority: Major > -- This message was sent by Atlassian JIRA (v7.6.14#76016)
[GitHub] [airflow] potiuk merged pull request #5829: [AIRFLOW-5227] Consistent licence for .sql files
potiuk merged pull request #5829: [AIRFLOW-5227] Consistent licence for .sql files URL: https://github.com/apache/airflow/pull/5829 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Commented] (AIRFLOW-5227) Consistent licences for all .sql files
[ https://issues.apache.org/jira/browse/AIRFLOW-5227?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16908557#comment-16908557 ] ASF GitHub Bot commented on AIRFLOW-5227: - potiuk commented on pull request #5829: [AIRFLOW-5227] Consistent licence for .sql files URL: https://github.com/apache/airflow/pull/5829 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org > Consistent licences for all .sql files > -- > > Key: AIRFLOW-5227 > URL: https://issues.apache.org/jira/browse/AIRFLOW-5227 > Project: Apache Airflow > Issue Type: Improvement > Components: ci >Affects Versions: 2.0.0 >Reporter: Jarek Potiuk >Priority: Major > -- This message was sent by Atlassian JIRA (v7.6.14#76016)
[jira] [Commented] (AIRFLOW-5227) Consistent licences for all .sql files
[ https://issues.apache.org/jira/browse/AIRFLOW-5227?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16908558#comment-16908558 ] ASF subversion and git services commented on AIRFLOW-5227: -- Commit 2ea2c539d8b2caf395bc1e0adfa2ce715f1617f7 in airflow's branch refs/heads/master from Jarek Potiuk [ https://gitbox.apache.org/repos/asf?p=airflow.git;h=2ea2c53 ] [AIRFLOW-5227] Consistent licence for .sql files (#5829) > Consistent licences for all .sql files > -- > > Key: AIRFLOW-5227 > URL: https://issues.apache.org/jira/browse/AIRFLOW-5227 > Project: Apache Airflow > Issue Type: Improvement > Components: ci >Affects Versions: 2.0.0 >Reporter: Jarek Potiuk >Priority: Major > -- This message was sent by Atlassian JIRA (v7.6.14#76016)
[GitHub] [airflow] milton0825 commented on issue #5498: [AIRFLOW-4509] SubDagOperator using scheduler instead of backfill
milton0825 commented on issue #5498: [AIRFLOW-4509] SubDagOperator using scheduler instead of backfill URL: https://github.com/apache/airflow/pull/5498#issuecomment-521825390 We can probably include this PR as part of our next release but I am not sure how the release timeline looks like. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [airflow] mik-laj commented on issue #5498: [AIRFLOW-4509] SubDagOperator using scheduler instead of backfill
mik-laj commented on issue #5498: [AIRFLOW-4509] SubDagOperator using scheduler instead of backfill URL: https://github.com/apache/airflow/pull/5498#issuecomment-521821005 it will be very difficult. Every releasr is a lot of work.it is difficult to make a separate release for one PR. Why don't you make your own private release with one change? it shouldn't be difficult. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [airflow] mik-laj commented on a change in pull request #5786: [AIRFLOW-5170] Fix encoding pragmas, consistent licences for python files and related pylint fixes
mik-laj commented on a change in pull request #5786: [AIRFLOW-5170] Fix encoding pragmas, consistent licences for python files and related pylint fixes URL: https://github.com/apache/airflow/pull/5786#discussion_r314521808 ## File path: tests/kubernetes/kubernetes_request_factory/test_pod_request_factory.py ## @@ -62,7 +63,7 @@ def setUp(self): 'fsGroup': 2000, } ) -self.maxDiff = None +self.max_diff = None Review comment: It doesn't work properly if it is written in a different way. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [airflow] mik-laj commented on a change in pull request #5786: [AIRFLOW-5170] Fix encoding pragmas, consistent licences for python files and related pylint fixes
mik-laj commented on a change in pull request #5786: [AIRFLOW-5170] Fix encoding pragmas, consistent licences for python files and related pylint fixes URL: https://github.com/apache/airflow/pull/5786#discussion_r314521724 ## File path: tests/kubernetes/kubernetes_request_factory/test_pod_request_factory.py ## @@ -62,7 +63,7 @@ def setUp(self): 'fsGroup': 2000, } ) -self.maxDiff = None +self.max_diff = None Review comment: ```suggestion ``` This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [airflow] mik-laj commented on a change in pull request #5786: [AIRFLOW-5170] Fix encoding pragmas, consistent licences for python files and related pylint fixes
mik-laj commented on a change in pull request #5786: [AIRFLOW-5170] Fix encoding pragmas, consistent licences for python files and related pylint fixes URL: https://github.com/apache/airflow/pull/5786#discussion_r314521061 ## File path: airflow/contrib/operators/aws_sqs_publish_operator.py ## @@ -16,29 +15,14 @@ # KIND, either express or implied. See the License for the # specific language governing permissions and limitations # under the License. - +"""Publish message to SQS queue""" from airflow.models import BaseOperator from airflow.utils.decorators import apply_defaults from airflow.contrib.hooks.aws_sqs_hook import SQSHook class SQSPublishOperator(BaseOperator): -""" -Publish message to a SQS queue. - -:param sqs_queue: The SQS queue url (templated) -:type sqs_queue: str -:param message_content: The message content (templated) -:type message_content: str -:param message_attributes: additional attributes for the message (default: None) -For details of the attributes parameter see :py:meth:`botocore.client.SQS.send_message` -:type message_attributes: dict -:param delay_seconds: message delay (templated) (default: 1 second) -:type delay_seconds: int -:param aws_conn_id: AWS connection id (default: aws_default) -:type aws_conn_id: str -""" - +"""Operator to publish message to AWS SQS queue """ Review comment: We should have docstring only in one place - constructor or class. Otherwise the documentation will not display correctly. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Commented] (AIRFLOW-5220) Easy form to create airflow dags
[ https://issues.apache.org/jira/browse/AIRFLOW-5220?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16908516#comment-16908516 ] Kamil Bregula commented on AIRFLOW-5220: Are you planning to work on this feature? > Easy form to create airflow dags > > > Key: AIRFLOW-5220 > URL: https://issues.apache.org/jira/browse/AIRFLOW-5220 > Project: Apache Airflow > Issue Type: New Feature > Components: DAG, database >Affects Versions: 1.10.5 >Reporter: huangyan >Assignee: huangyan >Priority: Minor > Labels: DAG, database, easy > Original Estimate: 336h > Remaining Estimate: 336h > > The airflow usage threshold is higher and the user must write a Python dag > file. However, many users don't write Python, they want to create dags > directly from forms. -- This message was sent by Atlassian JIRA (v7.6.14#76016)
[jira] [Commented] (AIRFLOW-5228) support statements for Airflow
[ https://issues.apache.org/jira/browse/AIRFLOW-5228?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16908513#comment-16908513 ] Kamil Bregula commented on AIRFLOW-5228: In Airflow, deployment of the environment is a very big challenge. Doing it correctly reduces the number of problems. If you are unfamiliar with this project, I recommend you consider outsourcing it to someone. > support statements for Airflow > --- > > Key: AIRFLOW-5228 > URL: https://issues.apache.org/jira/browse/AIRFLOW-5228 > Project: Apache Airflow > Issue Type: Task > Components: scheduler >Affects Versions: 1.10.3 >Reporter: Ramkumar >Priority: Major > > Hallo I am working on an Airflow POC. I understand Airflow is an Open source > project and i am able to create Jiras. > However my customer want to get the support statements for Airflow. > a) Can you guide me for getting the support statement? > b) SLAs, response times for the jiras ? > c) other Support Level Agreements? > d) when a version considered end-of-life & retired? > e) Is support provided with Cloud services like google composer ? > f) For bug fixes patches are provided ? How can we patch the bug fixes? > g) Can you provide any community forums or groups which are open for > discussions on Airflow? > > -- This message was sent by Atlassian JIRA (v7.6.14#76016)
[jira] [Closed] (AIRFLOW-5228) support statements for Airflow
[ https://issues.apache.org/jira/browse/AIRFLOW-5228?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kamil Bregula closed AIRFLOW-5228. -- Resolution: Invalid Jira is used to report bugs and requests for new features. This is not a place for conversations. > support statements for Airflow > --- > > Key: AIRFLOW-5228 > URL: https://issues.apache.org/jira/browse/AIRFLOW-5228 > Project: Apache Airflow > Issue Type: Task > Components: scheduler >Affects Versions: 1.10.3 >Reporter: Ramkumar >Priority: Major > > Hallo I am working on an Airflow POC. I understand Airflow is an Open source > project and i am able to create Jiras. > However my customer want to get the support statements for Airflow. > a) Can you guide me for getting the support statement? > b) SLAs, response times for the jiras ? > c) other Support Level Agreements? > d) when a version considered end-of-life & retired? > e) Is support provided with Cloud services like google composer ? > f) For bug fixes patches are provided ? How can we patch the bug fixes? > g) Can you provide any community forums or groups which are open for > discussions on Airflow? > > -- This message was sent by Atlassian JIRA (v7.6.14#76016)
[jira] [Commented] (AIRFLOW-4285) Current TI DEPS are confusing and not up to date
[ https://issues.apache.org/jira/browse/AIRFLOW-4285?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16908509#comment-16908509 ] ASF GitHub Bot commented on AIRFLOW-4285: - KevinYang21 commented on pull request #5079: [AIRFLOW-4285] Update task dependency context defination and usage URL: https://github.com/apache/airflow/pull/5079 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org > Current TI DEPS are confusing and not up to date > > > Key: AIRFLOW-4285 > URL: https://issues.apache.org/jira/browse/AIRFLOW-4285 > Project: Apache Airflow > Issue Type: Improvement >Reporter: Kevin Yang >Assignee: Kevin Yang >Priority: Minor > > The name of the TI DEPS are not representative, not used efficiently and not > up to date. -- This message was sent by Atlassian JIRA (v7.6.14#76016)
[GitHub] [airflow] KevinYang21 merged pull request #5079: [AIRFLOW-4285] Update task dependency context defination and usage
KevinYang21 merged pull request #5079: [AIRFLOW-4285] Update task dependency context defination and usage URL: https://github.com/apache/airflow/pull/5079 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Commented] (AIRFLOW-5228) support statements for Airflow
[
https://issues.apache.org/jira/browse/AIRFLOW-5228?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16908508#comment-16908508
]
Kamil Bregula commented on AIRFLOW-5228:
{quote}> a) Can you guide me for getting the support statement?{quote}
Technical support is provided by external companies. I'm work on Polidea
Company and my work is associated with resolving ticket for integration with
Google Cloud Platform.
I invite you to contact:
https://www.polidea.com/services/cloud-computing
{quote}> b) SLAs, response times for the jiras ?{quote}
There are no SLAs for Jira public ticket. You must contact external company
that will provide you with technical support.
{quote}> c) other Support Level Agreements?{quote}
N/A
{quote}> d) when a version considered end-of-life & retired?{quote}
Airflow does not have the LTS version. We recommend upgrading to the latest
version that includes the latest patches.
{quote}> e) Is support provided with Cloud services like google composer?
{quote}
Cloud Composer is a service that facilitates application deployment and
management. Use of this service is not eligible for technical support
Support is provided by Google Proffessional Service (consulting) and Google
Support
https://cloud.google.com/consulting/
https://cloud.google.com/support/
{quote}f) For bug fixes patches are provided ? How can we patch the bug
fixes?{quote}
All patches are available in the repository.
Feel free to send/take changes directly to/from apache/airflow repo.
https://github.com/apache/airflow/blob/master/CONTRIBUTING.md
{quote}> g) Can you provide any community forums or groups which are open for
discussions on Airflow?{quote}
Community have a Slack channel and user/dev mailing list. Contact to my company
is possible via the email address: he...@polidea.com
> support statements for Airflow
> ---
>
> Key: AIRFLOW-5228
> URL: https://issues.apache.org/jira/browse/AIRFLOW-5228
> Project: Apache Airflow
> Issue Type: Task
> Components: scheduler
>Affects Versions: 1.10.3
>Reporter: Ramkumar
>Priority: Major
>
> Hallo I am working on an Airflow POC. I understand Airflow is an Open source
> project and i am able to create Jiras.
> However my customer want to get the support statements for Airflow.
> a) Can you guide me for getting the support statement?
> b) SLAs, response times for the jiras ?
> c) other Support Level Agreements?
> d) when a version considered end-of-life & retired?
> e) Is support provided with Cloud services like google composer ?
> f) For bug fixes patches are provided ? How can we patch the bug fixes?
> g) Can you provide any community forums or groups which are open for
> discussions on Airflow?
>
>
--
This message was sent by Atlassian JIRA
(v7.6.14#76016)
[jira] [Updated] (AIRFLOW-5229) Add licence to all other file types
[ https://issues.apache.org/jira/browse/AIRFLOW-5229?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kamil Bregula updated AIRFLOW-5229: --- Summary: Add licence to all other file types (was: Add licence to all otehr file types) > Add licence to all other file types > --- > > Key: AIRFLOW-5229 > URL: https://issues.apache.org/jira/browse/AIRFLOW-5229 > Project: Apache Airflow > Issue Type: Improvement > Components: ci >Affects Versions: 2.0.0 >Reporter: Jarek Potiuk >Priority: Major > > Consistent licences to all other file types -- This message was sent by Atlassian JIRA (v7.6.14#76016)
[jira] [Updated] (AIRFLOW-5224) gcs_to_bq.GoogleCloudStorageToBigQueryOperator - Specify Encoding for BQ ingestion
[ https://issues.apache.org/jira/browse/AIRFLOW-5224?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kamil Bregula updated AIRFLOW-5224: --- Component/s: gcp > gcs_to_bq.GoogleCloudStorageToBigQueryOperator - Specify Encoding for BQ > ingestion > -- > > Key: AIRFLOW-5224 > URL: https://issues.apache.org/jira/browse/AIRFLOW-5224 > Project: Apache Airflow > Issue Type: Bug > Components: DAG, gcp >Affects Versions: 1.10.0 > Environment: airflow software platform >Reporter: Anand Kumar >Priority: Blocker > > Hi, > The current business project we are enabling has been built completely on GCP > components with composer with airflow being one of the key process. We have > built various data pipelines using airflow for multiple work-streams where > data is being ingested from gcs bucket to Big query. > Based on the recent updates on Google BQ infra end, there seems to be some > tightened validations on UTF-8 characters which has resulted in mutiple > failures of our existing business process. > On further analysis we found out that while ingesting data to BQ from a > Google bucket the encoding needs to be explicitly specified going forward but > the below operator currently doesn't supply any params to specify explicit > encoding > _*gcs_to_bq.GoogleCloudStorageToBigQueryOperator*_ > Could someone please treat this as a priority and help us with a fix to > bring us back in BAU mode > -- This message was sent by Atlassian JIRA (v7.6.14#76016)
[GitHub] [airflow] richgong edited a comment on issue #5498: [AIRFLOW-4509] SubDagOperator using scheduler instead of backfill
richgong edited a comment on issue #5498: [AIRFLOW-4509] SubDagOperator using scheduler instead of backfill URL: https://github.com/apache/airflow/pull/5498#issuecomment-521805775 @milton0825 @potiuk @Fokko @kaxil @ashb @feng-tao Can we get a new release with this feature included? https://github.com/apache/airflow/releases We're using SubDags for our new experimentation pipelines at Instacart, and this would be really helpful to us. Thank you! This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [airflow] richgong edited a comment on issue #5498: [AIRFLOW-4509] SubDagOperator using scheduler instead of backfill
richgong edited a comment on issue #5498: [AIRFLOW-4509] SubDagOperator using scheduler instead of backfill URL: https://github.com/apache/airflow/pull/5498#issuecomment-521805775 @milton0825 @potiuk @Fokko @kaxil @ashb Can we get a new release with this feature included? https://github.com/apache/airflow/releases We're using SubDags for our new experimentation pipelines at Instacart, and this would be really helpful to us. Thank you! This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [airflow] richgong commented on issue #5498: [AIRFLOW-4509] SubDagOperator using scheduler instead of backfill
richgong commented on issue #5498: [AIRFLOW-4509] SubDagOperator using scheduler instead of backfill URL: https://github.com/apache/airflow/pull/5498#issuecomment-521805775 @milton0825 @potiuk @Fokko @kaxil Can we get a new release with this feature included? https://github.com/apache/airflow/releases We're using SubDags for our new experimentation pipelines at Instacart, and this would be really helpful to us. Thank you! This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [airflow] darrenleeweber commented on issue #5825: [AIRFLOW-5218] less polling for AWS Batch status
darrenleeweber commented on issue #5825: [AIRFLOW-5218] less polling for AWS Batch status URL: https://github.com/apache/airflow/pull/5825#issuecomment-521802371 Most of the CI builds are OK but one of them timed-out and I don't have access to restart it - https://travis-ci.org/apache/airflow/builds/572411318 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Updated] (AIRFLOW-5228) support statements for Airflow
[ https://issues.apache.org/jira/browse/AIRFLOW-5228?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ramkumar updated AIRFLOW-5228: -- Description: Hallo I am working on an Airflow POC. I understand Airflow is an Open source project and i am able to create Jiras. However my customer want to get the support statements for Airflow. a) Can you guide me for getting the support statement? b) SLAs, response times for the jiras ? c) other Support Level Agreements? d) when a version considered end-of-life & retired? e) Is support provided with Cloud services like google composer ? f) For bug fixes patches are provided ? How can we patch the bug fixes? g) Can you provide any community forums or groups which are open for discussions on Airflow? was: Hallo I am working on an Airflow POC. I understand Airflow is an Open source project and i can create Jiras. However my customer want to get the support statements. Can you provide the support statement? a) SLAs, response times for the jiras ? b) Any other Support Level Agreements? c) when a version considered end-of-life & retired? d) is support provided with Cloud Managed services like google composer > support statements for Airflow > --- > > Key: AIRFLOW-5228 > URL: https://issues.apache.org/jira/browse/AIRFLOW-5228 > Project: Apache Airflow > Issue Type: Task > Components: scheduler >Affects Versions: 1.10.3 >Reporter: Ramkumar >Priority: Major > > Hallo I am working on an Airflow POC. I understand Airflow is an Open source > project and i am able to create Jiras. > However my customer want to get the support statements for Airflow. > a) Can you guide me for getting the support statement? > b) SLAs, response times for the jiras ? > c) other Support Level Agreements? > d) when a version considered end-of-life & retired? > e) Is support provided with Cloud services like google composer ? > f) For bug fixes patches are provided ? How can we patch the bug fixes? > g) Can you provide any community forums or groups which are open for > discussions on Airflow? > > -- This message was sent by Atlassian JIRA (v7.6.14#76016)
[GitHub] [airflow] mik-laj opened a new pull request #5834: [AIRFLOW-XXX] Group executors in one section
mik-laj opened a new pull request #5834: [AIRFLOW-XXX] Group executors in one
section
URL: https://github.com/apache/airflow/pull/5834
Make sure you have checked _all_ steps below.
### Jira
- [ ] My PR addresses the following [Airflow
Jira](https://issues.apache.org/jira/browse/AIRFLOW/) issues and references
them in the PR title. For example, "\[AIRFLOW-XXX\] My Airflow PR"
- https://issues.apache.org/jira/browse/AIRFLOW-XXX
- In case you are fixing a typo in the documentation you can prepend your
commit with \[AIRFLOW-XXX\], code changes always need a Jira issue.
- In case you are proposing a fundamental code change, you need to create
an Airflow Improvement Proposal
([AIP](https://cwiki.apache.org/confluence/display/AIRFLOW/Airflow+Improvements+Proposals)).
- In case you are adding a dependency, check if the license complies with
the [ASF 3rd Party License
Policy](https://www.apache.org/legal/resolved.html#category-x).
### Description
- [ ] Here are some details about my PR, including screenshots of any UI
changes:
### Tests
- [ ] My PR adds the following unit tests __OR__ does not need testing for
this extremely good reason:
### Commits
- [ ] My commits all reference Jira issues in their subject lines, and I
have squashed multiple commits if they address the same issue. In addition, my
commits follow the guidelines from "[How to write a good git commit
message](http://chris.beams.io/posts/git-commit/)":
1. Subject is separated from body by a blank line
1. Subject is limited to 50 characters (not including Jira issue reference)
1. Subject does not end with a period
1. Subject uses the imperative mood ("add", not "adding")
1. Body wraps at 72 characters
1. Body explains "what" and "why", not "how"
### Documentation
- [ ] In case of new functionality, my PR adds documentation that describes
how to use it.
- All the public functions and the classes in the PR contain docstrings
that explain what it does
- If you implement backwards incompatible changes, please leave a note in
the [Updating.md](https://github.com/apache/airflow/blob/master/UPDATING.md) so
we can assign it to a appropriate release
### Code Quality
- [ ] Passes `flake8`
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [airflow] ashb commented on issue #5743: [AIRFLOW-5088][AIP-24] Persisting serialized DAG in DB for webserver scalability
ashb commented on issue #5743: [AIRFLOW-5088][AIP-24] Persisting serialized DAG in DB for webserver scalability URL: https://github.com/apache/airflow/pull/5743#issuecomment-521767547 I'm testing out a few changes on my branch based off this one: https://github.com/coufon/airflow/compare/dagcached...ashb:dagcached This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Commented] (AIRFLOW-5224) gcs_to_bq.GoogleCloudStorageToBigQueryOperator - Specify Encoding for BQ ingestion
[ https://issues.apache.org/jira/browse/AIRFLOW-5224?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16908420#comment-16908420 ] Eric Walisko commented on AIRFLOW-5224: --- I represent another team that is being impacted by this issue. Is there any documentation on the encoding specifications required by bigquery [~anandkumar_b]? > gcs_to_bq.GoogleCloudStorageToBigQueryOperator - Specify Encoding for BQ > ingestion > -- > > Key: AIRFLOW-5224 > URL: https://issues.apache.org/jira/browse/AIRFLOW-5224 > Project: Apache Airflow > Issue Type: Bug > Components: DAG >Affects Versions: 1.10.0 > Environment: airflow software platform >Reporter: Anand Kumar >Priority: Blocker > > Hi, > The current business project we are enabling has been built completely on GCP > components with composer with airflow being one of the key process. We have > built various data pipelines using airflow for multiple work-streams where > data is being ingested from gcs bucket to Big query. > Based on the recent updates on Google BQ infra end, there seems to be some > tightened validations on UTF-8 characters which has resulted in mutiple > failures of our existing business process. > On further analysis we found out that while ingesting data to BQ from a > Google bucket the encoding needs to be explicitly specified going forward but > the below operator currently doesn't supply any params to specify explicit > encoding > _*gcs_to_bq.GoogleCloudStorageToBigQueryOperator*_ > Could someone please treat this as a priority and help us with a fix to > bring us back in BAU mode > -- This message was sent by Atlassian JIRA (v7.6.14#76016)
[jira] [Created] (AIRFLOW-5232) Add pagerdutyhook to the airflow default plugins
sri ram chegondi created AIRFLOW-5232: - Summary: Add pagerdutyhook to the airflow default plugins Key: AIRFLOW-5232 URL: https://issues.apache.org/jira/browse/AIRFLOW-5232 Project: Apache Airflow Issue Type: Wish Components: plugins Affects Versions: 1.10.5 Reporter: sri ram chegondi Create pagerdutyhook in the airflow plugins, to support pagerduty. -- This message was sent by Atlassian JIRA (v7.6.14#76016)
[jira] [Updated] (AIRFLOW-5232) Create pagerdutyhook in the airflow plugins, to support pagerduty.
[ https://issues.apache.org/jira/browse/AIRFLOW-5232?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] sri ram chegondi updated AIRFLOW-5232: -- Summary: Create pagerdutyhook in the airflow plugins, to support pagerduty. (was: Add pagerdutyhook to the airflow default plugins) > Create pagerdutyhook in the airflow plugins, to support pagerduty. > -- > > Key: AIRFLOW-5232 > URL: https://issues.apache.org/jira/browse/AIRFLOW-5232 > Project: Apache Airflow > Issue Type: Wish > Components: plugins >Affects Versions: 1.10.5 >Reporter: sri ram chegondi >Priority: Major > > Create pagerdutyhook in the airflow plugins, to support pagerduty. -- This message was sent by Atlassian JIRA (v7.6.14#76016)
[GitHub] [airflow] pgagnon commented on issue #5824: [AIRFLOW-5215] Add sidecar containers support to Pod class
pgagnon commented on issue #5824: [AIRFLOW-5215] Add sidecar containers support to Pod class URL: https://github.com/apache/airflow/pull/5824#issuecomment-521748700 @cBournhonesque At this time I am trying keep this as API-compatible as possible. Changing the `Pod` interface to accept a list of containers would indeed be preferable but this would be a breaking change since the main container is built by combining other arguments (i.e. `image`). This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [airflow] mik-laj opened a new pull request #5833: [AIRFLOW-XXX] Remove "Setup Test Environment using MySQL" article
mik-laj opened a new pull request #5833: [AIRFLOW-XXX] Remove "Setup Test
Environment using MySQL" article
URL: https://github.com/apache/airflow/pull/5833
The CI environment has changed, so the page is no longer valid.
https://github.com/apache/airflow/pull/4938
---
Make sure you have checked _all_ steps below.
### Jira
- [ ] My PR addresses the following [Airflow
Jira](https://issues.apache.org/jira/browse/AIRFLOW/) issues and references
them in the PR title. For example, "\[AIRFLOW-XXX\] My Airflow PR"
- https://issues.apache.org/jira/browse/AIRFLOW-XXX
- In case you are fixing a typo in the documentation you can prepend your
commit with \[AIRFLOW-XXX\], code changes always need a Jira issue.
- In case you are proposing a fundamental code change, you need to create
an Airflow Improvement Proposal
([AIP](https://cwiki.apache.org/confluence/display/AIRFLOW/Airflow+Improvements+Proposals)).
- In case you are adding a dependency, check if the license complies with
the [ASF 3rd Party License
Policy](https://www.apache.org/legal/resolved.html#category-x).
### Description
- [ ] Here are some details about my PR, including screenshots of any UI
changes:
### Tests
- [ ] My PR adds the following unit tests __OR__ does not need testing for
this extremely good reason:
### Commits
- [ ] My commits all reference Jira issues in their subject lines, and I
have squashed multiple commits if they address the same issue. In addition, my
commits follow the guidelines from "[How to write a good git commit
message](http://chris.beams.io/posts/git-commit/)":
1. Subject is separated from body by a blank line
1. Subject is limited to 50 characters (not including Jira issue reference)
1. Subject does not end with a period
1. Subject uses the imperative mood ("add", not "adding")
1. Body wraps at 72 characters
1. Body explains "what" and "why", not "how"
### Documentation
- [ ] In case of new functionality, my PR adds documentation that describes
how to use it.
- All the public functions and the classes in the PR contain docstrings
that explain what it does
- If you implement backwards incompatible changes, please leave a note in
the [Updating.md](https://github.com/apache/airflow/blob/master/UPDATING.md) so
we can assign it to a appropriate release
### Code Quality
- [ ] Passes `flake8`
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [airflow] cBournhonesque commented on issue #5824: [AIRFLOW-5215] Add sidecar containers support to Pod class
cBournhonesque commented on issue #5824: [AIRFLOW-5215] Add sidecar containers support to Pod class URL: https://github.com/apache/airflow/pull/5824#issuecomment-521745204 Do we want to treat sidecar containers differently then the main container? Shouldn't the API allow for the creation of multiple containers? (for instance by passing lists for each argument) This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Created] (AIRFLOW-5231) S3Hook delete fails with over 1000 keys
Silviu Tantos created AIRFLOW-5231:
--
Summary: S3Hook delete fails with over 1000 keys
Key: AIRFLOW-5231
URL: https://issues.apache.org/jira/browse/AIRFLOW-5231
Project: Apache Airflow
Issue Type: Bug
Components: aws, hooks
Affects Versions: 2.0.0
Reporter: Silviu Tantos
Error raised:
{noformat}
botocore.exceptions.ClientError: An error occurred (MalformedXML) when calling
the DeleteObjects operation: The XML you provided was not well-formed or did
not validate against our published schema{noformat}
See also: https://github.com/spotify/luigi/issues/2511
--
This message was sent by Atlassian JIRA
(v7.6.14#76016)
[GitHub] [airflow] darrenleeweber commented on a change in pull request #5825: [AIRFLOW-5218] less polling for AWS Batch status
darrenleeweber commented on a change in pull request #5825: [AIRFLOW-5218] less polling for AWS Batch status URL: https://github.com/apache/airflow/pull/5825#discussion_r314426834 ## File path: airflow/contrib/operators/awsbatch_operator.py ## @@ -105,6 +106,7 @@ def execute(self, context): self.jobId = response['jobId'] self.jobName = response['jobName'] +sleep(randint(10, 60)) Review comment: amended commit pushed This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [airflow] darrenleeweber commented on a change in pull request #5825: [AIRFLOW-5218] less polling for AWS Batch status
darrenleeweber commented on a change in pull request #5825: [AIRFLOW-5218] less
polling for AWS Batch status
URL: https://github.com/apache/airflow/pull/5825#discussion_r314423519
##
File path: airflow/contrib/operators/awsbatch_operator.py
##
@@ -105,6 +106,7 @@ def execute(self, context):
self.jobId = response['jobId']
self.jobName = response['jobName']
+sleep(randint(10, 60))
Review comment:
- There is already a log, i.e. `self.log.info('AWS Batch Job started: %s',
response)`
- The use of a random interval decreases the chances of exceeding an AWS API
throttle limit
- concern about it is reasonable, it's there as an extra measure to avoid
throttle limits when there are 100's or 1000's of concurrent tasks all trying
to start up batch jobs at about the same time, so a relatively wide window was
selected in the hope that it can stagger the API calls randomly within about a
minute, assuming the API throttle limits are assessed per second; the initial
delay of at least 10 sec allows the batch job a bit of time to spin up, maybe
it could be longer, but there it is.
- this call to the random-sleep should move to within the wait-method (in a
follow-up amended commit)
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[jira] [Commented] (AIRFLOW-5230) Refactor template file loading functions
[
https://issues.apache.org/jira/browse/AIRFLOW-5230?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16908349#comment-16908349
]
ASF GitHub Bot commented on AIRFLOW-5230:
-
BasPH commented on pull request #5832: [AIRFLOW-5230] Refactor template file
loading functionality
URL: https://github.com/apache/airflow/pull/5832
Make sure you have checked _all_ steps below.
### Jira
- [x] My PR addresses the following [Airflow
Jira](https://issues.apache.org/jira/browse/AIRFLOW/) issues and references
them in the PR title. For example, "\[AIRFLOW-XXX\] My Airflow PR"
- https://issues.apache.org/jira/browse/AIRFLOW-5230
- In case you are fixing a typo in the documentation you can prepend your
commit with \[AIRFLOW-XXX\], code changes always need a Jira issue.
- In case you are proposing a fundamental code change, you need to create
an Airflow Improvement Proposal
([AIP](https://cwiki.apache.org/confluence/display/AIRFLOW/Airflow+Improvements+Proposals)).
- In case you are adding a dependency, check if the license complies with
the [ASF 3rd Party License
Policy](https://www.apache.org/legal/resolved.html#category-x).
### Description
- [x] Here are some details about my PR, including screenshots of any UI
changes:
Following https://issues.apache.org/jira/browse/AIRFLOW-4835, I've also
refactored the template file loading functionality to:
- Avoid building multiple Jinja environments
- Avoid code duplication around `env.loader.get_source`
- Rename `resolve_template_files` to `load_template_files` which follows the
Jinja naming
- Support other collection types for storing template file paths (tuple,
namedtuple, dict and set)
- Add tests for all types
- Move related test from `core.py` to `test_baseoperator.py`
- Adhere to 110 char line length
- Add Pydoc
- Add typing
### Tests
- [x] My PR adds the following unit tests __OR__ does not need testing for
this extremely good reason:
Few tests in `test_baseoperator.py` to test providing template filepaths in
various data structures (dict, list, str, etc...), and moved a test from
`core.py` to `test_baseoperator.py`.
### Commits
- [ ] My commits all reference Jira issues in their subject lines, and I
have squashed multiple commits if they address the same issue. In addition, my
commits follow the guidelines from "[How to write a good git commit
message](http://chris.beams.io/posts/git-commit/)":
1. Subject is separated from body by a blank line
1. Subject is limited to 50 characters (not including Jira issue reference)
1. Subject does not end with a period
1. Subject uses the imperative mood ("add", not "adding")
1. Body wraps at 72 characters
1. Body explains "what" and "why", not "how"
### Documentation
- [ ] In case of new functionality, my PR adds documentation that describes
how to use it.
- All the public functions and the classes in the PR contain docstrings
that explain what it does
- If you implement backwards incompatible changes, please leave a note in
the [Updating.md](https://github.com/apache/airflow/blob/master/UPDATING.md) so
we can assign it to a appropriate release
### Code Quality
- [ ] Passes `flake8`
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
> Refactor template file loading functions
>
>
> Key: AIRFLOW-5230
> URL: https://issues.apache.org/jira/browse/AIRFLOW-5230
> Project: Apache Airflow
> Issue Type: Improvement
> Components: core
>Affects Versions: 2.0.0
>Reporter: Bas Harenslak
>Priority: Major
>
--
This message was sent by Atlassian JIRA
(v7.6.14#76016)
[GitHub] [airflow] BasPH opened a new pull request #5832: [AIRFLOW-5230] Refactor template file loading functionality
BasPH opened a new pull request #5832: [AIRFLOW-5230] Refactor template file
loading functionality
URL: https://github.com/apache/airflow/pull/5832
Make sure you have checked _all_ steps below.
### Jira
- [x] My PR addresses the following [Airflow
Jira](https://issues.apache.org/jira/browse/AIRFLOW/) issues and references
them in the PR title. For example, "\[AIRFLOW-XXX\] My Airflow PR"
- https://issues.apache.org/jira/browse/AIRFLOW-5230
- In case you are fixing a typo in the documentation you can prepend your
commit with \[AIRFLOW-XXX\], code changes always need a Jira issue.
- In case you are proposing a fundamental code change, you need to create
an Airflow Improvement Proposal
([AIP](https://cwiki.apache.org/confluence/display/AIRFLOW/Airflow+Improvements+Proposals)).
- In case you are adding a dependency, check if the license complies with
the [ASF 3rd Party License
Policy](https://www.apache.org/legal/resolved.html#category-x).
### Description
- [x] Here are some details about my PR, including screenshots of any UI
changes:
Following https://issues.apache.org/jira/browse/AIRFLOW-4835, I've also
refactored the template file loading functionality to:
- Avoid building multiple Jinja environments
- Avoid code duplication around `env.loader.get_source`
- Rename `resolve_template_files` to `load_template_files` which follows the
Jinja naming
- Support other collection types for storing template file paths (tuple,
namedtuple, dict and set)
- Add tests for all types
- Move related test from `core.py` to `test_baseoperator.py`
- Adhere to 110 char line length
- Add Pydoc
- Add typing
### Tests
- [x] My PR adds the following unit tests __OR__ does not need testing for
this extremely good reason:
Few tests in `test_baseoperator.py` to test providing template filepaths in
various data structures (dict, list, str, etc...), and moved a test from
`core.py` to `test_baseoperator.py`.
### Commits
- [ ] My commits all reference Jira issues in their subject lines, and I
have squashed multiple commits if they address the same issue. In addition, my
commits follow the guidelines from "[How to write a good git commit
message](http://chris.beams.io/posts/git-commit/)":
1. Subject is separated from body by a blank line
1. Subject is limited to 50 characters (not including Jira issue reference)
1. Subject does not end with a period
1. Subject uses the imperative mood ("add", not "adding")
1. Body wraps at 72 characters
1. Body explains "what" and "why", not "how"
### Documentation
- [ ] In case of new functionality, my PR adds documentation that describes
how to use it.
- All the public functions and the classes in the PR contain docstrings
that explain what it does
- If you implement backwards incompatible changes, please leave a note in
the [Updating.md](https://github.com/apache/airflow/blob/master/UPDATING.md) so
we can assign it to a appropriate release
### Code Quality
- [ ] Passes `flake8`
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[jira] [Created] (AIRFLOW-5230) Refactor template file loading functions
Bas Harenslak created AIRFLOW-5230: -- Summary: Refactor template file loading functions Key: AIRFLOW-5230 URL: https://issues.apache.org/jira/browse/AIRFLOW-5230 Project: Apache Airflow Issue Type: Improvement Components: core Affects Versions: 2.0.0 Reporter: Bas Harenslak -- This message was sent by Atlassian JIRA (v7.6.14#76016)
[GitHub] [airflow] StevenReitsma commented on issue #5645: [AIRFLOW-5027] Get logs from CloudWatch for ECSOperators
StevenReitsma commented on issue #5645: [AIRFLOW-5027] Get logs from CloudWatch for ECSOperators URL: https://github.com/apache/airflow/pull/5645#issuecomment-521714189 PTAL @ashb This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [airflow] akki commented on issue #5489: [AIRFLOW-4843] Allow orchestration via Docker Swarm (SwarmOperator)
akki commented on issue #5489: [AIRFLOW-4843] Allow orchestration via Docker Swarm (SwarmOperator) URL: https://github.com/apache/airflow/pull/5489#issuecomment-521706461 @acordiner That's great. Would you like to list down your thoughts/ideas as a ticket in [JIRA](https://issues.apache.org/jira/projects/AIRFLOW/issues/AIRFLOW-5224?filter=allopenissues). Airflow accepts PRs/commits only being tracked on their JIRA. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Updated] (AIRFLOW-5224) gcs_to_bq.GoogleCloudStorageToBigQueryOperator - Specify Encoding for BQ ingestion
[ https://issues.apache.org/jira/browse/AIRFLOW-5224?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Anand Kumar updated AIRFLOW-5224: - Description: Hi, The current business project we are enabling has been built completely on GCP components with composer with airflow being one of the key process. We have built various data pipelines using airflow for multiple work-streams where data is being ingested from gcs bucket to Big query. Based on the recent updates on Google BQ infra end, there seems to be some tightened validations on UTF-8 characters which has resulted in mutiple failures of our existing business process. On further analysis we found out that while ingesting data to BQ from a Google bucket the encoding needs to be explicitly specified going forward but the below operator currently doesn't supply any params to specify explicit encoding _*gcs_to_bq.GoogleCloudStorageToBigQueryOperator*_ Could someone please treat this as a priority and help us with a fix to bring us back in BAU mode was: Hi, The current business project we are enabling has been built completely on GCP components with composer with airflow being one of the key process. We have built various data pipelines using airflow for multiple work-streams where data is being ingested from gcs bucket to Big query. Based on the recent updates on Google BQ infra end, there seems to be some tightened validations on UTF-8 characters which has resulted in mutiple failures of our existing business process. On further analysis we found out that while ingesting data to BQ from a Google bucket the encoding needs to be explicitly specified going forward but the below operator currently doesn't supply any params to specify explicit encoding _*gcs_to_bq.GoogleCloudStorageToBigQueryOperator*_ Could you please treat this as a priority and help us with a fix to bring us back in BAU mode > gcs_to_bq.GoogleCloudStorageToBigQueryOperator - Specify Encoding for BQ > ingestion > -- > > Key: AIRFLOW-5224 > URL: https://issues.apache.org/jira/browse/AIRFLOW-5224 > Project: Apache Airflow > Issue Type: Bug > Components: DAG >Affects Versions: 1.10.0 > Environment: airflow software platform >Reporter: Anand Kumar >Priority: Blocker > > Hi, > The current business project we are enabling has been built completely on GCP > components with composer with airflow being one of the key process. We have > built various data pipelines using airflow for multiple work-streams where > data is being ingested from gcs bucket to Big query. > Based on the recent updates on Google BQ infra end, there seems to be some > tightened validations on UTF-8 characters which has resulted in mutiple > failures of our existing business process. > On further analysis we found out that while ingesting data to BQ from a > Google bucket the encoding needs to be explicitly specified going forward but > the below operator currently doesn't supply any params to specify explicit > encoding > _*gcs_to_bq.GoogleCloudStorageToBigQueryOperator*_ > Could someone please treat this as a priority and help us with a fix to > bring us back in BAU mode > -- This message was sent by Atlassian JIRA (v7.6.14#76016)
[GitHub] [airflow] feluelle commented on a change in pull request #5825: [AIRFLOW-5218] less polling for AWS Batch status
feluelle commented on a change in pull request #5825: [AIRFLOW-5218] less polling for AWS Batch status URL: https://github.com/apache/airflow/pull/5825#discussion_r314349833 ## File path: airflow/contrib/operators/awsbatch_operator.py ## @@ -105,6 +106,7 @@ def execute(self, context): self.jobId = response['jobId'] self.jobName = response['jobName'] +sleep(randint(10, 60)) Review comment: Does it really need to be random? - also one `self.log` before this would be nice imo This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [airflow] ashb commented on a change in pull request #5743: [AIRFLOW-5088][AIP-24] Persisting serialized DAG in DB for webserver scalability
ashb commented on a change in pull request #5743: [AIRFLOW-5088][AIP-24] Persisting serialized DAG in DB for webserver scalability URL: https://github.com/apache/airflow/pull/5743#discussion_r314335561 ## File path: airflow/models/serialized_dag.py ## @@ -0,0 +1,173 @@ +# -*- coding: utf-8 -*- +# +# Licensed to the Apache Software Foundation (ASF) under one +# or more contributor license agreements. See the NOTICE file +# distributed with this work for additional information +# regarding copyright ownership. The ASF licenses this file +# to you under the Apache License, Version 2.0 (the +# "License"); you may not use this file except in compliance +# with the License. You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, +# software distributed under the License is distributed on an +# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY +# KIND, either express or implied. See the License for the +# specific language governing permissions and limitations +# under the License. + +"""Serialzed DAG table in database.""" + +import hashlib +from datetime import timedelta +from typing import Any, Dict, List, Optional, TYPE_CHECKING + +from sqlalchemy import Column, Index, Integer, String, Text, and_ +from sqlalchemy.sql import exists + +from airflow.models.base import Base, ID_LEN +from airflow.utils import db, timezone +from airflow.utils.log.logging_mixin import LoggingMixin +from airflow.utils.sqlalchemy import UtcDateTime + +if TYPE_CHECKING: +from airflow.dag.serialization.serialized_dag import SerializedDAG # noqa: F401, E501; # pylint: disable=cyclic-import +from airflow.models import DAG # noqa: F401; # pylint: disable=cyclic-import + +log = LoggingMixin().log + + +class SerializedDagModel(Base): Review comment: ```suggestion class SerializedDag(Base): ``` And then call this `SerialzedDagModel` at import time if we need to distinguish this. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Updated] (AIRFLOW-5228) support statements for Airflow
[ https://issues.apache.org/jira/browse/AIRFLOW-5228?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ramkumar updated AIRFLOW-5228: -- Description: Hallo I am working on an Airflow POC. I understand Airflow is an Open source project and i can create Jiras. However my customer want to get the support statements. Can you provide the support statement? a) SLAs, response times for the jiras ? b) Any other Support Level Agreements? c) when a version considered end-of-life & retired? d) is support provided with Cloud Managed services like google composer was: Hallo I am working on an Airflow POC. I understand Airflow is an Open source project and i can create Jiras. However my customer want to get the support statements. Can you provide the support statement? a) SLAs, response times for the jiras ? b) Any other Support Level Agreements? c) when a version considered end-of-life & retired? > support statements for Airflow > --- > > Key: AIRFLOW-5228 > URL: https://issues.apache.org/jira/browse/AIRFLOW-5228 > Project: Apache Airflow > Issue Type: Task > Components: scheduler >Affects Versions: 1.10.3 >Reporter: Ramkumar >Priority: Major > > Hallo I am working on an Airflow POC. I understand Airflow is an Open source > project and i can create Jiras. > However my customer want to get the support statements. Can you provide the > support statement? > a) SLAs, response times for the jiras ? > b) Any other Support Level Agreements? > c) when a version considered end-of-life & retired? > d) is support provided with Cloud Managed services like google composer > > -- This message was sent by Atlassian JIRA (v7.6.14#76016)
[jira] [Updated] (AIRFLOW-5228) support statements for Airflow
[ https://issues.apache.org/jira/browse/AIRFLOW-5228?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ramkumar updated AIRFLOW-5228: -- Summary: support statements for Airflow (was: Questions on Airflow licensing and support) > support statements for Airflow > --- > > Key: AIRFLOW-5228 > URL: https://issues.apache.org/jira/browse/AIRFLOW-5228 > Project: Apache Airflow > Issue Type: Task > Components: scheduler >Affects Versions: 1.10.3 >Reporter: Ramkumar >Priority: Major > > Hallo I am working on an Airflow POC. I understand Airflow is an Open source > project and i can create Jiras. > However my customer want to get the support statements. Can you provide the > support statement? > a) SLAs, response times for the jiras ? > b) Any other Support Level Agreements? > c) when a version considered end-of-life & retired? > > -- This message was sent by Atlassian JIRA (v7.6.14#76016)
[GitHub] [airflow] potiuk opened a new pull request #5831: [AIRFLOW-5229] Consistent licences to all other files
potiuk opened a new pull request #5831: [AIRFLOW-5229] Consistent licences to
all other files
URL: https://github.com/apache/airflow/pull/5831
Make sure you have checked _all_ steps below.
### Jira
- [x] My PR addresses the following [Airflow
Jira](https://issues.apache.org/jira/browse/AIRFLOW/) issues and references
them in the PR title. For example, "\[AIRFLOW-XXX\] My Airflow PR"
- https://issues.apache.org/jira/browse/AIRFLOW-5229
### Description
- [x] Here are some details about my PR, including screenshots of any UI
changes:
### Tests
- [x] My PR adds the following unit tests __OR__ does not need testing for
this extremely good reason:
### Commits
- [x] My commits all reference Jira issues in their subject lines, and I
have squashed multiple commits if they address the same issue. In addition, my
commits follow the guidelines from "[How to write a good git commit
message](http://chris.beams.io/posts/git-commit/)":
1. Subject is separated from body by a blank line
1. Subject is limited to 50 characters (not including Jira issue reference)
1. Subject does not end with a period
1. Subject uses the imperative mood ("add", not "adding")
1. Body wraps at 72 characters
1. Body explains "what" and "why", not "how"
### Documentation
- [x] In case of new functionality, my PR adds documentation that describes
how to use it.
- All the public functions and the classes in the PR contain docstrings
that explain what it does
- If you implement backwards incompatible changes, please leave a note in
the [Updating.md](https://github.com/apache/airflow/blob/master/UPDATING.md) so
we can assign it to a appropriate release
### Code Quality
- [x] Passes `flake8`
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[jira] [Commented] (AIRFLOW-5229) Add licence to all otehr file types
[
https://issues.apache.org/jira/browse/AIRFLOW-5229?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16908092#comment-16908092
]
ASF GitHub Bot commented on AIRFLOW-5229:
-
potiuk commented on pull request #5831: [AIRFLOW-5229] Consistent licences to
all other files
URL: https://github.com/apache/airflow/pull/5831
Make sure you have checked _all_ steps below.
### Jira
- [x] My PR addresses the following [Airflow
Jira](https://issues.apache.org/jira/browse/AIRFLOW/) issues and references
them in the PR title. For example, "\[AIRFLOW-XXX\] My Airflow PR"
- https://issues.apache.org/jira/browse/AIRFLOW-5229
### Description
- [x] Here are some details about my PR, including screenshots of any UI
changes:
### Tests
- [x] My PR adds the following unit tests __OR__ does not need testing for
this extremely good reason:
### Commits
- [x] My commits all reference Jira issues in their subject lines, and I
have squashed multiple commits if they address the same issue. In addition, my
commits follow the guidelines from "[How to write a good git commit
message](http://chris.beams.io/posts/git-commit/)":
1. Subject is separated from body by a blank line
1. Subject is limited to 50 characters (not including Jira issue reference)
1. Subject does not end with a period
1. Subject uses the imperative mood ("add", not "adding")
1. Body wraps at 72 characters
1. Body explains "what" and "why", not "how"
### Documentation
- [x] In case of new functionality, my PR adds documentation that describes
how to use it.
- All the public functions and the classes in the PR contain docstrings
that explain what it does
- If you implement backwards incompatible changes, please leave a note in
the [Updating.md](https://github.com/apache/airflow/blob/master/UPDATING.md) so
we can assign it to a appropriate release
### Code Quality
- [x] Passes `flake8`
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
> Add licence to all otehr file types
> ---
>
> Key: AIRFLOW-5229
> URL: https://issues.apache.org/jira/browse/AIRFLOW-5229
> Project: Apache Airflow
> Issue Type: Improvement
> Components: ci
>Affects Versions: 2.0.0
>Reporter: Jarek Potiuk
>Priority: Major
>
> Consistent licences to all other file types
--
This message was sent by Atlassian JIRA
(v7.6.14#76016)
[jira] [Created] (AIRFLOW-5229) Add licence to all otehr file types
Jarek Potiuk created AIRFLOW-5229: - Summary: Add licence to all otehr file types Key: AIRFLOW-5229 URL: https://issues.apache.org/jira/browse/AIRFLOW-5229 Project: Apache Airflow Issue Type: Improvement Components: ci Affects Versions: 2.0.0 Reporter: Jarek Potiuk Consistent licences to all other file types -- This message was sent by Atlassian JIRA (v7.6.14#76016)
[jira] [Created] (AIRFLOW-5228) Questions on Airflow licensing and support
Ramkumar created AIRFLOW-5228: - Summary: Questions on Airflow licensing and support Key: AIRFLOW-5228 URL: https://issues.apache.org/jira/browse/AIRFLOW-5228 Project: Apache Airflow Issue Type: Task Components: scheduler Affects Versions: 1.10.3 Reporter: Ramkumar Hallo I am working on an Airflow POC. I understand Airflow is an Open source project and i can create Jiras. However my customer want to get the support statements. Can you provide the support statement? a) SLAs, response times for the jiras ? b) Any other Support Level Agreements? c) when a version considered end-of-life & retired? -- This message was sent by Atlassian JIRA (v7.6.14#76016)
[jira] [Resolved] (AIRFLOW-4952) Remove unused arguments in tests
[ https://issues.apache.org/jira/browse/AIRFLOW-4952?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kaxil Naik resolved AIRFLOW-4952. - Resolution: Fixed Fix Version/s: 2.0.0 > Remove unused arguments in tests > > > Key: AIRFLOW-4952 > URL: https://issues.apache.org/jira/browse/AIRFLOW-4952 > Project: Apache Airflow > Issue Type: Sub-task > Components: core >Affects Versions: 1.10.3 >Reporter: Kamil >Priority: Major > Fix For: 2.0.0 > > -- This message was sent by Atlassian JIRA (v7.6.14#76016)
[jira] [Commented] (AIRFLOW-5161) Add pre-commit hooks to run static checks for only changed files
[ https://issues.apache.org/jira/browse/AIRFLOW-5161?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16908063#comment-16908063 ] ASF subversion and git services commented on AIRFLOW-5161: -- Commit 3f390db97070228726cc7b062f1fdfe21544199f in airflow's branch refs/heads/v1-10-test from Jarek Potiuk [ https://gitbox.apache.org/repos/asf?p=airflow.git;h=3f390db ] [AIRFLOW-5161] Static checks are run automatically in pre-commit hooks (#5777) (cherry-picked from 70e937a8d8ff308a9fb9055ceb7ef2c034200b36) > Add pre-commit hooks to run static checks for only changed files > > > Key: AIRFLOW-5161 > URL: https://issues.apache.org/jira/browse/AIRFLOW-5161 > Project: Apache Airflow > Issue Type: Improvement > Components: ci >Affects Versions: 2.0.0 >Reporter: Jarek Potiuk >Priority: Major > Fix For: 1.10.5 > > -- This message was sent by Atlassian JIRA (v7.6.14#76016)
[GitHub] [airflow] potiuk merged pull request #5830: [AIRFLOW-XXX] Remove duplicate lines from CONTRIBUTING.md
potiuk merged pull request #5830: [AIRFLOW-XXX] Remove duplicate lines from CONTRIBUTING.md URL: https://github.com/apache/airflow/pull/5830 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [airflow] kaxil opened a new pull request #5830: [AIRFLOW-XXX] Remove duplicate lines from CONTRIBUTING.md
kaxil opened a new pull request #5830: [AIRFLOW-XXX] Remove duplicate lines
from CONTRIBUTING.md
URL: https://github.com/apache/airflow/pull/5830
Make sure you have checked _all_ steps below.
### Jira
- [x] My PR addresses the following [Airflow
Jira](https://issues.apache.org/jira/browse/AIRFLOW/) issues and references
them in the PR title. For example, "\[AIRFLOW-XXX\] My Airflow PR"
### Description
- [x] Here are some details about my PR, including screenshots of any UI
changes:
### Tests
- [x] My PR adds the following unit tests __OR__ does not need testing for
this extremely good reason:
Doc change
### Commits
- [x] My commits all reference Jira issues in their subject lines, and I
have squashed multiple commits if they address the same issue. In addition, my
commits follow the guidelines from "[How to write a good git commit
message](http://chris.beams.io/posts/git-commit/)":
1. Subject is separated from body by a blank line
1. Subject is limited to 50 characters (not including Jira issue reference)
1. Subject does not end with a period
1. Subject uses the imperative mood ("add", not "adding")
1. Body wraps at 72 characters
1. Body explains "what" and "why", not "how"
### Documentation
- [x] In case of new functionality, my PR adds documentation that describes
how to use it.
- All the public functions and the classes in the PR contain docstrings
that explain what it does
- If you implement backwards incompatible changes, please leave a note in
the [Updating.md](https://github.com/apache/airflow/blob/master/UPDATING.md) so
we can assign it to a appropriate release
### Code Quality
- [x] Passes `flake8`
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[jira] [Commented] (AIRFLOW-5227) Consistent licences for all .sql files
[
https://issues.apache.org/jira/browse/AIRFLOW-5227?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16908061#comment-16908061
]
ASF GitHub Bot commented on AIRFLOW-5227:
-
potiuk commented on pull request #5829: [AIRFLOW-5227] Consistent licence for
.sql files
URL: https://github.com/apache/airflow/pull/5829
Make sure you have checked _all_ steps below.
### Jira
- [x] My PR addresses the following [Airflow
Jira](https://issues.apache.org/jira/browse/AIRFLOW/) issues and references
them in the PR title. For example, "\[AIRFLOW-XXX\] My Airflow PR"
- https://issues.apache.org/jira/browse/AIRFLOW-5227
### Description
- [x] Here are some details about my PR, including screenshots of any UI
changes:
### Tests
- [x] My PR adds the following unit tests __OR__ does not need testing for
this extremely good reason:
### Commits
- [x] My commits all reference Jira issues in their subject lines, and I
have squashed multiple commits if they address the same issue. In addition, my
commits follow the guidelines from "[How to write a good git commit
message](http://chris.beams.io/posts/git-commit/)":
1. Subject is separated from body by a blank line
1. Subject is limited to 50 characters (not including Jira issue reference)
1. Subject does not end with a period
1. Subject uses the imperative mood ("add", not "adding")
1. Body wraps at 72 characters
1. Body explains "what" and "why", not "how"
### Documentation
- [x] In case of new functionality, my PR adds documentation that describes
how to use it.
- All the public functions and the classes in the PR contain docstrings
that explain what it does
- If you implement backwards incompatible changes, please leave a note in
the [Updating.md](https://github.com/apache/airflow/blob/master/UPDATING.md) so
we can assign it to a appropriate release
### Code Quality
- [x] Passes `flake8`
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
> Consistent licences for all .sql files
> --
>
> Key: AIRFLOW-5227
> URL: https://issues.apache.org/jira/browse/AIRFLOW-5227
> Project: Apache Airflow
> Issue Type: Improvement
> Components: ci
>Affects Versions: 2.0.0
>Reporter: Jarek Potiuk
>Priority: Major
>
--
This message was sent by Atlassian JIRA
(v7.6.14#76016)
[GitHub] [airflow] potiuk opened a new pull request #5829: [AIRFLOW-5227] Consistent licence for .sql files
potiuk opened a new pull request #5829: [AIRFLOW-5227] Consistent licence for
.sql files
URL: https://github.com/apache/airflow/pull/5829
Make sure you have checked _all_ steps below.
### Jira
- [x] My PR addresses the following [Airflow
Jira](https://issues.apache.org/jira/browse/AIRFLOW/) issues and references
them in the PR title. For example, "\[AIRFLOW-XXX\] My Airflow PR"
- https://issues.apache.org/jira/browse/AIRFLOW-5227
### Description
- [x] Here are some details about my PR, including screenshots of any UI
changes:
### Tests
- [x] My PR adds the following unit tests __OR__ does not need testing for
this extremely good reason:
### Commits
- [x] My commits all reference Jira issues in their subject lines, and I
have squashed multiple commits if they address the same issue. In addition, my
commits follow the guidelines from "[How to write a good git commit
message](http://chris.beams.io/posts/git-commit/)":
1. Subject is separated from body by a blank line
1. Subject is limited to 50 characters (not including Jira issue reference)
1. Subject does not end with a period
1. Subject uses the imperative mood ("add", not "adding")
1. Body wraps at 72 characters
1. Body explains "what" and "why", not "how"
### Documentation
- [x] In case of new functionality, my PR adds documentation that describes
how to use it.
- All the public functions and the classes in the PR contain docstrings
that explain what it does
- If you implement backwards incompatible changes, please leave a note in
the [Updating.md](https://github.com/apache/airflow/blob/master/UPDATING.md) so
we can assign it to a appropriate release
### Code Quality
- [x] Passes `flake8`
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[jira] [Created] (AIRFLOW-5227) Consistent licences for all .sql files
Jarek Potiuk created AIRFLOW-5227: - Summary: Consistent licences for all .sql files Key: AIRFLOW-5227 URL: https://issues.apache.org/jira/browse/AIRFLOW-5227 Project: Apache Airflow Issue Type: Improvement Components: ci Affects Versions: 2.0.0 Reporter: Jarek Potiuk -- This message was sent by Atlassian JIRA (v7.6.14#76016)
[GitHub] [airflow] mik-laj commented on a change in pull request #5828: [AIRFLOW-5226] Consistent licences for all jinja templates
mik-laj commented on a change in pull request #5828: [AIRFLOW-5226] Consistent
licences for all jinja templates
URL: https://github.com/apache/airflow/pull/5828#discussion_r314296261
##
File path: airflow/www/templates/appbuilder/navbar.html
##
@@ -1,3 +1,22 @@
+{#
Review comment:
Duplicated Apache's licence.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [airflow] mik-laj commented on a change in pull request #5828: [AIRFLOW-5226] Consistent licences for all jinja templates
mik-laj commented on a change in pull request #5828: [AIRFLOW-5226] Consistent
licences for all jinja templates
URL: https://github.com/apache/airflow/pull/5828#discussion_r314296288
##
File path: airflow/www/templates/appbuilder/navbar_menu.html
##
@@ -1,3 +1,22 @@
+{#
+ Licensed to the Apache Software Foundation (ASF) under one
+ or more contributor license agreements. See the NOTICE file
+ distributed with this work for additional information
+ regarding copyright ownership. The ASF licenses this file
+ to you under the Apache License, Version 2.0 (the
+ "License"); you may not use this file except in compliance
+ with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+ Unless required by applicable law or agreed to in writing,
+ software distributed under the License is distributed on an
+ "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ KIND, either express or implied. See the License for the
+ specific language governing permissions and limitations
+ under the License.
+#}
+
{#
Review comment:
Duplicated Apache's licence.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [airflow] mik-laj commented on a change in pull request #5828: [AIRFLOW-5226] Consistent licences for all jinja templates
mik-laj commented on a change in pull request #5828: [AIRFLOW-5226] Consistent
licences for all jinja templates
URL: https://github.com/apache/airflow/pull/5828#discussion_r314296336
##
File path: airflow/www/templates/appbuilder/navbar_right.html
##
@@ -1,3 +1,22 @@
+{#
+ Licensed to the Apache Software Foundation (ASF) under one
+ or more contributor license agreements. See the NOTICE file
+ distributed with this work for additional information
+ regarding copyright ownership. The ASF licenses this file
+ to you under the Apache License, Version 2.0 (the
+ "License"); you may not use this file except in compliance
+ with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+ Unless required by applicable law or agreed to in writing,
+ software distributed under the License is distributed on an
+ "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ KIND, either express or implied. See the License for the
+ specific language governing permissions and limitations
+ under the License.
+#}
+
{#
Review comment:
Duplicated Apache's licence.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [airflow] mik-laj commented on a change in pull request #5828: [AIRFLOW-5226] Consistent licences for all jinja templates
mik-laj commented on a change in pull request #5828: [AIRFLOW-5226] Consistent
licences for all jinja templates
URL: https://github.com/apache/airflow/pull/5828#discussion_r314296200
##
File path: airflow/www/templates/appbuilder/index.html
##
@@ -1,3 +1,22 @@
+{#
+ Licensed to the Apache Software Foundation (ASF) under one
+ or more contributor license agreements. See the NOTICE file
+ distributed with this work for additional information
+ regarding copyright ownership. The ASF licenses this file
+ to you under the Apache License, Version 2.0 (the
+ "License"); you may not use this file except in compliance
+ with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+ Unless required by applicable law or agreed to in writing,
+ software distributed under the License is distributed on an
+ "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ KIND, either express or implied. See the License for the
+ specific language governing permissions and limitations
+ under the License.
+#}
+
{#
Review comment:
Duplicated Apache's licence.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[jira] [Commented] (AIRFLOW-5226) Consistent licences for all html JINJA templatesc
[
https://issues.apache.org/jira/browse/AIRFLOW-5226?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16908054#comment-16908054
]
ASF GitHub Bot commented on AIRFLOW-5226:
-
potiuk commented on pull request #5828: [AIRFLOW-5226] Consistent licences for
all jinja templates
URL: https://github.com/apache/airflow/pull/5828
Make sure you have checked _all_ steps below.
### Jira
- [x] My PR addresses the following [Airflow
Jira](https://issues.apache.org/jira/browse/AIRFLOW/) issues and references
them in the PR title. For example, "\[AIRFLOW-XXX\] My Airflow PR"
- https://issues.apache.org/jira/browse/AIRFLOW-5226
### Description
- [x] Here are some details about my PR, including screenshots of any UI
changes:
### Tests
- [x] My PR adds the following unit tests __OR__ does not need testing for
this extremely good reason:
### Commits
- [x] My commits all reference Jira issues in their subject lines, and I
have squashed multiple commits if they address the same issue. In addition, my
commits follow the guidelines from "[How to write a good git commit
message](http://chris.beams.io/posts/git-commit/)":
1. Subject is separated from body by a blank line
1. Subject is limited to 50 characters (not including Jira issue reference)
1. Subject does not end with a period
1. Subject uses the imperative mood ("add", not "adding")
1. Body wraps at 72 characters
1. Body explains "what" and "why", not "how"
### Documentation
- [x] In case of new functionality, my PR adds documentation that describes
how to use it.
- All the public functions and the classes in the PR contain docstrings
that explain what it does
- If you implement backwards incompatible changes, please leave a note in
the [Updating.md](https://github.com/apache/airflow/blob/master/UPDATING.md) so
we can assign it to a appropriate release
### Code Quality
- [x] Passes `flake8`
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
> Consistent licences for all html JINJA templatesc
> -
>
> Key: AIRFLOW-5226
> URL: https://issues.apache.org/jira/browse/AIRFLOW-5226
> Project: Apache Airflow
> Issue Type: Improvement
> Components: ci
>Affects Versions: 2.0.0
>Reporter: Jarek Potiuk
>Priority: Major
>
--
This message was sent by Atlassian JIRA
(v7.6.14#76016)
[GitHub] [airflow] potiuk opened a new pull request #5828: [AIRFLOW-5226] Consistent licences for all jinja templates
potiuk opened a new pull request #5828: [AIRFLOW-5226] Consistent licences for
all jinja templates
URL: https://github.com/apache/airflow/pull/5828
Make sure you have checked _all_ steps below.
### Jira
- [x] My PR addresses the following [Airflow
Jira](https://issues.apache.org/jira/browse/AIRFLOW/) issues and references
them in the PR title. For example, "\[AIRFLOW-XXX\] My Airflow PR"
- https://issues.apache.org/jira/browse/AIRFLOW-5226
### Description
- [x] Here are some details about my PR, including screenshots of any UI
changes:
### Tests
- [x] My PR adds the following unit tests __OR__ does not need testing for
this extremely good reason:
### Commits
- [x] My commits all reference Jira issues in their subject lines, and I
have squashed multiple commits if they address the same issue. In addition, my
commits follow the guidelines from "[How to write a good git commit
message](http://chris.beams.io/posts/git-commit/)":
1. Subject is separated from body by a blank line
1. Subject is limited to 50 characters (not including Jira issue reference)
1. Subject does not end with a period
1. Subject uses the imperative mood ("add", not "adding")
1. Body wraps at 72 characters
1. Body explains "what" and "why", not "how"
### Documentation
- [x] In case of new functionality, my PR adds documentation that describes
how to use it.
- All the public functions and the classes in the PR contain docstrings
that explain what it does
- If you implement backwards incompatible changes, please leave a note in
the [Updating.md](https://github.com/apache/airflow/blob/master/UPDATING.md) so
we can assign it to a appropriate release
### Code Quality
- [x] Passes `flake8`
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[jira] [Created] (AIRFLOW-5226) Consistent licences for all html JINJA templatesc
Jarek Potiuk created AIRFLOW-5226: - Summary: Consistent licences for all html JINJA templatesc Key: AIRFLOW-5226 URL: https://issues.apache.org/jira/browse/AIRFLOW-5226 Project: Apache Airflow Issue Type: Improvement Components: ci Affects Versions: 2.0.0 Reporter: Jarek Potiuk -- This message was sent by Atlassian JIRA (v7.6.14#76016)
[jira] [Updated] (AIRFLOW-5225) Consistent licences can be added automatically for all JS files
[ https://issues.apache.org/jira/browse/AIRFLOW-5225?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jarek Potiuk updated AIRFLOW-5225: -- Summary: Consistent licences can be added automatically for all JS files (was: Licences can be added automatically for all JS files) > Consistent licences can be added automatically for all JS files > --- > > Key: AIRFLOW-5225 > URL: https://issues.apache.org/jira/browse/AIRFLOW-5225 > Project: Apache Airflow > Issue Type: Improvement > Components: ci >Affects Versions: 2.0.0 >Reporter: Jarek Potiuk >Priority: Major > -- This message was sent by Atlassian JIRA (v7.6.14#76016)
[jira] [Commented] (AIRFLOW-5225) Licences can be added automatically for all JS files
[
https://issues.apache.org/jira/browse/AIRFLOW-5225?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16908050#comment-16908050
]
ASF GitHub Bot commented on AIRFLOW-5225:
-
potiuk commented on pull request #5827: [AIRFLOW-5225] Consistent licence for
all JS files
URL: https://github.com/apache/airflow/pull/5827
Make sure you have checked _all_ steps below.
### Jira
- [x] My PR addresses the following [Airflow
Jira](https://issues.apache.org/jira/browse/AIRFLOW/) issues and references
them in the PR title. For example, "\[AIRFLOW-XXX\] My Airflow PR"
- https://issues.apache.org/jira/browse/AIRFLOW-5225
### Description
- [x] Here are some details about my PR, including screenshots of any UI
changes:
### Tests
- [x] My PR adds the following unit tests __OR__ does not need testing for
this extremely good reason:
### Commits
- [x] My commits all reference Jira issues in their subject lines, and I
have squashed multiple commits if they address the same issue. In addition, my
commits follow the guidelines from "[How to write a good git commit
message](http://chris.beams.io/posts/git-commit/)":
1. Subject is separated from body by a blank line
1. Subject is limited to 50 characters (not including Jira issue reference)
1. Subject does not end with a period
1. Subject uses the imperative mood ("add", not "adding")
1. Body wraps at 72 characters
1. Body explains "what" and "why", not "how"
### Documentation
- [x] In case of new functionality, my PR adds documentation that describes
how to use it.
- All the public functions and the classes in the PR contain docstrings
that explain what it does
- If you implement backwards incompatible changes, please leave a note in
the [Updating.md](https://github.com/apache/airflow/blob/master/UPDATING.md) so
we can assign it to a appropriate release
### Code Quality
- [x] Passes `flake8`
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
> Licences can be added automatically for all JS files
>
>
> Key: AIRFLOW-5225
> URL: https://issues.apache.org/jira/browse/AIRFLOW-5225
> Project: Apache Airflow
> Issue Type: Improvement
> Components: ci
>Affects Versions: 2.0.0
>Reporter: Jarek Potiuk
>Priority: Major
>
--
This message was sent by Atlassian JIRA
(v7.6.14#76016)
[GitHub] [airflow] potiuk opened a new pull request #5827: [AIRFLOW-5225] Consistent licence for all JS files
potiuk opened a new pull request #5827: [AIRFLOW-5225] Consistent licence for
all JS files
URL: https://github.com/apache/airflow/pull/5827
Make sure you have checked _all_ steps below.
### Jira
- [x] My PR addresses the following [Airflow
Jira](https://issues.apache.org/jira/browse/AIRFLOW/) issues and references
them in the PR title. For example, "\[AIRFLOW-XXX\] My Airflow PR"
- https://issues.apache.org/jira/browse/AIRFLOW-5225
### Description
- [x] Here are some details about my PR, including screenshots of any UI
changes:
### Tests
- [x] My PR adds the following unit tests __OR__ does not need testing for
this extremely good reason:
### Commits
- [x] My commits all reference Jira issues in their subject lines, and I
have squashed multiple commits if they address the same issue. In addition, my
commits follow the guidelines from "[How to write a good git commit
message](http://chris.beams.io/posts/git-commit/)":
1. Subject is separated from body by a blank line
1. Subject is limited to 50 characters (not including Jira issue reference)
1. Subject does not end with a period
1. Subject uses the imperative mood ("add", not "adding")
1. Body wraps at 72 characters
1. Body explains "what" and "why", not "how"
### Documentation
- [x] In case of new functionality, my PR adds documentation that describes
how to use it.
- All the public functions and the classes in the PR contain docstrings
that explain what it does
- If you implement backwards incompatible changes, please leave a note in
the [Updating.md](https://github.com/apache/airflow/blob/master/UPDATING.md) so
we can assign it to a appropriate release
### Code Quality
- [x] Passes `flake8`
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[jira] [Created] (AIRFLOW-5225) Licences can be added automatically for all JS files
Jarek Potiuk created AIRFLOW-5225: - Summary: Licences can be added automatically for all JS files Key: AIRFLOW-5225 URL: https://issues.apache.org/jira/browse/AIRFLOW-5225 Project: Apache Airflow Issue Type: Improvement Components: ci Affects Versions: 2.0.0 Reporter: Jarek Potiuk -- This message was sent by Atlassian JIRA (v7.6.14#76016)
[jira] [Commented] (AIRFLOW-5223) Use kind for Kubernetes in CI
[ https://issues.apache.org/jira/browse/AIRFLOW-5223?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16908033#comment-16908033 ] Jarek Potiuk commented on AIRFLOW-5223: --- (y) > Use kind for Kubernetes in CI > - > > Key: AIRFLOW-5223 > URL: https://issues.apache.org/jira/browse/AIRFLOW-5223 > Project: Apache Airflow > Issue Type: Improvement > Components: ci >Affects Versions: 1.10.4 >Reporter: Gerardo Curiel >Priority: Minor > Fix For: 2.0.0 > > -- This message was sent by Atlassian JIRA (v7.6.14#76016)
[GitHub] [airflow] acordiner commented on issue #5489: [AIRFLOW-4843] Allow orchestration via Docker Swarm (SwarmOperator)
acordiner commented on issue #5489: [AIRFLOW-4843] Allow orchestration via Docker Swarm (SwarmOperator) URL: https://github.com/apache/airflow/pull/5489#issuecomment-521621285 @akki - Great! Yes, they're part of the API: https://docker-py.readthedocs.io/en/stable/api.html#docker.types.ContainerSpec https://docker-py.readthedocs.io/en/stable/api.html#docker.types.TaskTemplate This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [airflow] akki commented on issue #5489: [AIRFLOW-4843] Allow orchestration via Docker Swarm (SwarmOperator)
akki commented on issue #5489: [AIRFLOW-4843] Allow orchestration via Docker Swarm (SwarmOperator) URL: https://github.com/apache/airflow/pull/5489#issuecomment-521615573 @acordiner thanks. If the feature is supported by the Python Docker client API, it should be pretty straightforward to add it. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
