[GitHub] [airflow] codecov-io commented on issue #4936: [AIRFLOW-4115] Multi-staging Aiflow Docker image [Step 1/3]
codecov-io commented on issue #4936: [AIRFLOW-4115] Multi-staging Aiflow Docker image [Step 1/3] URL: https://github.com/apache/airflow/pull/4936#issuecomment-478445340 # [Codecov](https://codecov.io/gh/apache/airflow/pull/4936?src=pr=h1) Report > Merging [#4936](https://codecov.io/gh/apache/airflow/pull/4936?src=pr=desc) into [master](https://codecov.io/gh/apache/airflow/commit/75c633e70fdc2537a0112939a52666a5c0c2e114?src=pr=desc) will **increase** coverage by `65.38%`. > The diff coverage is `n/a`. [](https://codecov.io/gh/apache/airflow/pull/4936?src=pr=tree) ```diff @@ Coverage Diff @@ ## master#4936 +/- ## === + Coverage10.6% 75.99% +65.38% === Files 461 461 Lines 2996829968 === + Hits 317922774+19595 + Misses 26789 7194-19595 ``` | [Impacted Files](https://codecov.io/gh/apache/airflow/pull/4936?src=pr=tree) | Coverage Δ | | |---|---|---| | [airflow/plugins\_manager.py](https://codecov.io/gh/apache/airflow/pull/4936/diff?src=pr=tree#diff-YWlyZmxvdy9wbHVnaW5zX21hbmFnZXIucHk=) | `86.91% <0%> (+0.93%)` | :arrow_up: | | [airflow/executors/dask\_executor.py](https://codecov.io/gh/apache/airflow/pull/4936/diff?src=pr=tree#diff-YWlyZmxvdy9leGVjdXRvcnMvZGFza19leGVjdXRvci5weQ==) | `2% <0%> (+2%)` | :arrow_up: | | [airflow/exceptions.py](https://codecov.io/gh/apache/airflow/pull/4936/diff?src=pr=tree#diff-YWlyZmxvdy9leGNlcHRpb25zLnB5) | `100% <0%> (+2.56%)` | :arrow_up: | | [airflow/config\_templates/airflow\_local\_settings.py](https://codecov.io/gh/apache/airflow/pull/4936/diff?src=pr=tree#diff-YWlyZmxvdy9jb25maWdfdGVtcGxhdGVzL2FpcmZsb3dfbG9jYWxfc2V0dGluZ3MucHk=) | `76.47% <0%> (+2.94%)` | :arrow_up: | | [airflow/utils/operator\_resources.py](https://codecov.io/gh/apache/airflow/pull/4936/diff?src=pr=tree#diff-YWlyZmxvdy91dGlscy9vcGVyYXRvcl9yZXNvdXJjZXMucHk=) | `86.95% <0%> (+4.34%)` | :arrow_up: | | [...etes\_request\_factory/kubernetes\_request\_factory.py](https://codecov.io/gh/apache/airflow/pull/4936/diff?src=pr=tree#diff-YWlyZmxvdy9jb250cmliL2t1YmVybmV0ZXMva3ViZXJuZXRlc19yZXF1ZXN0X2ZhY3Rvcnkva3ViZXJuZXRlc19yZXF1ZXN0X2ZhY3RvcnkucHk=) | `72.11% <0%> (+4.8%)` | :arrow_up: | | [airflow/executors/\_\_init\_\_.py](https://codecov.io/gh/apache/airflow/pull/4936/diff?src=pr=tree#diff-YWlyZmxvdy9leGVjdXRvcnMvX19pbml0X18ucHk=) | `63.46% <0%> (+5.76%)` | :arrow_up: | | [airflow/settings.py](https://codecov.io/gh/apache/airflow/pull/4936/diff?src=pr=tree#diff-YWlyZmxvdy9zZXR0aW5ncy5weQ==) | `84.25% <0%> (+11.81%)` | :arrow_up: | | [airflow/\_vendor/nvd3/ipynb.py](https://codecov.io/gh/apache/airflow/pull/4936/diff?src=pr=tree#diff-YWlyZmxvdy9fdmVuZG9yL252ZDMvaXB5bmIucHk=) | `14.28% <0%> (+14.28%)` | :arrow_up: | | [airflow/stats.py](https://codecov.io/gh/apache/airflow/pull/4936/diff?src=pr=tree#diff-YWlyZmxvdy9zdGF0cy5weQ==) | `69.69% <0%> (+16.66%)` | :arrow_up: | | ... and [360 more](https://codecov.io/gh/apache/airflow/pull/4936/diff?src=pr=tree-more) | | -- [Continue to review full report at Codecov](https://codecov.io/gh/apache/airflow/pull/4936?src=pr=continue). > **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta) > `Δ = absolute (impact)`, `ø = not affected`, `? = missing data` > Powered by [Codecov](https://codecov.io/gh/apache/airflow/pull/4936?src=pr=footer). Last update [75c633e...346bb16](https://codecov.io/gh/apache/airflow/pull/4936?src=pr=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments). This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [airflow] potiuk commented on a change in pull request #4936: [AIRFLOW-4115] Multi-staging Aiflow Docker image [Step 1/3]
potiuk commented on a change in pull request #4936: [AIRFLOW-4115] Multi-staging Aiflow Docker image [Step 1/3] URL: https://github.com/apache/airflow/pull/4936#discussion_r270718562 ## File path: .dockerignore ## @@ -0,0 +1,107 @@ +# +# Licensed to the Apache Software Foundation (ASF) under one +# or more contributor license agreements. See the NOTICE file +# distributed with this work for additional information +# regarding copyright ownership. The ASF licenses this file +# to you under the Apache License, Version 2.0 (the +# "License"); you may not use this file except in compliance +# with the License. You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, +# software distributed under the License is distributed on an +# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY +# KIND, either express or implied. See the License for the +# specific language governing permissions and limitations +# under the License. + +# NOTE! This docker ignore uses recommended technique +# Where everything is excluded by default and you deliberately +# Add only those directories/files you need. This is very useful +# To make sure that Docker context is always the same on any machine +# So that generated files are not accidentally added to the context +# This allows Docker's `COPY .` to behave in predictable way + +# Ignore everything +** Review comment: I can give some more examples where things could go south. When I run unit tests from the IDE, the logs by default are written in the working directory - which is by default airflow's root folder. They are now ignored with .gitignore but when docker build is executed the logs are potentially big and should not be sent as part of the context. It's safer to exclude those by default. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Commented] (AIRFLOW-4194) set dag_run state to failed when user terminate backfill
[
https://issues.apache.org/jira/browse/AIRFLOW-4194?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16806392#comment-16806392
]
ASF GitHub Bot commented on AIRFLOW-4194:
-
milton0825 commented on pull request #5016: [AIRFLOW-4194] Set dag_run state to
failed when user terminate backfill
URL: https://github.com/apache/airflow/pull/5016
Make sure you have checked _all_ steps below.
### Jira
- [X] My PR addresses the following [Airflow
Jira](https://issues.apache.org/jira/browse/AIRFLOW/) issues and references
them in the PR title. For example, "\[AIRFLOW-XXX\] My Airflow PR"
- https://issues.apache.org/jira/browse/AIRFLOW-XXX
- In case you are fixing a typo in the documentation you can prepend your
commit with \[AIRFLOW-XXX\], code changes always need a Jira issue.
- In case you are proposing a fundamental code change, you need to create
an Airflow Improvement Proposal
([AIP](https://cwiki.apache.org/confluence/display/AIRFLOW/Airflow+Improvements+Proposals)).
- In case you are adding a dependency, check if the license complies with
the [ASF 3rd Party License
Policy](https://www.apache.org/legal/resolved.html#category-x).
### Description
- [ ] Here are some details about my PR, including screenshots of any UI
changes:
### Tests
- [ ] My PR adds the following unit tests __OR__ does not need testing for
this extremely good reason:
### Commits
- [ ] My commits all reference Jira issues in their subject lines, and I
have squashed multiple commits if they address the same issue. In addition, my
commits follow the guidelines from "[How to write a good git commit
message](http://chris.beams.io/posts/git-commit/)":
1. Subject is separated from body by a blank line
1. Subject is limited to 50 characters (not including Jira issue reference)
1. Subject does not end with a period
1. Subject uses the imperative mood ("add", not "adding")
1. Body wraps at 72 characters
1. Body explains "what" and "why", not "how"
### Documentation
- [ ] In case of new functionality, my PR adds documentation that describes
how to use it.
- All the public functions and the classes in the PR contain docstrings
that explain what it does
- If you implement backwards incompatible changes, please leave a note in
the [Updating.md](https://github.com/apache/airflow/blob/master/UPDATING.md) so
we can assign it to a appropriate release
### Code Quality
- [ ] Passes `flake8`
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
> set dag_run state to failed when user terminate backfill
>
>
> Key: AIRFLOW-4194
> URL: https://issues.apache.org/jira/browse/AIRFLOW-4194
> Project: Apache Airflow
> Issue Type: New Feature
>Reporter: Chao-Han Tsai
>Assignee: Chao-Han Tsai
>Priority: Major
>
> Reset dag_run state to failed if user terminate backfill. Otherwise the
> dag_run state will stay in running state which consumes max_active_dagruns.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (AIRFLOW-4194) set dag_run state to failed when user terminate backfill
[ https://issues.apache.org/jira/browse/AIRFLOW-4194?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chao-Han Tsai updated AIRFLOW-4194: --- Description: Reset dag_run state to failed if user terminate backfill. Otherwise the dag_run state will stay in running state which consumes max_active_dagruns. (was: Currently when user terminate the backfill, we set the task_instance state to failed. We should also set the dag_run state to failed.) > set dag_run state to failed when user terminate backfill > > > Key: AIRFLOW-4194 > URL: https://issues.apache.org/jira/browse/AIRFLOW-4194 > Project: Apache Airflow > Issue Type: New Feature >Reporter: Chao-Han Tsai >Assignee: Chao-Han Tsai >Priority: Major > > Reset dag_run state to failed if user terminate backfill. Otherwise the > dag_run state will stay in running state which consumes max_active_dagruns. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[GitHub] [airflow] bryanyang0528 commented on a change in pull request #5004: [AIRFLOW-4187] SlackWebhookOperator do not pass http_conn_id to its parent class
bryanyang0528 commented on a change in pull request #5004: [AIRFLOW-4187] SlackWebhookOperator do not pass http_conn_id to its parent class URL: https://github.com/apache/airflow/pull/5004#discussion_r270700074 ## File path: tests/contrib/hooks/test_slack_webhook_hook.py ## @@ -91,6 +92,16 @@ def test_build_slack_message(self): # Then self.assertEqual(self.expected_message, message) +def test_get_base_url_default(self): +hook = SlackWebhookHook(http_conn_id='http_default') +hook.get_conn() + +self.assertEqual(hook._get_base_url(), 'https://www.google.com/') Review comment: @ashb Does it make sense? This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [airflow] bryanyang0528 commented on a change in pull request #5004: [AIRFLOW-4187] SlackWebhookOperator do not pass http_conn_id to its parent class
bryanyang0528 commented on a change in pull request #5004: [AIRFLOW-4187]
SlackWebhookOperator do not pass http_conn_id to its parent class
URL: https://github.com/apache/airflow/pull/5004#discussion_r270492480
##
File path: tests/contrib/hooks/test_slack_webhook_hook.py
##
@@ -91,6 +92,16 @@ def test_build_slack_message(self):
# Then
self.assertEqual(self.expected_message, message)
+def test_get_base_url_default(self):
+hook = SlackWebhookHook(http_conn_id='http_default')
+hook.get_conn()
+
+self.assertEqual(hook._get_base_url(), 'https://www.google.com/')
Review comment:
According to this
article:https://stackoverflow.com/questions/52054427/how-to-integrate-apache-airflow-with-slack/52054663
SlackWebhookOperator could be set like this:
```
from airflow.contrib.operators.slack_webhook_operator import
SlackWebhookOperator
slack_msg="Hi Wssup?"
slack_test = SlackWebhookOperator(
task_id='slack_test',
http_conn_id='slack_connection',
webhook_token='/1234/abcd',
message=slack_msg,
channel='#airflow_updates',
username='airflow_'+os.environ['ENVIRONMENT'],
icon_emoji=None,
link_names=False,
dag=dag)
```
And set the host of slack in connection like this:
`host=https://hooks.slack.com/services/`
The argument-`http_conn_id` will be used for getting the hostname of Slack.
But in the current version, `http_conn_id` can't be passed to the parent
class-`HTTPHOOK`, so that `HTTPHook` will read the host using the default
http_conn_id `http_default` and send the msg to
`https://www.google.com/YOUR_ENDPOINT`. This the result
```
[2019-03-29 10:23:21,444] {base_hook.py:83} INFO - Using connection to: id:
http_default. Host: https://www.google.com/, Port: None, Schema: None, Login:
None, Password: None, extra: {}
[2019-03-29 10:23:21,446] {http_hook.py:128} INFO - Sending 'POST' to url:
https://www.google.com/YOUR_ENDPOINT
/usr/local/lib/python3.6/site-packages/urllib3/connectionpool.py:847:
InsecureRequestWarning: Unverified HTTPS request is being made. Adding
certificate verification is strongly advised. See:
https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings
InsecureRequestWarning)
[2019-03-29 10:23:21,462] {http_hook.py:142} ERROR - HTTP error: Not Found
[2019-03-29 10:23:21,462] {http_hook.py:144} ERROR -
```
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[jira] [Created] (AIRFLOW-4211) Add tests for WebHDFSHook
Felix Uellendall created AIRFLOW-4211: - Summary: Add tests for WebHDFSHook Key: AIRFLOW-4211 URL: https://issues.apache.org/jira/browse/AIRFLOW-4211 Project: Apache Airflow Issue Type: New Feature Reporter: Felix Uellendall Assignee: Felix Uellendall -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (AIRFLOW-4211) Add tests for WebHDFSHook
[
https://issues.apache.org/jira/browse/AIRFLOW-4211?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16806323#comment-16806323
]
ASF GitHub Bot commented on AIRFLOW-4211:
-
feluelle commented on pull request #5015: [AIRFLOW-4211] Add tests for
WebHDFSHook
URL: https://github.com/apache/airflow/pull/5015
Make sure you have checked _all_ steps below.
### Jira
- [x] My PR addresses the following [Airflow
Jira](https://issues.apache.org/jira/browse/AIRFLOW/) issues and references
them in the PR title. For example, "\[AIRFLOW-XXX\] My Airflow PR"
- https://issues.apache.org/jira/browse/AIRFLOW-XXX
- In case you are fixing a typo in the documentation you can prepend your
commit with \[AIRFLOW-XXX\], code changes always need a Jira issue.
- In case you are proposing a fundamental code change, you need to create
an Airflow Improvement Proposal
([AIP](https://cwiki.apache.org/confluence/display/AIRFLOW/Airflow+Improvements+Proposals)).
- In case you are adding a dependency, check if the license complies with
the [ASF 3rd Party License
Policy](https://www.apache.org/legal/resolved.html#category-x).
### Description
- [x] Here are some details about my PR, including screenshots of any UI
changes:
- add tests
- update docs
- refactor code
### Tests
- [x] My PR adds the following unit tests __OR__ does not need testing for
this extremely good reason:
### Commits
- [x] My commits all reference Jira issues in their subject lines, and I
have squashed multiple commits if they address the same issue. In addition, my
commits follow the guidelines from "[How to write a good git commit
message](http://chris.beams.io/posts/git-commit/)":
1. Subject is separated from body by a blank line
1. Subject is limited to 50 characters (not including Jira issue reference)
1. Subject does not end with a period
1. Subject uses the imperative mood ("add", not "adding")
1. Body wraps at 72 characters
1. Body explains "what" and "why", not "how"
### Documentation
- [x] In case of new functionality, my PR adds documentation that describes
how to use it.
- All the public functions and the classes in the PR contain docstrings
that explain what it does
- If you implement backwards incompatible changes, please leave a note in
the [Updating.md](https://github.com/apache/airflow/blob/master/UPDATING.md) so
we can assign it to a appropriate release
### Code Quality
- [x] Passes `flake8`
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
> Add tests for WebHDFSHook
> -
>
> Key: AIRFLOW-4211
> URL: https://issues.apache.org/jira/browse/AIRFLOW-4211
> Project: Apache Airflow
> Issue Type: New Feature
>Reporter: Felix Uellendall
>Assignee: Felix Uellendall
>Priority: Major
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[GitHub] [airflow] feluelle opened a new pull request #5015: [AIRFLOW-4211] Add tests for WebHDFSHook
feluelle opened a new pull request #5015: [AIRFLOW-4211] Add tests for
WebHDFSHook
URL: https://github.com/apache/airflow/pull/5015
Make sure you have checked _all_ steps below.
### Jira
- [x] My PR addresses the following [Airflow
Jira](https://issues.apache.org/jira/browse/AIRFLOW/) issues and references
them in the PR title. For example, "\[AIRFLOW-XXX\] My Airflow PR"
- https://issues.apache.org/jira/browse/AIRFLOW-XXX
- In case you are fixing a typo in the documentation you can prepend your
commit with \[AIRFLOW-XXX\], code changes always need a Jira issue.
- In case you are proposing a fundamental code change, you need to create
an Airflow Improvement Proposal
([AIP](https://cwiki.apache.org/confluence/display/AIRFLOW/Airflow+Improvements+Proposals)).
- In case you are adding a dependency, check if the license complies with
the [ASF 3rd Party License
Policy](https://www.apache.org/legal/resolved.html#category-x).
### Description
- [x] Here are some details about my PR, including screenshots of any UI
changes:
- add tests
- update docs
- refactor code
### Tests
- [x] My PR adds the following unit tests __OR__ does not need testing for
this extremely good reason:
### Commits
- [x] My commits all reference Jira issues in their subject lines, and I
have squashed multiple commits if they address the same issue. In addition, my
commits follow the guidelines from "[How to write a good git commit
message](http://chris.beams.io/posts/git-commit/)":
1. Subject is separated from body by a blank line
1. Subject is limited to 50 characters (not including Jira issue reference)
1. Subject does not end with a period
1. Subject uses the imperative mood ("add", not "adding")
1. Body wraps at 72 characters
1. Body explains "what" and "why", not "how"
### Documentation
- [x] In case of new functionality, my PR adds documentation that describes
how to use it.
- All the public functions and the classes in the PR contain docstrings
that explain what it does
- If you implement backwards incompatible changes, please leave a note in
the [Updating.md](https://github.com/apache/airflow/blob/master/UPDATING.md) so
we can assign it to a appropriate release
### Code Quality
- [x] Passes `flake8`
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [airflow] mik-laj commented on a change in pull request #4821: [AIRFLOW-3998] Use nested commands in cli.
mik-laj commented on a change in pull request #4821: [AIRFLOW-3998] Use nested
commands in cli.
URL: https://github.com/apache/airflow/pull/4821#discussion_r270695422
##
File path: airflow/bin/cli.py
##
@@ -328,34 +348,44 @@ def pool_export_helper(filepath):
@cli_utils.action_logging
-def variables(args):
-if args.get:
-try:
-var = Variable.get(args.get,
- deserialize_json=args.json,
- default_var=args.default)
-print(var)
-except ValueError as e:
-print(e)
-if args.delete:
-Variable.delete(args.delete)
-if args.set:
-Variable.set(args.set[0], args.set[1])
-# Work around 'import' as a reserved keyword
-imp = getattr(args, 'import')
-if imp:
-if os.path.exists(imp):
-import_helper(imp)
-else:
-print("Missing variables file.")
-if args.export:
-export_helper(args.export)
-if not (args.set or args.get or imp or args.export or args.delete):
-# list all variables
-with db.create_session() as session:
-vars = session.query(Variable)
-msg = "\n".join(var.key for var in vars)
-print(msg)
+def variables_list(args):
+with db.create_session() as session:
+vars = session.query(Variable)
+print("\n".join(var.key for var in vars))
+
+
+@cli_utils.action_logging
+def variables_get(args):
+try:
+var = Variable.get(args.key,
+ deserialize_json=args.json,
+ default_var=args.default)
+print(var)
+except ValueError as e:
+print(e)
+
+
+@cli_utils.action_logging
+def variables_set(args):
+Variable.set(args.key, args.value, serialize_json=args.json)
+
+
+@cli_utils.action_logging
+def variables_delete(args):
+Variable.delete(args.key)
+
+
+@cli_utils.action_logging
+def variables_import(args):
+if os.path.exists(args.file):
+import_helper(args.file)
+else:
+print("Missing variables file.")
Review comment:
What error code will be returned here?
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [airflow] XD-DENG commented on issue #4780: [AIRFLOW-3960] Create Google Cloud Speech operators
XD-DENG commented on issue #4780: [AIRFLOW-3960] Create Google Cloud Speech operators URL: https://github.com/apache/airflow/pull/4780#issuecomment-478402853 LGTM @kaxil @Fokko . (On a vacation so responding slowly, sorry for this) This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [airflow] feng-tao merged pull request #5014: [AIRFLOW-XXX] Update WeTransfer in "Who uses Apache Airflow"
feng-tao merged pull request #5014: [AIRFLOW-XXX] Update WeTransfer in "Who uses Apache Airflow" URL: https://github.com/apache/airflow/pull/5014 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [airflow] mik-laj edited a comment on issue #4980: [AIRFLOW-3971] Add Google Cloud Natural Language operators
mik-laj edited a comment on issue #4980: [AIRFLOW-3971] Add Google Cloud Natural Language operators URL: https://github.com/apache/airflow/pull/4980#issuecomment-478396520 @kaxil I apply all your suggestion. In addition, I changed all `literalinclude` to `exampleinclude`, so full code of example is available now. :-) Reference: https://github.com/apache/airflow/pull/4894 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [airflow] mik-laj commented on issue #4980: [AIRFLOW-3971] Add Google Cloud Natural Language operators
mik-laj commented on issue #4980: [AIRFLOW-3971] Add Google Cloud Natural Language operators URL: https://github.com/apache/airflow/pull/4980#issuecomment-478396520 @kaxil I apply all your suggestion. In addition, I changed all `literalinclude` to `exampleinclude`, so full code of example is available now. :-) Reference: This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Commented] (AIRFLOW-4194) set dag_run state to failed when user terminate backfill
[ https://issues.apache.org/jira/browse/AIRFLOW-4194?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16806300#comment-16806300 ] Tao Feng commented on AIRFLOW-4194: --- [~milton0825], I never finish the pr([https://github.com/apache/airflow/pull/3562]) with all the tests. > set dag_run state to failed when user terminate backfill > > > Key: AIRFLOW-4194 > URL: https://issues.apache.org/jira/browse/AIRFLOW-4194 > Project: Apache Airflow > Issue Type: New Feature >Reporter: Chao-Han Tsai >Assignee: Chao-Han Tsai >Priority: Major > > Currently when user terminate the backfill, we set the task_instance state to > failed. We should also set the dag_run state to failed. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[GitHub] [airflow] mik-laj commented on a change in pull request #4980: [AIRFLOW-3971] Add Google Cloud Natural Language operators
mik-laj commented on a change in pull request #4980: [AIRFLOW-3971] Add Google
Cloud Natural Language operators
URL: https://github.com/apache/airflow/pull/4980#discussion_r270690928
##
File path: airflow/contrib/example_dags/example_gcp_natural_language.py
##
@@ -0,0 +1,116 @@
+# -*- coding: utf-8 -*-
+#
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+
+"""
+Example Airflow DAG for Google Natural Language service
+"""
+
+
+from google.cloud.language_v1.proto.language_service_pb2 import Document
+
+import airflow
+from airflow import models
+from airflow.contrib.operators.gcp_natural_language_operator import (
+CloudLanguageAnalyzeEntitiesOperator,
+CloudLanguageAnalyzeEntitySentimentOperator,
+CloudLanguageAnalyzeSentimentOperator,
+CloudLanguageClassifyTextOperator,
+)
+from airflow.operators.bash_operator import BashOperator
+
+# [START howto_operator_natural_language_document_text]
+TEXT = """
+Airflow is a platform to programmatically author, schedule and monitor
workflows.
+
+Use Airflow to author workflows as Directed Acyclic Graphs (DAGs) of tasks.
The Airflow scheduler executes
+ your tasks on an array of workers while following the specified dependencies.
Rich command line utilities
+ make performing complex surgeries on DAGs a snap. The rich user interface
makes it easy to visualize
+ pipelines running in production, monitor progress, and troubleshoot issues
when needed.
+"""
+document = Document(content=TEXT, type="PLAIN_TEXT")
+# [END howto_operator_natural_language_document_text]
+
+# [START howto_operator_natural_language_document_gcs]
+GCS_CONTENT_URI = "gs://my-text-bucket/sentiment-me.txt"
+document_gcs = Document(gcs_content_uri=GCS_CONTENT_URI, type="PLAIN_TEXT")
+# [END howto_operator_natural_language_document_gcs]
+
+
+default_args = {"start_date": airflow.utils.dates.days_ago(1)}
+
+with models.DAG(
+"example_gcp_natural_language",
+default_args=default_args,
+schedule_interval=None, # Override to match your needs
+) as dag:
+
+# [START howto_operator_natural_language_analyze_entities]
Review comment:
Yes. I'm working on it now.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [airflow] kaxil commented on a change in pull request #4980: [AIRFLOW-3971] Add Google Cloud Natural Language operators
kaxil commented on a change in pull request #4980: [AIRFLOW-3971] Add Google
Cloud Natural Language operators
URL: https://github.com/apache/airflow/pull/4980#discussion_r270690822
##
File path: airflow/contrib/example_dags/example_gcp_natural_language.py
##
@@ -0,0 +1,116 @@
+# -*- coding: utf-8 -*-
+#
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+
+"""
+Example Airflow DAG for Google Natural Language service
+"""
+
+
+from google.cloud.language_v1.proto.language_service_pb2 import Document
+
+import airflow
+from airflow import models
+from airflow.contrib.operators.gcp_natural_language_operator import (
+CloudLanguageAnalyzeEntitiesOperator,
+CloudLanguageAnalyzeEntitySentimentOperator,
+CloudLanguageAnalyzeSentimentOperator,
+CloudLanguageClassifyTextOperator,
+)
+from airflow.operators.bash_operator import BashOperator
+
+# [START howto_operator_natural_language_document_text]
+TEXT = """
+Airflow is a platform to programmatically author, schedule and monitor
workflows.
+
+Use Airflow to author workflows as Directed Acyclic Graphs (DAGs) of tasks.
The Airflow scheduler executes
+ your tasks on an array of workers while following the specified dependencies.
Rich command line utilities
+ make performing complex surgeries on DAGs a snap. The rich user interface
makes it easy to visualize
+ pipelines running in production, monitor progress, and troubleshoot issues
when needed.
+"""
+document = Document(content=TEXT, type="PLAIN_TEXT")
+# [END howto_operator_natural_language_document_text]
+
+# [START howto_operator_natural_language_document_gcs]
+GCS_CONTENT_URI = "gs://my-text-bucket/sentiment-me.txt"
+document_gcs = Document(gcs_content_uri=GCS_CONTENT_URI, type="PLAIN_TEXT")
+# [END howto_operator_natural_language_document_gcs]
+
+
+default_args = {"start_date": airflow.utils.dates.days_ago(1)}
+
+with models.DAG(
+"example_gcp_natural_language",
+default_args=default_args,
+schedule_interval=None, # Override to match your needs
+) as dag:
+
+# [START howto_operator_natural_language_analyze_entities]
Review comment:
yea agree.. Can you resolve conflicts with `docs/conf.py`
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [airflow] kaxil commented on a change in pull request #4633: AIRFLOW-3791: Dataflow
kaxil commented on a change in pull request #4633: AIRFLOW-3791: Dataflow
URL: https://github.com/apache/airflow/pull/4633#discussion_r270690735
##
File path: airflow/contrib/hooks/gcp_dataflow_hook.py
##
@@ -31,88 +31,111 @@
# This is the default location
# https://cloud.google.com/dataflow/pipelines/specifying-exec-params
DEFAULT_DATAFLOW_LOCATION = 'us-central1'
+FAILED_END_STATES = ['JOB_STATE_FAILED', 'JOB_STATE_CANCELLED']
+SUCCEEDED_END_STATES = ['JOB_STATE_DONE']
+END_STATES = SUCCEEDED_END_STATES + FAILED_END_STATES
class _DataflowJob(LoggingMixin):
def __init__(self, dataflow, project_number, name, location, poll_sleep=10,
- job_id=None):
+ job_id=None, multiple_jobs=None):
self._dataflow = dataflow
self._project_number = project_number
self._job_name = name
self._job_location = location
+self.multiple_jobs = multiple_jobs
self._job_id = job_id
-self._job = self._get_job()
+self._jobs = self._get_jobs()
self._poll_sleep = poll_sleep
+def is_job_running(self):
+for job in self._jobs:
+if job['currentState'] not in END_STATES:
+return True
+return False
+
def _get_job_id_from_name(self):
jobs = self._dataflow.projects().locations().jobs().list(
projectId=self._project_number,
location=self._job_location
).execute(num_retries=5)
+dataflow_jobs = []
for job in jobs['jobs']:
-if job['name'] == self._job_name:
-self._job_id = job['id']
-return job
-return None
-
-def _get_job(self):
-if self._job_id:
-job = self._dataflow.projects().locations().jobs().get(
+if job['name'].startswith(self._job_name):
+dataflow_jobs.append(job)
+if len(dataflow_jobs) == 1:
+self._job_id = dataflow_jobs[0]['id']
+return dataflow_jobs
+
+def _get_jobs(self):
+if not self.multiple_jobs and self._job_id:
+self._jobs = []
+self._jobs.append(self._dataflow.projects().locations().jobs().get(
projectId=self._project_number,
location=self._job_location,
-jobId=self._job_id).execute(num_retries=5)
+jobId=self._job_id).execute(num_retries=5))
elif self._job_name:
-job = self._get_job_id_from_name()
+self._jobs = self._get_job_id_from_name()
else:
raise Exception('Missing both dataflow job ID and name.')
-if job and 'currentState' in job:
-self.log.info(
-'Google Cloud DataFlow job %s is %s',
-job['name'], job['currentState']
-)
-elif job:
-self.log.info(
-'Google Cloud DataFlow with job_id %s has name %s',
-self._job_id, job['name']
-)
-else:
-self.log.info(
-'Google Cloud DataFlow job not available yet..'
-)
+for job in self._jobs:
+if job and 'currentState' in job:
+self._job_state = job['currentState']
+self.log.info(
+'Google Cloud DataFlow job %s is %s',
+job['name'], job['currentState']
+)
+elif job:
+self.log.info(
+'Google Cloud DataFlow with job_id %s has name %s',
+self._job_id, job['name']
+)
+else:
+self.log.info(
+'Google Cloud DataFlow job not available yet..'
+)
-return job
+return self._jobs
def wait_for_done(self):
while True:
-if self._job and 'currentState' in self._job:
-if 'JOB_STATE_DONE' == self._job['currentState']:
-return True
-elif 'JOB_STATE_RUNNING' == self._job['currentState'] and \
- 'JOB_TYPE_STREAMING' == self._job['type']:
-return True
-elif 'JOB_STATE_FAILED' == self._job['currentState']:
-raise Exception("Google Cloud Dataflow job {} has
failed.".format(
-self._job['name']))
-elif 'JOB_STATE_CANCELLED' == self._job['currentState']:
-raise Exception("Google Cloud Dataflow job {} was
cancelled.".format(
-self._job['name']))
-elif 'JOB_STATE_RUNNING' == self._job['currentState']:
-time.sleep(self._poll_sleep)
-elif 'JOB_STATE_PENDING' == self._job['currentState']:
-time.sleep(15)
-else:
-self.log.debug(str(self._job))
-raise Exception(
-
[GitHub] [airflow] kaxil commented on a change in pull request #4633: AIRFLOW-3791: Dataflow
kaxil commented on a change in pull request #4633: AIRFLOW-3791: Dataflow
URL: https://github.com/apache/airflow/pull/4633#discussion_r270690735
##
File path: airflow/contrib/hooks/gcp_dataflow_hook.py
##
@@ -31,88 +31,111 @@
# This is the default location
# https://cloud.google.com/dataflow/pipelines/specifying-exec-params
DEFAULT_DATAFLOW_LOCATION = 'us-central1'
+FAILED_END_STATES = ['JOB_STATE_FAILED', 'JOB_STATE_CANCELLED']
+SUCCEEDED_END_STATES = ['JOB_STATE_DONE']
+END_STATES = SUCCEEDED_END_STATES + FAILED_END_STATES
class _DataflowJob(LoggingMixin):
def __init__(self, dataflow, project_number, name, location, poll_sleep=10,
- job_id=None):
+ job_id=None, multiple_jobs=None):
self._dataflow = dataflow
self._project_number = project_number
self._job_name = name
self._job_location = location
+self.multiple_jobs = multiple_jobs
self._job_id = job_id
-self._job = self._get_job()
+self._jobs = self._get_jobs()
self._poll_sleep = poll_sleep
+def is_job_running(self):
+for job in self._jobs:
+if job['currentState'] not in END_STATES:
+return True
+return False
+
def _get_job_id_from_name(self):
jobs = self._dataflow.projects().locations().jobs().list(
projectId=self._project_number,
location=self._job_location
).execute(num_retries=5)
+dataflow_jobs = []
for job in jobs['jobs']:
-if job['name'] == self._job_name:
-self._job_id = job['id']
-return job
-return None
-
-def _get_job(self):
-if self._job_id:
-job = self._dataflow.projects().locations().jobs().get(
+if job['name'].startswith(self._job_name):
+dataflow_jobs.append(job)
+if len(dataflow_jobs) == 1:
+self._job_id = dataflow_jobs[0]['id']
+return dataflow_jobs
+
+def _get_jobs(self):
+if not self.multiple_jobs and self._job_id:
+self._jobs = []
+self._jobs.append(self._dataflow.projects().locations().jobs().get(
projectId=self._project_number,
location=self._job_location,
-jobId=self._job_id).execute(num_retries=5)
+jobId=self._job_id).execute(num_retries=5))
elif self._job_name:
-job = self._get_job_id_from_name()
+self._jobs = self._get_job_id_from_name()
else:
raise Exception('Missing both dataflow job ID and name.')
-if job and 'currentState' in job:
-self.log.info(
-'Google Cloud DataFlow job %s is %s',
-job['name'], job['currentState']
-)
-elif job:
-self.log.info(
-'Google Cloud DataFlow with job_id %s has name %s',
-self._job_id, job['name']
-)
-else:
-self.log.info(
-'Google Cloud DataFlow job not available yet..'
-)
+for job in self._jobs:
+if job and 'currentState' in job:
+self._job_state = job['currentState']
+self.log.info(
+'Google Cloud DataFlow job %s is %s',
+job['name'], job['currentState']
+)
+elif job:
+self.log.info(
+'Google Cloud DataFlow with job_id %s has name %s',
+self._job_id, job['name']
+)
+else:
+self.log.info(
+'Google Cloud DataFlow job not available yet..'
+)
-return job
+return self._jobs
def wait_for_done(self):
while True:
-if self._job and 'currentState' in self._job:
-if 'JOB_STATE_DONE' == self._job['currentState']:
-return True
-elif 'JOB_STATE_RUNNING' == self._job['currentState'] and \
- 'JOB_TYPE_STREAMING' == self._job['type']:
-return True
-elif 'JOB_STATE_FAILED' == self._job['currentState']:
-raise Exception("Google Cloud Dataflow job {} has
failed.".format(
-self._job['name']))
-elif 'JOB_STATE_CANCELLED' == self._job['currentState']:
-raise Exception("Google Cloud Dataflow job {} was
cancelled.".format(
-self._job['name']))
-elif 'JOB_STATE_RUNNING' == self._job['currentState']:
-time.sleep(self._poll_sleep)
-elif 'JOB_STATE_PENDING' == self._job['currentState']:
-time.sleep(15)
-else:
-self.log.debug(str(self._job))
-raise Exception(
-
[GitHub] [airflow] kaxil commented on issue #4633: AIRFLOW-3791: Dataflow
kaxil commented on issue #4633: AIRFLOW-3791: Dataflow URL: https://github.com/apache/airflow/pull/4633#issuecomment-478395287 cc @fenglu-g What do you think? This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [airflow] mik-laj commented on a change in pull request #4980: [AIRFLOW-3971] Add Google Cloud Natural Language operators
mik-laj commented on a change in pull request #4980: [AIRFLOW-3971] Add Google
Cloud Natural Language operators
URL: https://github.com/apache/airflow/pull/4980#discussion_r270690394
##
File path: airflow/contrib/example_dags/example_gcp_natural_language.py
##
@@ -0,0 +1,116 @@
+# -*- coding: utf-8 -*-
+#
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+
+"""
+Example Airflow DAG for Google Natural Language service
+"""
+
+
+from google.cloud.language_v1.proto.language_service_pb2 import Document
+
+import airflow
+from airflow import models
+from airflow.contrib.operators.gcp_natural_language_operator import (
+CloudLanguageAnalyzeEntitiesOperator,
+CloudLanguageAnalyzeEntitySentimentOperator,
+CloudLanguageAnalyzeSentimentOperator,
+CloudLanguageClassifyTextOperator,
+)
+from airflow.operators.bash_operator import BashOperator
+
+# [START howto_operator_natural_language_document_text]
+TEXT = """
+Airflow is a platform to programmatically author, schedule and monitor
workflows.
+
+Use Airflow to author workflows as Directed Acyclic Graphs (DAGs) of tasks.
The Airflow scheduler executes
+ your tasks on an array of workers while following the specified dependencies.
Rich command line utilities
+ make performing complex surgeries on DAGs a snap. The rich user interface
makes it easy to visualize
+ pipelines running in production, monitor progress, and troubleshoot issues
when needed.
+"""
+document = Document(content=TEXT, type="PLAIN_TEXT")
+# [END howto_operator_natural_language_document_text]
+
+# [START howto_operator_natural_language_document_gcs]
+GCS_CONTENT_URI = "gs://my-text-bucket/sentiment-me.txt"
+document_gcs = Document(gcs_content_uri=GCS_CONTENT_URI, type="PLAIN_TEXT")
+# [END howto_operator_natural_language_document_gcs]
+
+
+default_args = {"start_date": airflow.utils.dates.days_ago(1)}
+
+with models.DAG(
+"example_gcp_natural_language",
+default_args=default_args,
+schedule_interval=None, # Override to match your needs
+) as dag:
+
+# [START howto_operator_natural_language_analyze_entities]
Review comment:
Conflict will never happen because the tags are always associated with the
file. Prefixes are completely unnecessary. However, I will introduce your
suggestion in a moment.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [airflow] kaxil commented on issue #4780: [AIRFLOW-3960] Create Google Cloud Speech operators
kaxil commented on issue #4780: [AIRFLOW-3960] Create Google Cloud Speech operators URL: https://github.com/apache/airflow/pull/4780#issuecomment-478394717 @XD-DENG Any more comments on this? This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [airflow] kaxil commented on a change in pull request #4980: [AIRFLOW-3971] Add Google Cloud Natural Language operators
kaxil commented on a change in pull request #4980: [AIRFLOW-3971] Add Google Cloud Natural Language operators URL: https://github.com/apache/airflow/pull/4980#discussion_r270690034 ## File path: airflow/contrib/example_dags/example_gcp_natural_language.py ## @@ -0,0 +1,116 @@ +# -*- coding: utf-8 -*- +# +# Licensed to the Apache Software Foundation (ASF) under one +# or more contributor license agreements. See the NOTICE file +# distributed with this work for additional information +# regarding copyright ownership. The ASF licenses this file +# to you under the Apache License, Version 2.0 (the +# "License"); you may not use this file except in compliance +# with the License. You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, +# software distributed under the License is distributed on an +# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY +# KIND, either express or implied. See the License for the +# specific language governing permissions and limitations +# under the License. + +""" +Example Airflow DAG for Google Natural Language service Review comment: ```suggestion Example Airflow DAG for Google Cloud Natural Language service ``` This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [airflow] kaxil commented on a change in pull request #4980: [AIRFLOW-3971] Add Google Cloud Natural Language operators
kaxil commented on a change in pull request #4980: [AIRFLOW-3971] Add Google
Cloud Natural Language operators
URL: https://github.com/apache/airflow/pull/4980#discussion_r270690111
##
File path: airflow/contrib/example_dags/example_gcp_natural_language.py
##
@@ -0,0 +1,116 @@
+# -*- coding: utf-8 -*-
+#
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+
+"""
+Example Airflow DAG for Google Natural Language service
+"""
+
+
+from google.cloud.language_v1.proto.language_service_pb2 import Document
+
+import airflow
+from airflow import models
+from airflow.contrib.operators.gcp_natural_language_operator import (
+CloudLanguageAnalyzeEntitiesOperator,
+CloudLanguageAnalyzeEntitySentimentOperator,
+CloudLanguageAnalyzeSentimentOperator,
+CloudLanguageClassifyTextOperator,
+)
+from airflow.operators.bash_operator import BashOperator
+
+# [START howto_operator_natural_language_document_text]
+TEXT = """
+Airflow is a platform to programmatically author, schedule and monitor
workflows.
+
+Use Airflow to author workflows as Directed Acyclic Graphs (DAGs) of tasks.
The Airflow scheduler executes
+ your tasks on an array of workers while following the specified dependencies.
Rich command line utilities
+ make performing complex surgeries on DAGs a snap. The rich user interface
makes it easy to visualize
+ pipelines running in production, monitor progress, and troubleshoot issues
when needed.
+"""
+document = Document(content=TEXT, type="PLAIN_TEXT")
+# [END howto_operator_natural_language_document_text]
+
+# [START howto_operator_natural_language_document_gcs]
+GCS_CONTENT_URI = "gs://my-text-bucket/sentiment-me.txt"
+document_gcs = Document(gcs_content_uri=GCS_CONTENT_URI, type="PLAIN_TEXT")
+# [END howto_operator_natural_language_document_gcs]
+
+
+default_args = {"start_date": airflow.utils.dates.days_ago(1)}
+
+with models.DAG(
+"example_gcp_natural_language",
+default_args=default_args,
+schedule_interval=None, # Override to match your needs
+) as dag:
+
+# [START howto_operator_natural_language_analyze_entities]
Review comment:
And same in all the comments below
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [airflow] kaxil commented on a change in pull request #4980: [AIRFLOW-3971] Add Google Cloud Natural Language operators
kaxil commented on a change in pull request #4980: [AIRFLOW-3971] Add Google
Cloud Natural Language operators
URL: https://github.com/apache/airflow/pull/4980#discussion_r270690096
##
File path: airflow/contrib/example_dags/example_gcp_natural_language.py
##
@@ -0,0 +1,116 @@
+# -*- coding: utf-8 -*-
+#
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+
+"""
+Example Airflow DAG for Google Natural Language service
+"""
+
+
+from google.cloud.language_v1.proto.language_service_pb2 import Document
+
+import airflow
+from airflow import models
+from airflow.contrib.operators.gcp_natural_language_operator import (
+CloudLanguageAnalyzeEntitiesOperator,
+CloudLanguageAnalyzeEntitySentimentOperator,
+CloudLanguageAnalyzeSentimentOperator,
+CloudLanguageClassifyTextOperator,
+)
+from airflow.operators.bash_operator import BashOperator
+
+# [START howto_operator_natural_language_document_text]
+TEXT = """
+Airflow is a platform to programmatically author, schedule and monitor
workflows.
+
+Use Airflow to author workflows as Directed Acyclic Graphs (DAGs) of tasks.
The Airflow scheduler executes
+ your tasks on an array of workers while following the specified dependencies.
Rich command line utilities
+ make performing complex surgeries on DAGs a snap. The rich user interface
makes it easy to visualize
+ pipelines running in production, monitor progress, and troubleshoot issues
when needed.
+"""
+document = Document(content=TEXT, type="PLAIN_TEXT")
+# [END howto_operator_natural_language_document_text]
+
+# [START howto_operator_natural_language_document_gcs]
+GCS_CONTENT_URI = "gs://my-text-bucket/sentiment-me.txt"
+document_gcs = Document(gcs_content_uri=GCS_CONTENT_URI, type="PLAIN_TEXT")
+# [END howto_operator_natural_language_document_gcs]
+
+
+default_args = {"start_date": airflow.utils.dates.days_ago(1)}
+
+with models.DAG(
+"example_gcp_natural_language",
+default_args=default_args,
+schedule_interval=None, # Override to match your needs
+) as dag:
+
+# [START howto_operator_natural_language_analyze_entities]
Review comment:
Can we add `gcp` here... as other cloud providers might have similar
services and can conflict in future?
```suggestion
# [START howto_gcp_operator_natural_language_analyze_entities]
```
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [airflow] codecov-io edited a comment on issue #5010: [AIRFLOW-4192] Reorganize task context variables
codecov-io edited a comment on issue #5010: [AIRFLOW-4192] Reorganize task context variables URL: https://github.com/apache/airflow/pull/5010#issuecomment-478295602 # [Codecov](https://codecov.io/gh/apache/airflow/pull/5010?src=pr=h1) Report > Merging [#5010](https://codecov.io/gh/apache/airflow/pull/5010?src=pr=desc) into [master](https://codecov.io/gh/apache/airflow/commit/55aca52d1b45f228865d98de657db3b83eb2c718?src=pr=desc) will **decrease** coverage by `<.01%`. > The diff coverage is `100%`. [](https://codecov.io/gh/apache/airflow/pull/5010?src=pr=tree) ```diff @@Coverage Diff @@ ## master#5010 +/- ## == - Coverage 75.99% 75.98% -0.01% == Files 461 461 Lines 2996829961 -7 == - Hits2277522767 -8 - Misses 7193 7194 +1 ``` | [Impacted Files](https://codecov.io/gh/apache/airflow/pull/5010?src=pr=tree) | Coverage Δ | | |---|---|---| | [...flow/contrib/example\_dags/example\_qubole\_sensor.py](https://codecov.io/gh/apache/airflow/pull/5010/diff?src=pr=tree#diff-YWlyZmxvdy9jb250cmliL2V4YW1wbGVfZGFncy9leGFtcGxlX3F1Ym9sZV9zZW5zb3IucHk=) | `0% <ø> (ø)` | :arrow_up: | | [airflow/models/\_\_init\_\_.py](https://codecov.io/gh/apache/airflow/pull/5010/diff?src=pr=tree#diff-YWlyZmxvdy9tb2RlbHMvX19pbml0X18ucHk=) | `92.98% <100%> (-0.03%)` | :arrow_down: | | [airflow/operators/hive\_to\_druid.py](https://codecov.io/gh/apache/airflow/pull/5010/diff?src=pr=tree#diff-YWlyZmxvdy9vcGVyYXRvcnMvaGl2ZV90b19kcnVpZC5weQ==) | `63.07% <100%> (ø)` | :arrow_up: | | [airflow/contrib/operators/ssh\_operator.py](https://codecov.io/gh/apache/airflow/pull/5010/diff?src=pr=tree#diff-YWlyZmxvdy9jb250cmliL29wZXJhdG9ycy9zc2hfb3BlcmF0b3IucHk=) | `82.27% <0%> (-1.27%)` | :arrow_down: | -- [Continue to review full report at Codecov](https://codecov.io/gh/apache/airflow/pull/5010?src=pr=continue). > **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta) > `Δ = absolute (impact)`, `ø = not affected`, `? = missing data` > Powered by [Codecov](https://codecov.io/gh/apache/airflow/pull/5010?src=pr=footer). Last update [55aca52...785db6a](https://codecov.io/gh/apache/airflow/pull/5010?src=pr=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments). This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [airflow] codecov-io edited a comment on issue #5010: [AIRFLOW-4192] Reorganize task context variables
codecov-io edited a comment on issue #5010: [AIRFLOW-4192] Reorganize task context variables URL: https://github.com/apache/airflow/pull/5010#issuecomment-478295602 # [Codecov](https://codecov.io/gh/apache/airflow/pull/5010?src=pr=h1) Report > Merging [#5010](https://codecov.io/gh/apache/airflow/pull/5010?src=pr=desc) into [master](https://codecov.io/gh/apache/airflow/commit/55aca52d1b45f228865d98de657db3b83eb2c718?src=pr=desc) will **decrease** coverage by `<.01%`. > The diff coverage is `100%`. [](https://codecov.io/gh/apache/airflow/pull/5010?src=pr=tree) ```diff @@Coverage Diff @@ ## master#5010 +/- ## == - Coverage 75.99% 75.98% -0.01% == Files 461 461 Lines 2996829961 -7 == - Hits2277522767 -8 - Misses 7193 7194 +1 ``` | [Impacted Files](https://codecov.io/gh/apache/airflow/pull/5010?src=pr=tree) | Coverage Δ | | |---|---|---| | [...flow/contrib/example\_dags/example\_qubole\_sensor.py](https://codecov.io/gh/apache/airflow/pull/5010/diff?src=pr=tree#diff-YWlyZmxvdy9jb250cmliL2V4YW1wbGVfZGFncy9leGFtcGxlX3F1Ym9sZV9zZW5zb3IucHk=) | `0% <ø> (ø)` | :arrow_up: | | [airflow/models/\_\_init\_\_.py](https://codecov.io/gh/apache/airflow/pull/5010/diff?src=pr=tree#diff-YWlyZmxvdy9tb2RlbHMvX19pbml0X18ucHk=) | `92.98% <100%> (-0.03%)` | :arrow_down: | | [airflow/operators/hive\_to\_druid.py](https://codecov.io/gh/apache/airflow/pull/5010/diff?src=pr=tree#diff-YWlyZmxvdy9vcGVyYXRvcnMvaGl2ZV90b19kcnVpZC5weQ==) | `63.07% <100%> (ø)` | :arrow_up: | | [airflow/contrib/operators/ssh\_operator.py](https://codecov.io/gh/apache/airflow/pull/5010/diff?src=pr=tree#diff-YWlyZmxvdy9jb250cmliL29wZXJhdG9ycy9zc2hfb3BlcmF0b3IucHk=) | `82.27% <0%> (-1.27%)` | :arrow_down: | -- [Continue to review full report at Codecov](https://codecov.io/gh/apache/airflow/pull/5010?src=pr=continue). > **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta) > `Δ = absolute (impact)`, `ø = not affected`, `? = missing data` > Powered by [Codecov](https://codecov.io/gh/apache/airflow/pull/5010?src=pr=footer). Last update [55aca52...785db6a](https://codecov.io/gh/apache/airflow/pull/5010?src=pr=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments). This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [airflow] BasPH edited a comment on issue #5010: [AIRFLOW-4192] Reorganize task context variables
BasPH edited a comment on issue #5010: [AIRFLOW-4192] Reorganize task context variables URL: https://github.com/apache/airflow/pull/5010#issuecomment-478358376 ~@Fokko after the last push 1 task failed, which succeeded before, can you rerun that one?~ Noticed there was now a conflict with the current master, so merged and pushed once again. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Commented] (AIRFLOW-3458) Refactor: Move Connection out of models.py
[ https://issues.apache.org/jira/browse/AIRFLOW-3458?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16806260#comment-16806260 ] Fokko Driesprong commented on AIRFLOW-3458: --- Awesome, thanks! > Refactor: Move Connection out of models.py > -- > > Key: AIRFLOW-3458 > URL: https://issues.apache.org/jira/browse/AIRFLOW-3458 > Project: Apache Airflow > Issue Type: Task > Components: models >Affects Versions: 1.10.1 >Reporter: Fokko Driesprong >Assignee: Bas Harenslak >Priority: Major > Fix For: 2.0.0, 1.10.3 > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (AIRFLOW-3458) Refactor: Move Connection out of models.py
[ https://issues.apache.org/jira/browse/AIRFLOW-3458?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Fokko Driesprong updated AIRFLOW-3458: -- Fix Version/s: 1.10.3 > Refactor: Move Connection out of models.py > -- > > Key: AIRFLOW-3458 > URL: https://issues.apache.org/jira/browse/AIRFLOW-3458 > Project: Apache Airflow > Issue Type: Task > Components: models >Affects Versions: 1.10.1 >Reporter: Fokko Driesprong >Assignee: Bas Harenslak >Priority: Major > Fix For: 2.0.0, 1.10.3 > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (AIRFLOW-3458) Refactor: Move Connection out of models.py
[ https://issues.apache.org/jira/browse/AIRFLOW-3458?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16806259#comment-16806259 ] Ash Berlin-Taylor commented on AIRFLOW-3458: [~Fokko] I've already cherry-picked the code change on to v1-10-stable, so doing the tests too shouldn't be too much work. > Refactor: Move Connection out of models.py > -- > > Key: AIRFLOW-3458 > URL: https://issues.apache.org/jira/browse/AIRFLOW-3458 > Project: Apache Airflow > Issue Type: Task > Components: models >Affects Versions: 1.10.1 >Reporter: Fokko Driesprong >Assignee: Bas Harenslak >Priority: Major > Fix For: 2.0.0 > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[GitHub] [airflow] codecov-io edited a comment on issue #4821: [AIRFLOW-3998] Use nested commands in cli.
codecov-io edited a comment on issue #4821: [AIRFLOW-3998] Use nested commands in cli. URL: https://github.com/apache/airflow/pull/4821#issuecomment-468981700 # [Codecov](https://codecov.io/gh/apache/airflow/pull/4821?src=pr=h1) Report > Merging [#4821](https://codecov.io/gh/apache/airflow/pull/4821?src=pr=desc) into [master](https://codecov.io/gh/apache/airflow/commit/55aca52d1b45f228865d98de657db3b83eb2c718?src=pr=desc) will **decrease** coverage by `0.01%`. > The diff coverage is `83.73%`. [](https://codecov.io/gh/apache/airflow/pull/4821?src=pr=tree) ```diff @@Coverage Diff @@ ## master#4821 +/- ## == - Coverage 75.99% 75.98% -0.02% == Files 461 461 Lines 2996829938 -30 == - Hits2277522749 -26 + Misses 7193 7189 -4 ``` | [Impacted Files](https://codecov.io/gh/apache/airflow/pull/4821?src=pr=tree) | Coverage Δ | | |---|---|---| | [airflow/bin/cli.py](https://codecov.io/gh/apache/airflow/pull/4821/diff?src=pr=tree#diff-YWlyZmxvdy9iaW4vY2xpLnB5) | `66.56% <83.73%> (-0.42%)` | :arrow_down: | | [airflow/contrib/operators/ssh\_operator.py](https://codecov.io/gh/apache/airflow/pull/4821/diff?src=pr=tree#diff-YWlyZmxvdy9jb250cmliL29wZXJhdG9ycy9zc2hfb3BlcmF0b3IucHk=) | `82.27% <0%> (-1.27%)` | :arrow_down: | | [airflow/models/\_\_init\_\_.py](https://codecov.io/gh/apache/airflow/pull/4821/diff?src=pr=tree#diff-YWlyZmxvdy9tb2RlbHMvX19pbml0X18ucHk=) | `92.96% <0%> (-0.05%)` | :arrow_down: | -- [Continue to review full report at Codecov](https://codecov.io/gh/apache/airflow/pull/4821?src=pr=continue). > **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta) > `Δ = absolute (impact)`, `ø = not affected`, `? = missing data` > Powered by [Codecov](https://codecov.io/gh/apache/airflow/pull/4821?src=pr=footer). Last update [55aca52...3656e9e](https://codecov.io/gh/apache/airflow/pull/4821?src=pr=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments). This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Created] (AIRFLOW-4200) Remove all __future__ imports
Fokko Driesprong created AIRFLOW-4200: - Summary: Remove all __future__ imports Key: AIRFLOW-4200 URL: https://issues.apache.org/jira/browse/AIRFLOW-4200 Project: Apache Airflow Issue Type: Sub-task Reporter: Fokko Driesprong Fix For: 2.0.0 -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (AIRFLOW-4196) AIP-3 Drop support for Python 2
Fokko Driesprong created AIRFLOW-4196: - Summary: AIP-3 Drop support for Python 2 Key: AIRFLOW-4196 URL: https://issues.apache.org/jira/browse/AIRFLOW-4196 Project: Apache Airflow Issue Type: Task Reporter: Fokko Driesprong Fix For: 2.0.0 https://cwiki.apache.org/confluence/display/AIRFLOW/AIP-3+Drop+support+for+Python+2 -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (AIRFLOW-4210) Update the CI image to Python3
Fokko Driesprong created AIRFLOW-4210: - Summary: Update the CI image to Python3 Key: AIRFLOW-4210 URL: https://issues.apache.org/jira/browse/AIRFLOW-4210 Project: Apache Airflow Issue Type: Sub-task Reporter: Fokko Driesprong Currently, we build from plain Debian, but we could now directly build from Python3 image: https://github.com/apache/airflow-ci/blob/master/Dockerfile -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (AIRFLOW-4207) Replace __metaclass__ attribute by metaclass as a class keyword argument
Fokko Driesprong created AIRFLOW-4207: - Summary: Replace __metaclass__ attribute by metaclass as a class keyword argument Key: AIRFLOW-4207 URL: https://issues.apache.org/jira/browse/AIRFLOW-4207 Project: Apache Airflow Issue Type: Sub-task Reporter: Fokko Driesprong Fix For: 2.0.0 -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (AIRFLOW-4209) Replace imp by importlib
Fokko Driesprong created AIRFLOW-4209: - Summary: Replace imp by importlib Key: AIRFLOW-4209 URL: https://issues.apache.org/jira/browse/AIRFLOW-4209 Project: Apache Airflow Issue Type: Sub-task Reporter: Fokko Driesprong Fix For: 2.0.0 Remove references to the imp module, this is replaced by importlib -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (AIRFLOW-4208) Replace @abstractproperty by @abstractmethod
Fokko Driesprong created AIRFLOW-4208: - Summary: Replace @abstractproperty by @abstractmethod Key: AIRFLOW-4208 URL: https://issues.apache.org/jira/browse/AIRFLOW-4208 Project: Apache Airflow Issue Type: Sub-task Reporter: Fokko Driesprong Fix For: 2.0.0 Replace @abstractproperty by @abstractmethod (see https://docs.python.org/3/library/abc.html#abc.abstractproperty) -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (AIRFLOW-4206) Remove explicit inheritance of object
Fokko Driesprong created AIRFLOW-4206: - Summary: Remove explicit inheritance of object Key: AIRFLOW-4206 URL: https://issues.apache.org/jira/browse/AIRFLOW-4206 Project: Apache Airflow Issue Type: Sub-task Reporter: Fokko Driesprong Fix For: 2.0.0 Replace class MyClass(object) by class MyClass() (in Python 3 there's no need to state object). -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (AIRFLOW-4205) Uncomment TypeAnnotations
Fokko Driesprong created AIRFLOW-4205: - Summary: Uncomment TypeAnnotations Key: AIRFLOW-4205 URL: https://issues.apache.org/jira/browse/AIRFLOW-4205 Project: Apache Airflow Issue Type: Sub-task Reporter: Fokko Driesprong Fix For: 2.0.0 Fix all mypy annotations which are currently comments because Python 2 -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (AIRFLOW-4203) Replace os.path by pathlib (exists since Python 3.5)
Fokko Driesprong created AIRFLOW-4203: - Summary: Replace os.path by pathlib (exists since Python 3.5) Key: AIRFLOW-4203 URL: https://issues.apache.org/jira/browse/AIRFLOW-4203 Project: Apache Airflow Issue Type: Sub-task Reporter: Fokko Driesprong Fix For: 2.0.0 -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (AIRFLOW-4202) Make setup.py compatible with Python 3 only
Fokko Driesprong created AIRFLOW-4202: - Summary: Make setup.py compatible with Python 3 only Key: AIRFLOW-4202 URL: https://issues.apache.org/jira/browse/AIRFLOW-4202 Project: Apache Airflow Issue Type: Sub-task Reporter: Fokko Driesprong Fix For: 2.0.0 -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (AIRFLOW-4204) Update Class initialisation
Fokko Driesprong created AIRFLOW-4204: - Summary: Update Class initialisation Key: AIRFLOW-4204 URL: https://issues.apache.org/jira/browse/AIRFLOW-4204 Project: Apache Airflow Issue Type: Sub-task Reporter: Fokko Driesprong Fix For: 2.0.0 In all classes, replace super(__class__, self).__init__(...) by super().__init__(...) (In Python 3 super(__class__, self) == super()) -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (AIRFLOW-4198) Remove all try import py2 version, except import py3 version imports
Fokko Driesprong created AIRFLOW-4198: - Summary: Remove all try import py2 version, except import py3 version imports Key: AIRFLOW-4198 URL: https://issues.apache.org/jira/browse/AIRFLOW-4198 Project: Apache Airflow Issue Type: Sub-task Reporter: Fokko Driesprong Fix For: 2.0.0 -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (AIRFLOW-4197) Remove Python2 CI jobs
Fokko Driesprong created AIRFLOW-4197: - Summary: Remove Python2 CI jobs Key: AIRFLOW-4197 URL: https://issues.apache.org/jira/browse/AIRFLOW-4197 Project: Apache Airflow Issue Type: Sub-task Reporter: Fokko Driesprong Fix For: 2.0.0 -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (AIRFLOW-4201) Replace unicode strings by "normal" strings
Fokko Driesprong created AIRFLOW-4201: - Summary: Replace unicode strings by "normal" strings Key: AIRFLOW-4201 URL: https://issues.apache.org/jira/browse/AIRFLOW-4201 Project: Apache Airflow Issue Type: Sub-task Reporter: Fokko Driesprong Fix For: 2.0.0 -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (AIRFLOW-4199) Remove all sys.version_info[0] == 3 imports
Fokko Driesprong created AIRFLOW-4199: - Summary: Remove all sys.version_info[0] == 3 imports Key: AIRFLOW-4199 URL: https://issues.apache.org/jira/browse/AIRFLOW-4199 Project: Apache Airflow Issue Type: Sub-task Reporter: Fokko Driesprong Fix For: 2.0.0 -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[GitHub] [airflow] codecov-io commented on issue #5004: [AIRFLOW-4187] SlackWebhookOperator do not pass http_conn_id to its parent class
codecov-io commented on issue #5004: [AIRFLOW-4187] SlackWebhookOperator do not pass http_conn_id to its parent class URL: https://github.com/apache/airflow/pull/5004#issuecomment-478372353 # [Codecov](https://codecov.io/gh/apache/airflow/pull/5004?src=pr=h1) Report > Merging [#5004](https://codecov.io/gh/apache/airflow/pull/5004?src=pr=desc) into [master](https://codecov.io/gh/apache/airflow/commit/b8d5ff68bced86266eed697f3135830aafddd750?src=pr=desc) will **increase** coverage by `0.49%`. > The diff coverage is `100%`. [](https://codecov.io/gh/apache/airflow/pull/5004?src=pr=tree) ```diff @@Coverage Diff@@ ## master #5004 +/- ## = + Coverage75.1% 75.6% +0.49% = Files 461 461 Lines 29955 30019 +64 = + Hits22499 22695 +196 + Misses 74567324 -132 ``` | [Impacted Files](https://codecov.io/gh/apache/airflow/pull/5004?src=pr=tree) | Coverage Δ | | |---|---|---| | [airflow/contrib/hooks/slack\_webhook\_hook.py](https://codecov.io/gh/apache/airflow/pull/5004/diff?src=pr=tree#diff-YWlyZmxvdy9jb250cmliL2hvb2tzL3NsYWNrX3dlYmhvb2tfaG9vay5weQ==) | `85.71% <100%> (ø)` | :arrow_up: | | [airflow/models/kubernetes.py](https://codecov.io/gh/apache/airflow/pull/5004/diff?src=pr=tree#diff-YWlyZmxvdy9tb2RlbHMva3ViZXJuZXRlcy5weQ==) | `95.34% <0%> (-4.66%)` | :arrow_down: | | [airflow/contrib/hooks/datastore\_hook.py](https://codecov.io/gh/apache/airflow/pull/5004/diff?src=pr=tree#diff-YWlyZmxvdy9jb250cmliL2hvb2tzL2RhdGFzdG9yZV9ob29rLnB5) | `0% <0%> (ø)` | :arrow_up: | | [airflow/models/\_\_init\_\_.py](https://codecov.io/gh/apache/airflow/pull/5004/diff?src=pr=tree#diff-YWlyZmxvdy9tb2RlbHMvX19pbml0X18ucHk=) | `93.01% <0%> (+0.09%)` | :arrow_up: | | [airflow/jobs.py](https://codecov.io/gh/apache/airflow/pull/5004/diff?src=pr=tree#diff-YWlyZmxvdy9qb2JzLnB5) | `78.83% <0%> (+0.44%)` | :arrow_up: | | [airflow/models/connection.py](https://codecov.io/gh/apache/airflow/pull/5004/diff?src=pr=tree#diff-YWlyZmxvdy9tb2RlbHMvY29ubmVjdGlvbi5weQ==) | `65.53% <0%> (+1.12%)` | :arrow_up: | | [airflow/hooks/hive\_hooks.py](https://codecov.io/gh/apache/airflow/pull/5004/diff?src=pr=tree#diff-YWlyZmxvdy9ob29rcy9oaXZlX2hvb2tzLnB5) | `75.32% <0%> (+1.83%)` | :arrow_up: | | [airflow/hooks/postgres\_hook.py](https://codecov.io/gh/apache/airflow/pull/5004/diff?src=pr=tree#diff-YWlyZmxvdy9ob29rcy9wb3N0Z3Jlc19ob29rLnB5) | `94.44% <0%> (+2.77%)` | :arrow_up: | | [airflow/executors/base\_executor.py](https://codecov.io/gh/apache/airflow/pull/5004/diff?src=pr=tree#diff-YWlyZmxvdy9leGVjdXRvcnMvYmFzZV9leGVjdXRvci5weQ==) | `95.65% <0%> (+2.89%)` | :arrow_up: | | [airflow/hooks/dbapi\_hook.py](https://codecov.io/gh/apache/airflow/pull/5004/diff?src=pr=tree#diff-YWlyZmxvdy9ob29rcy9kYmFwaV9ob29rLnB5) | `88% <0%> (+3.2%)` | :arrow_up: | | ... and [6 more](https://codecov.io/gh/apache/airflow/pull/5004/diff?src=pr=tree-more) | | -- [Continue to review full report at Codecov](https://codecov.io/gh/apache/airflow/pull/5004?src=pr=continue). > **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta) > `Δ = absolute (impact)`, `ø = not affected`, `? = missing data` > Powered by [Codecov](https://codecov.io/gh/apache/airflow/pull/5004?src=pr=footer). Last update [b8d5ff6...9a8fd01](https://codecov.io/gh/apache/airflow/pull/5004?src=pr=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments). This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Commented] (AIRFLOW-4014) Add tests for DatastoreHook
[ https://issues.apache.org/jira/browse/AIRFLOW-4014?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16806238#comment-16806238 ] ASF subversion and git services commented on AIRFLOW-4014: -- Commit 55aca52d1b45f228865d98de657db3b83eb2c718 in airflow's branch refs/heads/master from Felix Uellendall [ https://gitbox.apache.org/repos/asf?p=airflow.git;h=55aca52 ] [AIRFLOW-4014] Change DatastoreHook and add tests (#4842) - update default used version for connecting to the Admin API from v1beta1 to v1 - move the establishment of the connection to the function calls instead of the hook init - change get_conn signature to be able to pass an is_admin arg to set an admin connection - rename GoogleCloudBaseHook._authorize function to GoogleCloudBaseHook.authorize - rename the `partialKeys` argument of function `allocate_ids` to `partial_keys`. - add tests - update docs - refactor code Move version attribute from get_conn to __init__ - revert renaming of authorize function - improve docs - refactor code > Add tests for DatastoreHook > --- > > Key: AIRFLOW-4014 > URL: https://issues.apache.org/jira/browse/AIRFLOW-4014 > Project: Apache Airflow > Issue Type: Test >Reporter: Felix Uellendall >Assignee: Felix Uellendall >Priority: Major > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Resolved] (AIRFLOW-4014) Add tests for DatastoreHook
[ https://issues.apache.org/jira/browse/AIRFLOW-4014?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Fokko Driesprong resolved AIRFLOW-4014. --- Resolution: Fixed Fix Version/s: 2.0.0 > Add tests for DatastoreHook > --- > > Key: AIRFLOW-4014 > URL: https://issues.apache.org/jira/browse/AIRFLOW-4014 > Project: Apache Airflow > Issue Type: Test >Reporter: Felix Uellendall >Assignee: Felix Uellendall >Priority: Major > Fix For: 2.0.0 > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (AIRFLOW-4014) Add tests for DatastoreHook
[ https://issues.apache.org/jira/browse/AIRFLOW-4014?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16806237#comment-16806237 ] ASF GitHub Bot commented on AIRFLOW-4014: - Fokko commented on pull request #4842: [AIRFLOW-4014] Change DatastoreHook and add tests URL: https://github.com/apache/airflow/pull/4842 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org > Add tests for DatastoreHook > --- > > Key: AIRFLOW-4014 > URL: https://issues.apache.org/jira/browse/AIRFLOW-4014 > Project: Apache Airflow > Issue Type: Test >Reporter: Felix Uellendall >Assignee: Felix Uellendall >Priority: Major > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[GitHub] [airflow] Fokko merged pull request #4842: [AIRFLOW-4014] Change DatastoreHook and add tests
Fokko merged pull request #4842: [AIRFLOW-4014] Change DatastoreHook and add tests URL: https://github.com/apache/airflow/pull/4842 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [airflow] Fokko merged pull request #5008: [AIRFLOW-XXX] Fix typo in README
Fokko merged pull request #5008: [AIRFLOW-XXX] Fix typo in README URL: https://github.com/apache/airflow/pull/5008 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [airflow] codecov-io edited a comment on issue #4821: [AIRFLOW-3998] Use nested commands in cli.
codecov-io edited a comment on issue #4821: [AIRFLOW-3998] Use nested commands in cli. URL: https://github.com/apache/airflow/pull/4821#issuecomment-468981700 # [Codecov](https://codecov.io/gh/apache/airflow/pull/4821?src=pr=h1) Report > Merging [#4821](https://codecov.io/gh/apache/airflow/pull/4821?src=pr=desc) into [master](https://codecov.io/gh/apache/airflow/commit/4d98a4127a7846e75261c0c1297a8cbadb91b9ab?src=pr=desc) will **decrease** coverage by `<.01%`. > The diff coverage is `82.82%`. [](https://codecov.io/gh/apache/airflow/pull/4821?src=pr=tree) ```diff @@Coverage Diff @@ ## master#4821 +/- ## == - Coverage 75.77% 75.76% -0.01% == Files 461 461 Lines 2995529927 -28 == - Hits2269722674 -23 + Misses 7258 7253 -5 ``` | [Impacted Files](https://codecov.io/gh/apache/airflow/pull/4821?src=pr=tree) | Coverage Δ | | |---|---|---| | [airflow/bin/cli.py](https://codecov.io/gh/apache/airflow/pull/4821/diff?src=pr=tree#diff-YWlyZmxvdy9iaW4vY2xpLnB5) | `66.52% <82.82%> (-0.45%)` | :arrow_down: | -- [Continue to review full report at Codecov](https://codecov.io/gh/apache/airflow/pull/4821?src=pr=continue). > **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta) > `Δ = absolute (impact)`, `ø = not affected`, `? = missing data` > Powered by [Codecov](https://codecov.io/gh/apache/airflow/pull/4821?src=pr=footer). Last update [4d98a41...62fb38d](https://codecov.io/gh/apache/airflow/pull/4821?src=pr=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments). This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [airflow] codecov-io edited a comment on issue #4821: [AIRFLOW-3998] Use nested commands in cli.
codecov-io edited a comment on issue #4821: [AIRFLOW-3998] Use nested commands in cli. URL: https://github.com/apache/airflow/pull/4821#issuecomment-468981700 # [Codecov](https://codecov.io/gh/apache/airflow/pull/4821?src=pr=h1) Report > Merging [#4821](https://codecov.io/gh/apache/airflow/pull/4821?src=pr=desc) into [master](https://codecov.io/gh/apache/airflow/commit/4d98a4127a7846e75261c0c1297a8cbadb91b9ab?src=pr=desc) will **decrease** coverage by `<.01%`. > The diff coverage is `82.82%`. [](https://codecov.io/gh/apache/airflow/pull/4821?src=pr=tree) ```diff @@Coverage Diff @@ ## master#4821 +/- ## == - Coverage 75.77% 75.76% -0.01% == Files 461 461 Lines 2995529927 -28 == - Hits2269722674 -23 + Misses 7258 7253 -5 ``` | [Impacted Files](https://codecov.io/gh/apache/airflow/pull/4821?src=pr=tree) | Coverage Δ | | |---|---|---| | [airflow/bin/cli.py](https://codecov.io/gh/apache/airflow/pull/4821/diff?src=pr=tree#diff-YWlyZmxvdy9iaW4vY2xpLnB5) | `66.52% <82.82%> (-0.45%)` | :arrow_down: | -- [Continue to review full report at Codecov](https://codecov.io/gh/apache/airflow/pull/4821?src=pr=continue). > **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta) > `Δ = absolute (impact)`, `ø = not affected`, `? = missing data` > Powered by [Codecov](https://codecov.io/gh/apache/airflow/pull/4821?src=pr=footer). Last update [4d98a41...62fb38d](https://codecov.io/gh/apache/airflow/pull/4821?src=pr=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments). This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [airflow] codecov-io edited a comment on issue #4821: [AIRFLOW-3998] Use nested commands in cli.
codecov-io edited a comment on issue #4821: [AIRFLOW-3998] Use nested commands in cli. URL: https://github.com/apache/airflow/pull/4821#issuecomment-468981700 # [Codecov](https://codecov.io/gh/apache/airflow/pull/4821?src=pr=h1) Report > Merging [#4821](https://codecov.io/gh/apache/airflow/pull/4821?src=pr=desc) into [master](https://codecov.io/gh/apache/airflow/commit/4d98a4127a7846e75261c0c1297a8cbadb91b9ab?src=pr=desc) will **decrease** coverage by `<.01%`. > The diff coverage is `82.82%`. [](https://codecov.io/gh/apache/airflow/pull/4821?src=pr=tree) ```diff @@Coverage Diff @@ ## master#4821 +/- ## == - Coverage 75.77% 75.76% -0.01% == Files 461 461 Lines 2995529927 -28 == - Hits2269722674 -23 + Misses 7258 7253 -5 ``` | [Impacted Files](https://codecov.io/gh/apache/airflow/pull/4821?src=pr=tree) | Coverage Δ | | |---|---|---| | [airflow/bin/cli.py](https://codecov.io/gh/apache/airflow/pull/4821/diff?src=pr=tree#diff-YWlyZmxvdy9iaW4vY2xpLnB5) | `66.52% <82.82%> (-0.45%)` | :arrow_down: | -- [Continue to review full report at Codecov](https://codecov.io/gh/apache/airflow/pull/4821?src=pr=continue). > **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta) > `Δ = absolute (impact)`, `ø = not affected`, `? = missing data` > Powered by [Codecov](https://codecov.io/gh/apache/airflow/pull/4821?src=pr=footer). Last update [4d98a41...62fb38d](https://codecov.io/gh/apache/airflow/pull/4821?src=pr=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments). This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [airflow] codecov-io edited a comment on issue #4861: [AIRFLOW-4034] Make string formatting explicit by removing **locals()
codecov-io edited a comment on issue #4861: [AIRFLOW-4034] Make string formatting explicit by removing **locals() URL: https://github.com/apache/airflow/pull/4861#issuecomment-478360669 # [Codecov](https://codecov.io/gh/apache/airflow/pull/4861?src=pr=h1) Report > Merging [#4861](https://codecov.io/gh/apache/airflow/pull/4861?src=pr=desc) into [master](https://codecov.io/gh/apache/airflow/commit/157f7bff837dda691012da8638dbc570bf5f1844?src=pr=desc) will **increase** coverage by `<.01%`. > The diff coverage is `82.75%`. [](https://codecov.io/gh/apache/airflow/pull/4861?src=pr=tree) ```diff @@Coverage Diff @@ ## master#4861 +/- ## == + Coverage 75.76% 75.76% +<.01% == Files 461 461 Lines 2995529949 -6 == - Hits2269422691 -3 + Misses 7261 7258 -3 ``` | [Impacted Files](https://codecov.io/gh/apache/airflow/pull/4861?src=pr=tree) | Coverage Δ | | |---|---|---| | [airflow/configuration.py](https://codecov.io/gh/apache/airflow/pull/4861/diff?src=pr=tree#diff-YWlyZmxvdy9jb25maWd1cmF0aW9uLnB5) | `92.12% <ø> (-0.06%)` | :arrow_down: | | [...low/contrib/operators/wasb\_delete\_blob\_operator.py](https://codecov.io/gh/apache/airflow/pull/4861/diff?src=pr=tree#diff-YWlyZmxvdy9jb250cmliL29wZXJhdG9ycy93YXNiX2RlbGV0ZV9ibG9iX29wZXJhdG9yLnB5) | `100% <ø> (ø)` | :arrow_up: | | [airflow/www/utils.py](https://codecov.io/gh/apache/airflow/pull/4861/diff?src=pr=tree#diff-YWlyZmxvdy93d3cvdXRpbHMucHk=) | `82.7% <ø> (ø)` | :arrow_up: | | [airflow/contrib/operators/file\_to\_wasb.py](https://codecov.io/gh/apache/airflow/pull/4861/diff?src=pr=tree#diff-YWlyZmxvdy9jb250cmliL29wZXJhdG9ycy9maWxlX3RvX3dhc2IucHk=) | `100% <ø> (ø)` | :arrow_up: | | [airflow/operators/hive\_to\_druid.py](https://codecov.io/gh/apache/airflow/pull/4861/diff?src=pr=tree#diff-YWlyZmxvdy9vcGVyYXRvcnMvaGl2ZV90b19kcnVpZC5weQ==) | `63.07% <ø> (ø)` | :arrow_up: | | [airflow/utils/helpers.py](https://codecov.io/gh/apache/airflow/pull/4861/diff?src=pr=tree#diff-YWlyZmxvdy91dGlscy9oZWxwZXJzLnB5) | `82.51% <ø> (ø)` | :arrow_up: | | [airflow/hooks/mysql\_hook.py](https://codecov.io/gh/apache/airflow/pull/4861/diff?src=pr=tree#diff-YWlyZmxvdy9ob29rcy9teXNxbF9ob29rLnB5) | `92.98% <ø> (ø)` | :arrow_up: | | [airflow/contrib/operators/qubole\_check\_operator.py](https://codecov.io/gh/apache/airflow/pull/4861/diff?src=pr=tree#diff-YWlyZmxvdy9jb250cmliL29wZXJhdG9ycy9xdWJvbGVfY2hlY2tfb3BlcmF0b3IucHk=) | `71.76% <ø> (ø)` | :arrow_up: | | [airflow/contrib/sensors/wasb\_sensor.py](https://codecov.io/gh/apache/airflow/pull/4861/diff?src=pr=tree#diff-YWlyZmxvdy9jb250cmliL3NlbnNvcnMvd2FzYl9zZW5zb3IucHk=) | `100% <ø> (ø)` | :arrow_up: | | [airflow/bin/cli.py](https://codecov.io/gh/apache/airflow/pull/4861/diff?src=pr=tree#diff-YWlyZmxvdy9iaW4vY2xpLnB5) | `66.97% <0%> (ø)` | :arrow_up: | | ... and [11 more](https://codecov.io/gh/apache/airflow/pull/4861/diff?src=pr=tree-more) | | -- [Continue to review full report at Codecov](https://codecov.io/gh/apache/airflow/pull/4861?src=pr=continue). > **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta) > `Δ = absolute (impact)`, `ø = not affected`, `? = missing data` > Powered by [Codecov](https://codecov.io/gh/apache/airflow/pull/4861?src=pr=footer). Last update [157f7bf...3cfe573](https://codecov.io/gh/apache/airflow/pull/4861?src=pr=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments). This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [airflow] codecov-io commented on issue #4861: [AIRFLOW-4034] Make string formatting explicit by removing **locals()
codecov-io commented on issue #4861: [AIRFLOW-4034] Make string formatting explicit by removing **locals() URL: https://github.com/apache/airflow/pull/4861#issuecomment-478360669 # [Codecov](https://codecov.io/gh/apache/airflow/pull/4861?src=pr=h1) Report > Merging [#4861](https://codecov.io/gh/apache/airflow/pull/4861?src=pr=desc) into [master](https://codecov.io/gh/apache/airflow/commit/157f7bff837dda691012da8638dbc570bf5f1844?src=pr=desc) will **increase** coverage by `<.01%`. > The diff coverage is `82.75%`. [](https://codecov.io/gh/apache/airflow/pull/4861?src=pr=tree) ```diff @@Coverage Diff @@ ## master#4861 +/- ## == + Coverage 75.76% 75.76% +<.01% == Files 461 461 Lines 2995529949 -6 == - Hits2269422691 -3 + Misses 7261 7258 -3 ``` | [Impacted Files](https://codecov.io/gh/apache/airflow/pull/4861?src=pr=tree) | Coverage Δ | | |---|---|---| | [airflow/configuration.py](https://codecov.io/gh/apache/airflow/pull/4861/diff?src=pr=tree#diff-YWlyZmxvdy9jb25maWd1cmF0aW9uLnB5) | `92.12% <ø> (-0.06%)` | :arrow_down: | | [...low/contrib/operators/wasb\_delete\_blob\_operator.py](https://codecov.io/gh/apache/airflow/pull/4861/diff?src=pr=tree#diff-YWlyZmxvdy9jb250cmliL29wZXJhdG9ycy93YXNiX2RlbGV0ZV9ibG9iX29wZXJhdG9yLnB5) | `100% <ø> (ø)` | :arrow_up: | | [airflow/www/utils.py](https://codecov.io/gh/apache/airflow/pull/4861/diff?src=pr=tree#diff-YWlyZmxvdy93d3cvdXRpbHMucHk=) | `82.7% <ø> (ø)` | :arrow_up: | | [airflow/contrib/operators/file\_to\_wasb.py](https://codecov.io/gh/apache/airflow/pull/4861/diff?src=pr=tree#diff-YWlyZmxvdy9jb250cmliL29wZXJhdG9ycy9maWxlX3RvX3dhc2IucHk=) | `100% <ø> (ø)` | :arrow_up: | | [airflow/operators/hive\_to\_druid.py](https://codecov.io/gh/apache/airflow/pull/4861/diff?src=pr=tree#diff-YWlyZmxvdy9vcGVyYXRvcnMvaGl2ZV90b19kcnVpZC5weQ==) | `63.07% <ø> (ø)` | :arrow_up: | | [airflow/utils/helpers.py](https://codecov.io/gh/apache/airflow/pull/4861/diff?src=pr=tree#diff-YWlyZmxvdy91dGlscy9oZWxwZXJzLnB5) | `82.51% <ø> (ø)` | :arrow_up: | | [airflow/hooks/mysql\_hook.py](https://codecov.io/gh/apache/airflow/pull/4861/diff?src=pr=tree#diff-YWlyZmxvdy9ob29rcy9teXNxbF9ob29rLnB5) | `92.98% <ø> (ø)` | :arrow_up: | | [airflow/contrib/operators/qubole\_check\_operator.py](https://codecov.io/gh/apache/airflow/pull/4861/diff?src=pr=tree#diff-YWlyZmxvdy9jb250cmliL29wZXJhdG9ycy9xdWJvbGVfY2hlY2tfb3BlcmF0b3IucHk=) | `71.76% <ø> (ø)` | :arrow_up: | | [airflow/contrib/sensors/wasb\_sensor.py](https://codecov.io/gh/apache/airflow/pull/4861/diff?src=pr=tree#diff-YWlyZmxvdy9jb250cmliL3NlbnNvcnMvd2FzYl9zZW5zb3IucHk=) | `100% <ø> (ø)` | :arrow_up: | | [airflow/bin/cli.py](https://codecov.io/gh/apache/airflow/pull/4861/diff?src=pr=tree#diff-YWlyZmxvdy9iaW4vY2xpLnB5) | `66.97% <0%> (ø)` | :arrow_up: | | ... and [11 more](https://codecov.io/gh/apache/airflow/pull/4861/diff?src=pr=tree-more) | | -- [Continue to review full report at Codecov](https://codecov.io/gh/apache/airflow/pull/4861?src=pr=continue). > **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta) > `Δ = absolute (impact)`, `ø = not affected`, `? = missing data` > Powered by [Codecov](https://codecov.io/gh/apache/airflow/pull/4861?src=pr=footer). Last update [157f7bf...3cfe573](https://codecov.io/gh/apache/airflow/pull/4861?src=pr=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments). This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Commented] (AIRFLOW-4172) Invalid parameter --driver-classpath in SparkSubmitHook
[ https://issues.apache.org/jira/browse/AIRFLOW-4172?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16806199#comment-16806199 ] ASF GitHub Bot commented on AIRFLOW-4172: - Fokko commented on pull request #4992: [AIRFLOW-4172] Fix changes for driver class path option in Spark Subm… URL: https://github.com/apache/airflow/pull/4992 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org > Invalid parameter --driver-classpath in SparkSubmitHook > --- > > Key: AIRFLOW-4172 > URL: https://issues.apache.org/jira/browse/AIRFLOW-4172 > Project: Apache Airflow > Issue Type: Bug > Components: contrib >Affects Versions: 1.10.0, 2.0.0 >Reporter: saurabh gulati >Assignee: saurabh gulati >Priority: Critical > Fix For: 1.10.0 > > > Spark option --driver-class-path is set wrongly as --driver-classpath while > building Spark Submit command in SparkSubmitHook which results in failure > while submitting spark job -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (AIRFLOW-4172) Invalid parameter --driver-classpath in SparkSubmitHook
[ https://issues.apache.org/jira/browse/AIRFLOW-4172?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Fokko Driesprong updated AIRFLOW-4172: -- Fix Version/s: (was: 1.10.0) 2.0.0 > Invalid parameter --driver-classpath in SparkSubmitHook > --- > > Key: AIRFLOW-4172 > URL: https://issues.apache.org/jira/browse/AIRFLOW-4172 > Project: Apache Airflow > Issue Type: Bug > Components: contrib >Affects Versions: 1.10.0, 2.0.0 >Reporter: saurabh gulati >Assignee: saurabh gulati >Priority: Critical > Fix For: 2.0.0 > > > Spark option --driver-class-path is set wrongly as --driver-classpath while > building Spark Submit command in SparkSubmitHook which results in failure > while submitting spark job -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Resolved] (AIRFLOW-4172) Invalid parameter --driver-classpath in SparkSubmitHook
[ https://issues.apache.org/jira/browse/AIRFLOW-4172?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Fokko Driesprong resolved AIRFLOW-4172. --- Resolution: Fixed > Invalid parameter --driver-classpath in SparkSubmitHook > --- > > Key: AIRFLOW-4172 > URL: https://issues.apache.org/jira/browse/AIRFLOW-4172 > Project: Apache Airflow > Issue Type: Bug > Components: contrib >Affects Versions: 1.10.0, 2.0.0 >Reporter: saurabh gulati >Assignee: saurabh gulati >Priority: Critical > Fix For: 2.0.0 > > > Spark option --driver-class-path is set wrongly as --driver-classpath while > building Spark Submit command in SparkSubmitHook which results in failure > while submitting spark job -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (AIRFLOW-4172) Invalid parameter --driver-classpath in SparkSubmitHook
[ https://issues.apache.org/jira/browse/AIRFLOW-4172?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16806202#comment-16806202 ] ASF subversion and git services commented on AIRFLOW-4172: -- Commit 06d2f53a32b928a8fe2fdafc385b4fd7406ac62f in airflow's branch refs/heads/master from saurabh gulati [ https://gitbox.apache.org/repos/asf?p=airflow.git;h=06d2f53 ] [AIRFLOW-4172] Fix changes for driver class path option in Spark Subm… (#4992) * [AIRFLOW-4172] Fix changes for driver class path option in Spark Submit Operator * [AIRFLOW-4172] Fix changes for driver class path option in Spark Submit > Invalid parameter --driver-classpath in SparkSubmitHook > --- > > Key: AIRFLOW-4172 > URL: https://issues.apache.org/jira/browse/AIRFLOW-4172 > Project: Apache Airflow > Issue Type: Bug > Components: contrib >Affects Versions: 1.10.0, 2.0.0 >Reporter: saurabh gulati >Assignee: saurabh gulati >Priority: Critical > Fix For: 1.10.0 > > > Spark option --driver-class-path is set wrongly as --driver-classpath while > building Spark Submit command in SparkSubmitHook which results in failure > while submitting spark job -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (AIRFLOW-4172) Invalid parameter --driver-classpath in SparkSubmitHook
[ https://issues.apache.org/jira/browse/AIRFLOW-4172?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16806201#comment-16806201 ] ASF subversion and git services commented on AIRFLOW-4172: -- Commit 06d2f53a32b928a8fe2fdafc385b4fd7406ac62f in airflow's branch refs/heads/master from saurabh gulati [ https://gitbox.apache.org/repos/asf?p=airflow.git;h=06d2f53 ] [AIRFLOW-4172] Fix changes for driver class path option in Spark Subm… (#4992) * [AIRFLOW-4172] Fix changes for driver class path option in Spark Submit Operator * [AIRFLOW-4172] Fix changes for driver class path option in Spark Submit > Invalid parameter --driver-classpath in SparkSubmitHook > --- > > Key: AIRFLOW-4172 > URL: https://issues.apache.org/jira/browse/AIRFLOW-4172 > Project: Apache Airflow > Issue Type: Bug > Components: contrib >Affects Versions: 1.10.0, 2.0.0 >Reporter: saurabh gulati >Assignee: saurabh gulati >Priority: Critical > Fix For: 1.10.0 > > > Spark option --driver-class-path is set wrongly as --driver-classpath while > building Spark Submit command in SparkSubmitHook which results in failure > while submitting spark job -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[GitHub] [airflow] Fokko merged pull request #4992: [AIRFLOW-4172] Fix changes for driver class path option in Spark Subm…
Fokko merged pull request #4992: [AIRFLOW-4172] Fix changes for driver class path option in Spark Subm… URL: https://github.com/apache/airflow/pull/4992 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Updated] (AIRFLOW-4193) Remove code duplication in test_gcp_api_base_hook
[ https://issues.apache.org/jira/browse/AIRFLOW-4193?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Fokko Driesprong updated AIRFLOW-4193: -- Fix Version/s: 2.0.0 > Remove code duplication in test_gcp_api_base_hook > - > > Key: AIRFLOW-4193 > URL: https://issues.apache.org/jira/browse/AIRFLOW-4193 > Project: Apache Airflow > Issue Type: Improvement >Reporter: Bas Harenslak >Assignee: Bas Harenslak >Priority: Major > Fix For: 2.0.0 > > > test_gcp_api_base_hook contains 5 tests which test pretty much the same > thing. `@parameterized` was already used in some way, so generalized all 5 > tests and removed lots of duplicate code as a result. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[GitHub] [airflow] BasPH commented on issue #5010: [AIRFLOW-4192] Reorganize task context variables
BasPH commented on issue #5010: [AIRFLOW-4192] Reorganize task context variables URL: https://github.com/apache/airflow/pull/5010#issuecomment-478358376 @Fokko after the last push 1 task failed, which succeeded before, can you rerun that one? This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [airflow] BasPH commented on issue #5010: [AIRFLOW-4192] Reorganize task context variables
BasPH commented on issue #5010: [AIRFLOW-4192] Reorganize task context variables URL: https://github.com/apache/airflow/pull/5010#issuecomment-478358280 @Fokko `yesterday_ds` is removed and `next_execution_date` is still in there, were on the same page then This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [airflow] Fokko commented on issue #4861: [AIRFLOW-4034] Make string formatting explicit by removing **locals()
Fokko commented on issue #4861: [AIRFLOW-4034] Make string formatting explicit by removing **locals() URL: https://github.com/apache/airflow/pull/4861#issuecomment-478358175 Restarted the job! Let's see. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Work started] (AIRFLOW-4193) Remove code duplication in test_gcp_api_base_hook
[ https://issues.apache.org/jira/browse/AIRFLOW-4193?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Work on AIRFLOW-4193 started by Fokko Driesprong. - > Remove code duplication in test_gcp_api_base_hook > - > > Key: AIRFLOW-4193 > URL: https://issues.apache.org/jira/browse/AIRFLOW-4193 > Project: Apache Airflow > Issue Type: Improvement >Reporter: Bas Harenslak >Assignee: Fokko Driesprong >Priority: Major > Fix For: 2.0.0 > > > test_gcp_api_base_hook contains 5 tests which test pretty much the same > thing. `@parameterized` was already used in some way, so generalized all 5 > tests and removed lots of duplicate code as a result. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (AIRFLOW-4193) Remove code duplication in test_gcp_api_base_hook
[ https://issues.apache.org/jira/browse/AIRFLOW-4193?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16806190#comment-16806190 ] ASF GitHub Bot commented on AIRFLOW-4193: - Fokko commented on pull request #5011: [AIRFLOW-4193] Remove code duplication in test_gcp_api_base_hook URL: https://github.com/apache/airflow/pull/5011 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org > Remove code duplication in test_gcp_api_base_hook > - > > Key: AIRFLOW-4193 > URL: https://issues.apache.org/jira/browse/AIRFLOW-4193 > Project: Apache Airflow > Issue Type: Improvement >Reporter: Bas Harenslak >Priority: Major > > test_gcp_api_base_hook contains 5 tests which test pretty much the same > thing. `@parameterized` was already used in some way, so generalized all 5 > tests and removed lots of duplicate code as a result. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (AIRFLOW-4193) Remove code duplication in test_gcp_api_base_hook
[ https://issues.apache.org/jira/browse/AIRFLOW-4193?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16806191#comment-16806191 ] ASF subversion and git services commented on AIRFLOW-4193: -- Commit 4d98a4127a7846e75261c0c1297a8cbadb91b9ab in airflow's branch refs/heads/master from BasPH [ https://gitbox.apache.org/repos/asf?p=airflow.git;h=4d98a41 ] [AIRFLOW-4193] Remove code duplication in test_gcp_api_base_hook (#5011) > Remove code duplication in test_gcp_api_base_hook > - > > Key: AIRFLOW-4193 > URL: https://issues.apache.org/jira/browse/AIRFLOW-4193 > Project: Apache Airflow > Issue Type: Improvement >Reporter: Bas Harenslak >Priority: Major > > test_gcp_api_base_hook contains 5 tests which test pretty much the same > thing. `@parameterized` was already used in some way, so generalized all 5 > tests and removed lots of duplicate code as a result. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Resolved] (AIRFLOW-4193) Remove code duplication in test_gcp_api_base_hook
[ https://issues.apache.org/jira/browse/AIRFLOW-4193?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Fokko Driesprong resolved AIRFLOW-4193. --- Resolution: Fixed > Remove code duplication in test_gcp_api_base_hook > - > > Key: AIRFLOW-4193 > URL: https://issues.apache.org/jira/browse/AIRFLOW-4193 > Project: Apache Airflow > Issue Type: Improvement >Reporter: Bas Harenslak >Assignee: Fokko Driesprong >Priority: Major > Fix For: 2.0.0 > > > test_gcp_api_base_hook contains 5 tests which test pretty much the same > thing. `@parameterized` was already used in some way, so generalized all 5 > tests and removed lots of duplicate code as a result. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[GitHub] [airflow] Fokko merged pull request #5011: [AIRFLOW-4193] Remove code duplication in test_gcp_api_base_hook
Fokko merged pull request #5011: [AIRFLOW-4193] Remove code duplication in test_gcp_api_base_hook URL: https://github.com/apache/airflow/pull/5011 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Updated] (AIRFLOW-3458) Refactor: Move Connection out of models.py
[ https://issues.apache.org/jira/browse/AIRFLOW-3458?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Fokko Driesprong updated AIRFLOW-3458: -- Fix Version/s: (was: 1.10.3) 2.0.0 > Refactor: Move Connection out of models.py > -- > > Key: AIRFLOW-3458 > URL: https://issues.apache.org/jira/browse/AIRFLOW-3458 > Project: Apache Airflow > Issue Type: Task > Components: models >Affects Versions: 1.10.1 >Reporter: Fokko Driesprong >Assignee: Bas Harenslak >Priority: Major > Fix For: 2.0.0 > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[GitHub] [airflow] Fokko commented on issue #5013: [AIRFLOW-XXX] Omit vendor packages and example dags from being covered by codecov
Fokko commented on issue #5013: [AIRFLOW-XXX] Omit vendor packages and example dags from being covered by codecov URL: https://github.com/apache/airflow/pull/5013#issuecomment-478357601 I know that (at least some) DAGs are used in the tests, but having coverage would be nice here. I'm not really interested in artificially increasing the code coverage, as I strongly believe it is a vanity metric. I do think we need to exclude `airflow/_vendor/*`. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Commented] (AIRFLOW-3458) Refactor: Move Connection out of models.py
[ https://issues.apache.org/jira/browse/AIRFLOW-3458?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16806186#comment-16806186 ] Fokko Driesprong commented on AIRFLOW-3458: --- [~ash] Do you want to cherry-pick this into 1.10.3? I don't know if you want to have the hassle of the cherry-picking here. > Refactor: Move Connection out of models.py > -- > > Key: AIRFLOW-3458 > URL: https://issues.apache.org/jira/browse/AIRFLOW-3458 > Project: Apache Airflow > Issue Type: Task > Components: models >Affects Versions: 1.10.1 >Reporter: Fokko Driesprong >Assignee: Bas Harenslak >Priority: Major > Fix For: 2.0.0 > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (AIRFLOW-3458) Refactor: Move Connection out of models.py
[ https://issues.apache.org/jira/browse/AIRFLOW-3458?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16806185#comment-16806185 ] ASF subversion and git services commented on AIRFLOW-3458: -- Commit 675bfcdd10178d45717ed3a64a50c51e76eb9e3d in airflow's branch refs/heads/master from BasPH [ https://gitbox.apache.org/repos/asf?p=airflow.git;h=675bfcd ] [AIRFLOW-3458] Move connection tests (#4680) > Refactor: Move Connection out of models.py > -- > > Key: AIRFLOW-3458 > URL: https://issues.apache.org/jira/browse/AIRFLOW-3458 > Project: Apache Airflow > Issue Type: Task > Components: models >Affects Versions: 1.10.1 >Reporter: Fokko Driesprong >Assignee: Bas Harenslak >Priority: Major > Fix For: 1.10.3 > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (AIRFLOW-3458) Refactor: Move Connection out of models.py
[ https://issues.apache.org/jira/browse/AIRFLOW-3458?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16806184#comment-16806184 ] ASF GitHub Bot commented on AIRFLOW-3458: - Fokko commented on pull request #4680: [AIRFLOW-3458] Move connection tests URL: https://github.com/apache/airflow/pull/4680 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org > Refactor: Move Connection out of models.py > -- > > Key: AIRFLOW-3458 > URL: https://issues.apache.org/jira/browse/AIRFLOW-3458 > Project: Apache Airflow > Issue Type: Task > Components: models >Affects Versions: 1.10.1 >Reporter: Fokko Driesprong >Assignee: Bas Harenslak >Priority: Major > Fix For: 1.10.3 > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[GitHub] [airflow] Fokko merged pull request #4680: [AIRFLOW-3458] Move connection tests
Fokko merged pull request #4680: [AIRFLOW-3458] Move connection tests URL: https://github.com/apache/airflow/pull/4680 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [airflow] Fokko edited a comment on issue #5010: [AIRFLOW-4192] Reorganize task context variables
Fokko edited a comment on issue #5010: [AIRFLOW-4192] Reorganize task context
variables
URL: https://github.com/apache/airflow/pull/5010#issuecomment-478356870
Thanks for working on this @BasPH
I believe that `yesterday_ds` does not make sense since we cannot assume
that the schedule is daily. I don't see any usage of this variable. Personally
I do use `next_execution_date` quite extensively. When you have a job that runs
daily, but you want to change this to an hourly job. In such a case you don't
want to change `{{ (execution_date + macros.timedelta(days=1)) }}` to `{{
(execution_date + macros.timedelta(hours=1)) }}`everywhere.
@bolkedebruin What was your plan with the `{in,out}lets`?
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [airflow] Fokko commented on issue #5010: [AIRFLOW-4192] Reorganize task context variables
Fokko commented on issue #5010: [AIRFLOW-4192] Reorganize task context variables
URL: https://github.com/apache/airflow/pull/5010#issuecomment-478356870
Thanks for working on thie @BasPH
I believe that `yesterday_ds` does not make sense since we cannot assume
that the schedule is daily. I don't see any usage of this variable. Personally
I do use `next_execution_date` quite extensively. When you have a job that runs
daily, but you want to change this to an hourly job. In such a case you don't
want to change `{{ (execution_date + macros.timedelta(days=1)) }}` to `{{
(execution_date + macros.timedelta(hours=1)) }}`everywhere.
@bolkedebruin What was your plan with the `{in,out}lets`?
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [airflow] codecov-io edited a comment on issue #4821: [AIRFLOW-3998] Use nested commands in cli.
codecov-io edited a comment on issue #4821: [AIRFLOW-3998] Use nested commands in cli. URL: https://github.com/apache/airflow/pull/4821#issuecomment-468981700 # [Codecov](https://codecov.io/gh/apache/airflow/pull/4821?src=pr=h1) Report > Merging [#4821](https://codecov.io/gh/apache/airflow/pull/4821?src=pr=desc) into [master](https://codecov.io/gh/apache/airflow/commit/157f7bff837dda691012da8638dbc570bf5f1844?src=pr=desc) will **decrease** coverage by `<.01%`. > The diff coverage is `82.68%`. [](https://codecov.io/gh/apache/airflow/pull/4821?src=pr=tree) ```diff @@Coverage Diff @@ ## master#4821 +/- ## == - Coverage 75.76% 75.76% -0.01% == Files 461 461 Lines 2995529930 -25 == - Hits2269422675 -19 + Misses 7261 7255 -6 ``` | [Impacted Files](https://codecov.io/gh/apache/airflow/pull/4821?src=pr=tree) | Coverage Δ | | |---|---|---| | [airflow/bin/cli.py](https://codecov.io/gh/apache/airflow/pull/4821/diff?src=pr=tree#diff-YWlyZmxvdy9iaW4vY2xpLnB5) | `66.63% <82.68%> (-0.35%)` | :arrow_down: | | [airflow/models/\_\_init\_\_.py](https://codecov.io/gh/apache/airflow/pull/4821/diff?src=pr=tree#diff-YWlyZmxvdy9tb2RlbHMvX19pbml0X18ucHk=) | `93.01% <0%> (+0.04%)` | :arrow_up: | -- [Continue to review full report at Codecov](https://codecov.io/gh/apache/airflow/pull/4821?src=pr=continue). > **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta) > `Δ = absolute (impact)`, `ø = not affected`, `? = missing data` > Powered by [Codecov](https://codecov.io/gh/apache/airflow/pull/4821?src=pr=footer). Last update [157f7bf...1e70b68](https://codecov.io/gh/apache/airflow/pull/4821?src=pr=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments). This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [airflow] codecov-io edited a comment on issue #4821: [AIRFLOW-3998] Use nested commands in cli.
codecov-io edited a comment on issue #4821: [AIRFLOW-3998] Use nested commands in cli. URL: https://github.com/apache/airflow/pull/4821#issuecomment-468981700 # [Codecov](https://codecov.io/gh/apache/airflow/pull/4821?src=pr=h1) Report > Merging [#4821](https://codecov.io/gh/apache/airflow/pull/4821?src=pr=desc) into [master](https://codecov.io/gh/apache/airflow/commit/157f7bff837dda691012da8638dbc570bf5f1844?src=pr=desc) will **decrease** coverage by `<.01%`. > The diff coverage is `82.68%`. [](https://codecov.io/gh/apache/airflow/pull/4821?src=pr=tree) ```diff @@Coverage Diff @@ ## master#4821 +/- ## == - Coverage 75.76% 75.76% -0.01% == Files 461 461 Lines 2995529930 -25 == - Hits2269422675 -19 + Misses 7261 7255 -6 ``` | [Impacted Files](https://codecov.io/gh/apache/airflow/pull/4821?src=pr=tree) | Coverage Δ | | |---|---|---| | [airflow/bin/cli.py](https://codecov.io/gh/apache/airflow/pull/4821/diff?src=pr=tree#diff-YWlyZmxvdy9iaW4vY2xpLnB5) | `66.63% <82.68%> (-0.35%)` | :arrow_down: | | [airflow/models/\_\_init\_\_.py](https://codecov.io/gh/apache/airflow/pull/4821/diff?src=pr=tree#diff-YWlyZmxvdy9tb2RlbHMvX19pbml0X18ucHk=) | `93.01% <0%> (+0.04%)` | :arrow_up: | -- [Continue to review full report at Codecov](https://codecov.io/gh/apache/airflow/pull/4821?src=pr=continue). > **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta) > `Δ = absolute (impact)`, `ø = not affected`, `? = missing data` > Powered by [Codecov](https://codecov.io/gh/apache/airflow/pull/4821?src=pr=footer). Last update [157f7bf...1e70b68](https://codecov.io/gh/apache/airflow/pull/4821?src=pr=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments). This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [airflow] BasPH commented on issue #4680: [AIRFLOW-3458] Move connection tests
BasPH commented on issue #4680: [AIRFLOW-3458] Move connection tests URL: https://github.com/apache/airflow/pull/4680#issuecomment-478352824 Finally managed to find some time. One failing test has been removed in the meantime, now the CI is nice and green :) This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [airflow] codecov-io edited a comment on issue #5010: [AIRFLOW-4192] Reorganize task context variables
codecov-io edited a comment on issue #5010: [AIRFLOW-4192] Reorganize task context variables URL: https://github.com/apache/airflow/pull/5010#issuecomment-478295602 # [Codecov](https://codecov.io/gh/apache/airflow/pull/5010?src=pr=h1) Report > Merging [#5010](https://codecov.io/gh/apache/airflow/pull/5010?src=pr=desc) into [master](https://codecov.io/gh/apache/airflow/commit/157f7bff837dda691012da8638dbc570bf5f1844?src=pr=desc) will **increase** coverage by `<.01%`. > The diff coverage is `100%`. [](https://codecov.io/gh/apache/airflow/pull/5010?src=pr=tree) ```diff @@Coverage Diff @@ ## master#5010 +/- ## == + Coverage 75.76% 75.76% +<.01% == Files 461 461 Lines 2995529948 -7 == - Hits2269422689 -5 + Misses 7261 7259 -2 ``` | [Impacted Files](https://codecov.io/gh/apache/airflow/pull/5010?src=pr=tree) | Coverage Δ | | |---|---|---| | [...flow/contrib/example\_dags/example\_qubole\_sensor.py](https://codecov.io/gh/apache/airflow/pull/5010/diff?src=pr=tree#diff-YWlyZmxvdy9jb250cmliL2V4YW1wbGVfZGFncy9leGFtcGxlX3F1Ym9sZV9zZW5zb3IucHk=) | `0% <ø> (ø)` | :arrow_up: | | [airflow/models/\_\_init\_\_.py](https://codecov.io/gh/apache/airflow/pull/5010/diff?src=pr=tree#diff-YWlyZmxvdy9tb2RlbHMvX19pbml0X18ucHk=) | `93.03% <100%> (+0.07%)` | :arrow_up: | | [airflow/operators/hive\_to\_druid.py](https://codecov.io/gh/apache/airflow/pull/5010/diff?src=pr=tree#diff-YWlyZmxvdy9vcGVyYXRvcnMvaGl2ZV90b19kcnVpZC5weQ==) | `63.07% <100%> (ø)` | :arrow_up: | -- [Continue to review full report at Codecov](https://codecov.io/gh/apache/airflow/pull/5010?src=pr=continue). > **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta) > `Δ = absolute (impact)`, `ø = not affected`, `? = missing data` > Powered by [Codecov](https://codecov.io/gh/apache/airflow/pull/5010?src=pr=footer). Last update [157f7bf...aa0f577](https://codecov.io/gh/apache/airflow/pull/5010?src=pr=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments). This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [airflow] codecov-io edited a comment on issue #5010: [AIRFLOW-4192] Reorganize task context variables
codecov-io edited a comment on issue #5010: [AIRFLOW-4192] Reorganize task context variables URL: https://github.com/apache/airflow/pull/5010#issuecomment-478295602 # [Codecov](https://codecov.io/gh/apache/airflow/pull/5010?src=pr=h1) Report > Merging [#5010](https://codecov.io/gh/apache/airflow/pull/5010?src=pr=desc) into [master](https://codecov.io/gh/apache/airflow/commit/157f7bff837dda691012da8638dbc570bf5f1844?src=pr=desc) will **increase** coverage by `<.01%`. > The diff coverage is `100%`. [](https://codecov.io/gh/apache/airflow/pull/5010?src=pr=tree) ```diff @@Coverage Diff @@ ## master#5010 +/- ## == + Coverage 75.76% 75.76% +<.01% == Files 461 461 Lines 2995529948 -7 == - Hits2269422689 -5 + Misses 7261 7259 -2 ``` | [Impacted Files](https://codecov.io/gh/apache/airflow/pull/5010?src=pr=tree) | Coverage Δ | | |---|---|---| | [...flow/contrib/example\_dags/example\_qubole\_sensor.py](https://codecov.io/gh/apache/airflow/pull/5010/diff?src=pr=tree#diff-YWlyZmxvdy9jb250cmliL2V4YW1wbGVfZGFncy9leGFtcGxlX3F1Ym9sZV9zZW5zb3IucHk=) | `0% <ø> (ø)` | :arrow_up: | | [airflow/models/\_\_init\_\_.py](https://codecov.io/gh/apache/airflow/pull/5010/diff?src=pr=tree#diff-YWlyZmxvdy9tb2RlbHMvX19pbml0X18ucHk=) | `93.03% <100%> (+0.07%)` | :arrow_up: | | [airflow/operators/hive\_to\_druid.py](https://codecov.io/gh/apache/airflow/pull/5010/diff?src=pr=tree#diff-YWlyZmxvdy9vcGVyYXRvcnMvaGl2ZV90b19kcnVpZC5weQ==) | `63.07% <100%> (ø)` | :arrow_up: | -- [Continue to review full report at Codecov](https://codecov.io/gh/apache/airflow/pull/5010?src=pr=continue). > **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta) > `Δ = absolute (impact)`, `ø = not affected`, `? = missing data` > Powered by [Codecov](https://codecov.io/gh/apache/airflow/pull/5010?src=pr=footer). Last update [157f7bf...aa0f577](https://codecov.io/gh/apache/airflow/pull/5010?src=pr=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments). This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [airflow] codecov-io edited a comment on issue #5010: [AIRFLOW-4192] Reorganize task context variables
codecov-io edited a comment on issue #5010: [AIRFLOW-4192] Reorganize task context variables URL: https://github.com/apache/airflow/pull/5010#issuecomment-478295602 # [Codecov](https://codecov.io/gh/apache/airflow/pull/5010?src=pr=h1) Report > Merging [#5010](https://codecov.io/gh/apache/airflow/pull/5010?src=pr=desc) into [master](https://codecov.io/gh/apache/airflow/commit/157f7bff837dda691012da8638dbc570bf5f1844?src=pr=desc) will **increase** coverage by `<.01%`. > The diff coverage is `100%`. [](https://codecov.io/gh/apache/airflow/pull/5010?src=pr=tree) ```diff @@Coverage Diff @@ ## master#5010 +/- ## == + Coverage 75.76% 75.76% +<.01% == Files 461 461 Lines 2995529948 -7 == - Hits2269422689 -5 + Misses 7261 7259 -2 ``` | [Impacted Files](https://codecov.io/gh/apache/airflow/pull/5010?src=pr=tree) | Coverage Δ | | |---|---|---| | [...flow/contrib/example\_dags/example\_qubole\_sensor.py](https://codecov.io/gh/apache/airflow/pull/5010/diff?src=pr=tree#diff-YWlyZmxvdy9jb250cmliL2V4YW1wbGVfZGFncy9leGFtcGxlX3F1Ym9sZV9zZW5zb3IucHk=) | `0% <ø> (ø)` | :arrow_up: | | [airflow/models/\_\_init\_\_.py](https://codecov.io/gh/apache/airflow/pull/5010/diff?src=pr=tree#diff-YWlyZmxvdy9tb2RlbHMvX19pbml0X18ucHk=) | `93.03% <100%> (+0.07%)` | :arrow_up: | | [airflow/operators/hive\_to\_druid.py](https://codecov.io/gh/apache/airflow/pull/5010/diff?src=pr=tree#diff-YWlyZmxvdy9vcGVyYXRvcnMvaGl2ZV90b19kcnVpZC5weQ==) | `63.07% <100%> (ø)` | :arrow_up: | -- [Continue to review full report at Codecov](https://codecov.io/gh/apache/airflow/pull/5010?src=pr=continue). > **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta) > `Δ = absolute (impact)`, `ø = not affected`, `? = missing data` > Powered by [Codecov](https://codecov.io/gh/apache/airflow/pull/5010?src=pr=footer). Last update [157f7bf...aa0f577](https://codecov.io/gh/apache/airflow/pull/5010?src=pr=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments). This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [airflow] codecov-io edited a comment on issue #5010: [AIRFLOW-4192] Reorganize task context variables
codecov-io edited a comment on issue #5010: [AIRFLOW-4192] Reorganize task context variables URL: https://github.com/apache/airflow/pull/5010#issuecomment-478295602 # [Codecov](https://codecov.io/gh/apache/airflow/pull/5010?src=pr=h1) Report > Merging [#5010](https://codecov.io/gh/apache/airflow/pull/5010?src=pr=desc) into [master](https://codecov.io/gh/apache/airflow/commit/157f7bff837dda691012da8638dbc570bf5f1844?src=pr=desc) will **increase** coverage by `<.01%`. > The diff coverage is `100%`. [](https://codecov.io/gh/apache/airflow/pull/5010?src=pr=tree) ```diff @@Coverage Diff @@ ## master#5010 +/- ## == + Coverage 75.76% 75.76% +<.01% == Files 461 461 Lines 2995529948 -7 == - Hits2269422689 -5 + Misses 7261 7259 -2 ``` | [Impacted Files](https://codecov.io/gh/apache/airflow/pull/5010?src=pr=tree) | Coverage Δ | | |---|---|---| | [...flow/contrib/example\_dags/example\_qubole\_sensor.py](https://codecov.io/gh/apache/airflow/pull/5010/diff?src=pr=tree#diff-YWlyZmxvdy9jb250cmliL2V4YW1wbGVfZGFncy9leGFtcGxlX3F1Ym9sZV9zZW5zb3IucHk=) | `0% <ø> (ø)` | :arrow_up: | | [airflow/models/\_\_init\_\_.py](https://codecov.io/gh/apache/airflow/pull/5010/diff?src=pr=tree#diff-YWlyZmxvdy9tb2RlbHMvX19pbml0X18ucHk=) | `93.03% <100%> (+0.07%)` | :arrow_up: | | [airflow/operators/hive\_to\_druid.py](https://codecov.io/gh/apache/airflow/pull/5010/diff?src=pr=tree#diff-YWlyZmxvdy9vcGVyYXRvcnMvaGl2ZV90b19kcnVpZC5weQ==) | `63.07% <100%> (ø)` | :arrow_up: | -- [Continue to review full report at Codecov](https://codecov.io/gh/apache/airflow/pull/5010?src=pr=continue). > **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta) > `Δ = absolute (impact)`, `ø = not affected`, `? = missing data` > Powered by [Codecov](https://codecov.io/gh/apache/airflow/pull/5010?src=pr=footer). Last update [157f7bf...aa0f577](https://codecov.io/gh/apache/airflow/pull/5010?src=pr=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments). This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [airflow] BasPH commented on issue #5010: [AIRFLOW-4192] Reorganize task context variables
BasPH commented on issue #5010: [AIRFLOW-4192] Reorganize task context variables URL: https://github.com/apache/airflow/pull/5010#issuecomment-478342697 @KevinYang21 I've updated UPDATING.md. Would be nice to have a PMC's opinion on this. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [airflow] BasPH commented on issue #5010: [AIRFLOW-4192] Reorganize task context variables
BasPH commented on issue #5010: [AIRFLOW-4192] Reorganize task context variables URL: https://github.com/apache/airflow/pull/5010#issuecomment-478337744 I'd say some variables are debatable, e.g. if the PMCs have a very strong opinion about keeping `yesterday_ds`; find by me. However the values of `latest_date`, `end_date` and `END_DATE` do not correspond to the name of the variables and are thus incorrect, so I'd definitely change or drop those. Good point about the UPDATING.md, will add info there. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Updated] (AIRFLOW-4195) Add Scylla hook
[
https://issues.apache.org/jira/browse/AIRFLOW-4195?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
jack updated AIRFLOW-4195:
--
Description:
[https://www.scylladb.com/]
Scylla rebuild the architecture of {color:#00}Apache Cassandra with c++
instead of Java. {color}
{color:#00}The performance is 1,000,000 IOPS better than Cassandra per
server.{color}
{color:#00}It would be nice to have hook for this DB in Airflow.{color}
was:
[https://www.scylladb.com/]
Scylla rebuild the architecture of {color:#00}Apache Cassandra with c++
instead of Java. {color}
{color:#00}The performance is 1,000,000 IOPS better than Cassandra.{color}
{color:#00}It would be nice to have hook for this DB in Airflow.{color}
> Add Scylla hook
> ---
>
> Key: AIRFLOW-4195
> URL: https://issues.apache.org/jira/browse/AIRFLOW-4195
> Project: Apache Airflow
> Issue Type: New Feature
>Reporter: jack
>Priority: Major
>
> [https://www.scylladb.com/]
> Scylla rebuild the architecture of {color:#00}Apache Cassandra with c++
> instead of Java. {color}
> {color:#00}The performance is 1,000,000 IOPS better than Cassandra per
> server.{color}
> {color:#00}It would be nice to have hook for this DB in Airflow.{color}
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Created] (AIRFLOW-4195) Add Scylla hook
jack created AIRFLOW-4195:
-
Summary: Add Scylla hook
Key: AIRFLOW-4195
URL: https://issues.apache.org/jira/browse/AIRFLOW-4195
Project: Apache Airflow
Issue Type: New Feature
Reporter: jack
[https://www.scylladb.com/]
Scylla rebuild the architecture of {color:#00}Apache Cassandra with c++
instead of Java. {color}
{color:#00}The performance is 1,000,000 IOPS better than Cassandra.{color}
{color:#00}It would be nice to have hook for this DB in Airflow.{color}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[GitHub] [airflow] codecov-io commented on issue #5014: [AIRFLOW-XXX] Update WeTransfer in "Who uses Apache Airflow"
codecov-io commented on issue #5014: [AIRFLOW-XXX] Update WeTransfer in "Who uses Apache Airflow" URL: https://github.com/apache/airflow/pull/5014#issuecomment-478326604 # [Codecov](https://codecov.io/gh/apache/airflow/pull/5014?src=pr=h1) Report > Merging [#5014](https://codecov.io/gh/apache/airflow/pull/5014?src=pr=desc) into [master](https://codecov.io/gh/apache/airflow/commit/157f7bff837dda691012da8638dbc570bf5f1844?src=pr=desc) will **not change** coverage. > The diff coverage is `n/a`. [](https://codecov.io/gh/apache/airflow/pull/5014?src=pr=tree) ```diff @@ Coverage Diff @@ ## master#5014 +/- ## === Coverage 75.76% 75.76% === Files 461 461 Lines 2995529955 === Hits2269422694 Misses 7261 7261 ``` -- [Continue to review full report at Codecov](https://codecov.io/gh/apache/airflow/pull/5014?src=pr=continue). > **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta) > `Δ = absolute (impact)`, `ø = not affected`, `? = missing data` > Powered by [Codecov](https://codecov.io/gh/apache/airflow/pull/5014?src=pr=footer). Last update [157f7bf...db91c92](https://codecov.io/gh/apache/airflow/pull/5014?src=pr=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments). This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [airflow] higee commented on issue #5014: [AIRFLOW-XXX] Update WeTransfer in "Who uses Apache Airflow"
higee commented on issue #5014: [AIRFLOW-XXX] Update WeTransfer in "Who uses Apache Airflow" URL: https://github.com/apache/airflow/pull/5014#issuecomment-478323775 @KevinYang21 Thanks for the prompt response. I modified the commit message. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [airflow] KevinYang21 commented on issue #5013: [AIRFLOW-XXX] Omit vendor packages and example dags from being covered by codecov
KevinYang21 commented on issue #5013: [AIRFLOW-XXX] Omit vendor packages and example dags from being covered by codecov URL: https://github.com/apache/airflow/pull/5013#issuecomment-478323540 Think it is nice to include exampels if we have 100% coverage there but don't think that is strictly needed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [airflow] codecov-io commented on issue #5012: [AIRFLOW-833] New `airflow default_config` command
codecov-io commented on issue #5012: [AIRFLOW-833] New `airflow default_config` command URL: https://github.com/apache/airflow/pull/5012#issuecomment-478322404 # [Codecov](https://codecov.io/gh/apache/airflow/pull/5012?src=pr=h1) Report > Merging [#5012](https://codecov.io/gh/apache/airflow/pull/5012?src=pr=desc) into [master](https://codecov.io/gh/apache/airflow/commit/157f7bff837dda691012da8638dbc570bf5f1844?src=pr=desc) will **increase** coverage by `<.01%`. > The diff coverage is `90.62%`. [](https://codecov.io/gh/apache/airflow/pull/5012?src=pr=tree) ```diff @@Coverage Diff @@ ## master#5012 +/- ## == + Coverage 75.76% 75.76% +<.01% == Files 461 461 Lines 2995529961 +6 == + Hits2269422701 +7 + Misses 7261 7260 -1 ``` | [Impacted Files](https://codecov.io/gh/apache/airflow/pull/5012?src=pr=tree) | Coverage Δ | | |---|---|---| | [airflow/www/app.py](https://codecov.io/gh/apache/airflow/pull/5012/diff?src=pr=tree#diff-YWlyZmxvdy93d3cvYXBwLnB5) | `97.45% <100%> (+0.04%)` | :arrow_up: | | [airflow/bin/cli.py](https://codecov.io/gh/apache/airflow/pull/5012/diff?src=pr=tree#diff-YWlyZmxvdy9iaW4vY2xpLnB5) | `67.17% <100%> (+0.2%)` | :arrow_up: | | [airflow/utils/db.py](https://codecov.io/gh/apache/airflow/pull/5012/diff?src=pr=tree#diff-YWlyZmxvdy91dGlscy9kYi5weQ==) | `90.74% <100%> (+0.44%)` | :arrow_up: | | [airflow/www/views.py](https://codecov.io/gh/apache/airflow/pull/5012/diff?src=pr=tree#diff-YWlyZmxvdy93d3cvdmlld3MucHk=) | `76.09% <60%> (-0.1%)` | :arrow_down: | | [airflow/utils/log/file\_processor\_handler.py](https://codecov.io/gh/apache/airflow/pull/5012/diff?src=pr=tree#diff-YWlyZmxvdy91dGlscy9sb2cvZmlsZV9wcm9jZXNzb3JfaGFuZGxlci5weQ==) | `83.33% <85.71%> (-2.78%)` | :arrow_down: | | [airflow/configuration.py](https://codecov.io/gh/apache/airflow/pull/5012/diff?src=pr=tree#diff-YWlyZmxvdy9jb25maWd1cmF0aW9uLnB5) | `93.1% <96%> (+0.92%)` | :arrow_up: | | [airflow/models/\_\_init\_\_.py](https://codecov.io/gh/apache/airflow/pull/5012/diff?src=pr=tree#diff-YWlyZmxvdy9tb2RlbHMvX19pbml0X18ucHk=) | `93.01% <0%> (+0.04%)` | :arrow_up: | -- [Continue to review full report at Codecov](https://codecov.io/gh/apache/airflow/pull/5012?src=pr=continue). > **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta) > `Δ = absolute (impact)`, `ø = not affected`, `? = missing data` > Powered by [Codecov](https://codecov.io/gh/apache/airflow/pull/5012?src=pr=footer). Last update [157f7bf...6b6a891](https://codecov.io/gh/apache/airflow/pull/5012?src=pr=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments). This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [airflow] KevinYang21 commented on issue #5014: [AIRFLOW-XXX] Update WeTransfer in "Who uses Apache Airflow"
KevinYang21 commented on issue #5014: [AIRFLOW-XXX] Update WeTransfer in "Who uses Apache Airflow" URL: https://github.com/apache/airflow/pull/5014#issuecomment-478321751 The commit message should be aligned with PR title, otherwise LGTM This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Commented] (AIRFLOW-4194) set dag_run state to failed when user terminate backfill
[ https://issues.apache.org/jira/browse/AIRFLOW-4194?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16806071#comment-16806071 ] Chao-Han Tsai commented on AIRFLOW-4194: [~TaoFeng] I remember that you contributed something related before but I can't find it in the code. > set dag_run state to failed when user terminate backfill > > > Key: AIRFLOW-4194 > URL: https://issues.apache.org/jira/browse/AIRFLOW-4194 > Project: Apache Airflow > Issue Type: New Feature >Reporter: Chao-Han Tsai >Assignee: Chao-Han Tsai >Priority: Major > > Currently when user terminate the backfill, we set the task_instance state to > failed. We should also set the dag_run state to failed. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (AIRFLOW-4194) set dag_run state to failed when user terminate backfill
Chao-Han Tsai created AIRFLOW-4194: -- Summary: set dag_run state to failed when user terminate backfill Key: AIRFLOW-4194 URL: https://issues.apache.org/jira/browse/AIRFLOW-4194 Project: Apache Airflow Issue Type: New Feature Reporter: Chao-Han Tsai Assignee: Chao-Han Tsai Currently when user terminate the backfill, we set the task_instance state to failed. We should also set the dag_run state to failed. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
