[GitHub] codecov-io edited a comment on issue #4213: [AIRFLOW-3371] BigQueryHook's Ability to Create View
codecov-io edited a comment on issue #4213: [AIRFLOW-3371] BigQueryHook's Ability to Create View URL: https://github.com/apache/incubator-airflow/pull/4213#issuecomment-440142454 # [Codecov](https://codecov.io/gh/apache/incubator-airflow/pull/4213?src=pr=h1) Report > Merging [#4213](https://codecov.io/gh/apache/incubator-airflow/pull/4213?src=pr=desc) into [master](https://codecov.io/gh/apache/incubator-airflow/commit/55853c3c23fe177bcf4ca19697f1e664325c69ea?src=pr=desc) will **decrease** coverage by `<.01%`. > The diff coverage is `n/a`. [](https://codecov.io/gh/apache/incubator-airflow/pull/4213?src=pr=tree) ```diff @@Coverage Diff@@ ## master #4213 +/- ## = - Coverage 77.71% 77.7% -0.01% = Files 199 199 Lines 16317 16317 = - Hits12680 12679 -1 - Misses 36373638 +1 ``` | [Impacted Files](https://codecov.io/gh/apache/incubator-airflow/pull/4213?src=pr=tree) | Coverage Δ | | |---|---|---| | [airflow/models.py](https://codecov.io/gh/apache/incubator-airflow/pull/4213/diff?src=pr=tree#diff-YWlyZmxvdy9tb2RlbHMucHk=) | `92.29% <0%> (-0.05%)` | :arrow_down: | -- [Continue to review full report at Codecov](https://codecov.io/gh/apache/incubator-airflow/pull/4213?src=pr=continue). > **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta) > `Δ = absolute (impact)`, `ø = not affected`, `? = missing data` > Powered by [Codecov](https://codecov.io/gh/apache/incubator-airflow/pull/4213?src=pr=footer). Last update [55853c3...c7f6d76](https://codecov.io/gh/apache/incubator-airflow/pull/4213?src=pr=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments). This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] codecov-io commented on issue #4213: [AIRFLOW-3371] BigQueryHook's Ability to Create View
codecov-io commented on issue #4213: [AIRFLOW-3371] BigQueryHook's Ability to Create View URL: https://github.com/apache/incubator-airflow/pull/4213#issuecomment-440142454 # [Codecov](https://codecov.io/gh/apache/incubator-airflow/pull/4213?src=pr=h1) Report > Merging [#4213](https://codecov.io/gh/apache/incubator-airflow/pull/4213?src=pr=desc) into [master](https://codecov.io/gh/apache/incubator-airflow/commit/55853c3c23fe177bcf4ca19697f1e664325c69ea?src=pr=desc) will **decrease** coverage by `<.01%`. > The diff coverage is `n/a`. [](https://codecov.io/gh/apache/incubator-airflow/pull/4213?src=pr=tree) ```diff @@Coverage Diff@@ ## master #4213 +/- ## = - Coverage 77.71% 77.7% -0.01% = Files 199 199 Lines 16317 16317 = - Hits12680 12679 -1 - Misses 36373638 +1 ``` | [Impacted Files](https://codecov.io/gh/apache/incubator-airflow/pull/4213?src=pr=tree) | Coverage Δ | | |---|---|---| | [airflow/models.py](https://codecov.io/gh/apache/incubator-airflow/pull/4213/diff?src=pr=tree#diff-YWlyZmxvdy9tb2RlbHMucHk=) | `92.29% <0%> (-0.05%)` | :arrow_down: | -- [Continue to review full report at Codecov](https://codecov.io/gh/apache/incubator-airflow/pull/4213?src=pr=continue). > **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta) > `Δ = absolute (impact)`, `ø = not affected`, `? = missing data` > Powered by [Codecov](https://codecov.io/gh/apache/incubator-airflow/pull/4213?src=pr=footer). Last update [55853c3...c7f6d76](https://codecov.io/gh/apache/incubator-airflow/pull/4213?src=pr=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments). This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] codecov-io edited a comment on issue #4213: [AIRFLOW-3371] BigQueryHook's Ability to Create View
codecov-io edited a comment on issue #4213: [AIRFLOW-3371] BigQueryHook's Ability to Create View URL: https://github.com/apache/incubator-airflow/pull/4213#issuecomment-440142454 # [Codecov](https://codecov.io/gh/apache/incubator-airflow/pull/4213?src=pr=h1) Report > Merging [#4213](https://codecov.io/gh/apache/incubator-airflow/pull/4213?src=pr=desc) into [master](https://codecov.io/gh/apache/incubator-airflow/commit/55853c3c23fe177bcf4ca19697f1e664325c69ea?src=pr=desc) will **decrease** coverage by `<.01%`. > The diff coverage is `n/a`. [](https://codecov.io/gh/apache/incubator-airflow/pull/4213?src=pr=tree) ```diff @@Coverage Diff@@ ## master #4213 +/- ## = - Coverage 77.71% 77.7% -0.01% = Files 199 199 Lines 16317 16317 = - Hits12680 12679 -1 - Misses 36373638 +1 ``` | [Impacted Files](https://codecov.io/gh/apache/incubator-airflow/pull/4213?src=pr=tree) | Coverage Δ | | |---|---|---| | [airflow/models.py](https://codecov.io/gh/apache/incubator-airflow/pull/4213/diff?src=pr=tree#diff-YWlyZmxvdy9tb2RlbHMucHk=) | `92.29% <0%> (-0.05%)` | :arrow_down: | -- [Continue to review full report at Codecov](https://codecov.io/gh/apache/incubator-airflow/pull/4213?src=pr=continue). > **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta) > `Δ = absolute (impact)`, `ø = not affected`, `? = missing data` > Powered by [Codecov](https://codecov.io/gh/apache/incubator-airflow/pull/4213?src=pr=footer). Last update [55853c3...c7f6d76](https://codecov.io/gh/apache/incubator-airflow/pull/4213?src=pr=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments). This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Commented] (AIRFLOW-3371) BigQueryHook's Ability to Create View
[
https://issues.apache.org/jira/browse/AIRFLOW-3371?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16692671#comment-16692671
]
ASF GitHub Bot commented on AIRFLOW-3371:
-
ryanyuan opened a new pull request #4213: [AIRFLOW-3371] BigQueryHook's Ability
to Create View
URL: https://github.com/apache/incubator-airflow/pull/4213
1) Edit create_empty_table to take in optional 'view' dict
2) Add unit tests for creating view
Make sure you have checked _all_ steps below.
### Jira
- [ ] My PR addresses the following
[Airflow-3371](https://issues.apache.org/jira/browse/AIRFLOW-3371) issues and
references them in the PR title.
### Description
- [ ] Here are some details about my PR, including screenshots of any UI
changes:
To enable BigQueryBaseCursor.create_empty_table to create view in BigQuery
Ref: https://cloud.google.com/bigquery/docs/reference/rest/v2/tables#view
### Tests
- [ ] My PR adds the following unit tests __OR__ does not need testing for
this extremely good reason:
tests.contrib.hooks.test_bigquery_hook:TestBigQueryBaseCursor.test_create_view_fails_on_exception
tests.contrib.hooks.test_bigquery_hook:TestBigQueryBaseCursor.test_create_view
### Commits
- [ ] My commits all reference Jira issues in their subject lines, and I
have squashed multiple commits if they address the same issue. In addition, my
commits follow the guidelines from "[How to write a good git commit
message](http://chris.beams.io/posts/git-commit/)":
1. Subject is separated from body by a blank line
1. Subject is limited to 50 characters (not including Jira issue reference)
1. Subject does not end with a period
1. Subject uses the imperative mood ("add", not "adding")
1. Body wraps at 72 characters
1. Body explains "what" and "why", not "how"
### Documentation
- [ ] In case of new functionality, my PR adds documentation that describes
how to use it.
- When adding new operators/hooks/sensors, the autoclass documentation
generation needs to be added.
### Code Quality
- [ ] Passes `flake8`
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
> BigQueryHook's Ability to Create View
> -
>
> Key: AIRFLOW-3371
> URL: https://issues.apache.org/jira/browse/AIRFLOW-3371
> Project: Apache Airflow
> Issue Type: New Feature
>Reporter: Ryan Yuan
>Assignee: Ryan Yuan
>Priority: Major
>
> Modify *BigQueryBaseCursor.create_empty_table()* to take in an optional
> 'view' parameter to create view in BigQuery.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Work started] (AIRFLOW-3371) BigQueryHook's Ability to Create View
[ https://issues.apache.org/jira/browse/AIRFLOW-3371?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Work on AIRFLOW-3371 started by Ryan Yuan. -- > BigQueryHook's Ability to Create View > - > > Key: AIRFLOW-3371 > URL: https://issues.apache.org/jira/browse/AIRFLOW-3371 > Project: Apache Airflow > Issue Type: New Feature >Reporter: Ryan Yuan >Assignee: Ryan Yuan >Priority: Major > > Modify *BigQueryBaseCursor.create_empty_table()* to take in an optional > 'view' parameter to create view in BigQuery. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (AIRFLOW-3371) BigQueryHook's Ability to Create View
Ryan Yuan created AIRFLOW-3371: -- Summary: BigQueryHook's Ability to Create View Key: AIRFLOW-3371 URL: https://issues.apache.org/jira/browse/AIRFLOW-3371 Project: Apache Airflow Issue Type: New Feature Reporter: Ryan Yuan Assignee: Ryan Yuan Modify *BigQueryBaseCursor.create_empty_table()* to take in an optional 'view' parameter to create view in BigQuery. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[GitHub] codecov-io commented on issue #4212: [AIRFLOW-XXX] Fix copy mistake
codecov-io commented on issue #4212: [AIRFLOW-XXX] Fix copy mistake URL: https://github.com/apache/incubator-airflow/pull/4212#issuecomment-440123061 # [Codecov](https://codecov.io/gh/apache/incubator-airflow/pull/4212?src=pr=h1) Report > Merging [#4212](https://codecov.io/gh/apache/incubator-airflow/pull/4212?src=pr=desc) into [master](https://codecov.io/gh/apache/incubator-airflow/commit/55853c3c23fe177bcf4ca19697f1e664325c69ea?src=pr=desc) will **decrease** coverage by `<.01%`. > The diff coverage is `n/a`. [](https://codecov.io/gh/apache/incubator-airflow/pull/4212?src=pr=tree) ```diff @@Coverage Diff@@ ## master #4212 +/- ## = - Coverage 77.71% 77.7% -0.01% = Files 199 199 Lines 16317 16317 = - Hits12680 12679 -1 - Misses 36373638 +1 ``` | [Impacted Files](https://codecov.io/gh/apache/incubator-airflow/pull/4212?src=pr=tree) | Coverage Δ | | |---|---|---| | [airflow/models.py](https://codecov.io/gh/apache/incubator-airflow/pull/4212/diff?src=pr=tree#diff-YWlyZmxvdy9tb2RlbHMucHk=) | `92.29% <0%> (-0.05%)` | :arrow_down: | -- [Continue to review full report at Codecov](https://codecov.io/gh/apache/incubator-airflow/pull/4212?src=pr=continue). > **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta) > `Δ = absolute (impact)`, `ø = not affected`, `? = missing data` > Powered by [Codecov](https://codecov.io/gh/apache/incubator-airflow/pull/4212?src=pr=footer). Last update [55853c3...5446db8](https://codecov.io/gh/apache/incubator-airflow/pull/4212?src=pr=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments). This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] nykh opened a new pull request #4212: AIRFLOW-XXX Fix copy mistake
nykh opened a new pull request #4212: AIRFLOW-XXX Fix copy mistake
URL: https://github.com/apache/incubator-airflow/pull/4212
### Jira
- [x] My PR addresses the following [Airflow
Jira](https://issues.apache.org/jira/browse/AIRFLOW/) issues and references
them in the PR title. For example, "\[AIRFLOW-XXX\] My Airflow PR"
- https://issues.apache.org/jira/browse/AIRFLOW-XXX
- In case you are fixing a typo in the documentation you can prepend your
commit with \[AIRFLOW-XXX\], code changes always need a Jira issue.
### Description
Fix an obvious copy mistake
- [x] Here are some details about my PR, including screenshots of any UI
changes:
### Tests
- [x] My PR adds does not need testing for this extremely good reason:
### Commits
- [x] My commits all reference Jira issues in their subject lines, and I
have squashed multiple commits if they address the same issue. In addition, my
commits follow the guidelines from "[How to write a good git commit
message](http://chris.beams.io/posts/git-commit/)":
1. Subject is separated from body by a blank line
1. Subject is limited to 50 characters (not including Jira issue reference)
1. Subject does not end with a period
1. Subject uses the imperative mood ("add", not "adding")
1. Body wraps at 72 characters
1. Body explains "what" and "why", not "how"
### Documentation
- [ ] In case of new functionality, my PR adds documentation that describes
how to use it.
- When adding new operators/hooks/sensors, the autoclass documentation
generation needs to be added.
### Code Quality
- [x] Passes `flake8`
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[jira] [Commented] (AIRFLOW-3118) DAGs not successful on new installation
[
https://issues.apache.org/jira/browse/AIRFLOW-3118?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16692419#comment-16692419
]
Huy Nguyen commented on AIRFLOW-3118:
-
I also have [a
fix|https://github.com/huyanhvn/incubator-airflow/commit/4ab9ef7eaa82f5c5efe45e00c315492aee34b880]
but haven't created a PR yet. Confirmed this Jira duplicates AIRFLOW-1561.
Will cancel.
> DAGs not successful on new installation
> ---
>
> Key: AIRFLOW-3118
> URL: https://issues.apache.org/jira/browse/AIRFLOW-3118
> Project: Apache Airflow
> Issue Type: Bug
> Components: DAG
>Affects Versions: 1.10.0
> Environment: Ubuntu 18.04
> Python 3.6
>Reporter: Brylie Christopher Oxley
>Assignee: Huy Nguyen
>Priority: Blocker
> Fix For: 1.10.2
>
> Attachments: Screenshot_20180926_161837.png,
> image-2018-09-26-12-39-03-094.png
>
>
> When trying out Airflow, on localhost, none of the DAG runs are getting to
> the 'success' state. They are getting stuck in 'running', or I manually label
> them as failed:
> !image-2018-09-26-12-39-03-094.png!
> h2. Steps to reproduce
> # create new conda environment
> ** conda create -n airflow
> ** source activate airflow
> # install airflow
> ** pip install apache-airflow
> # initialize Airflow db
> ** airflow initdb
> # disable default paused setting in airflow.cfg
> ** dags_are_paused_at_creation = False
> # {color:#6a8759}run airflow and airflow scheduler (in separate
> terminal){color}
> ** {color:#6a8759}airflow scheduler{color}
> ** {color:#6a8759}airflow webserver{color}
> # {color:#6a8759}unpause example_bash_operator{color}
> ** {color:#6a8759}airflow unpause example_bash_operator{color}
> # {color:#6a8759}log in to Airflow UI{color}
> # {color:#6a8759}turn on example_bash_operator{color}
> # {color:#6a8759}click "Trigger DAG" in `example_bash_operator` row{color}
> h2. {color:#6a8759}Observed result{color}
> {color:#6a8759}The `example_bash_operator` never leaves the "running"
> state.{color}
> h2. {color:#6a8759}Expected result{color}
> {color:#6a8759}The `example_bash_operator` would quickly enter the "success"
> state{color}
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[GitHub] morgendave commented on issue #4101: [AIRFLOW-3272] Add base grpc hook
morgendave commented on issue #4101: [AIRFLOW-3272] Add base grpc hook URL: https://github.com/apache/incubator-airflow/pull/4101#issuecomment-440013513 > Would it be possible to add some tests to this PR as well? In progress. I would lean to merge this first since the operator and some other functionality would rely on it. Local integration has been done This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] villasv commented on issue #4211: AIRFLOW-3365 and AIRFLOW-3366
villasv commented on issue #4211: AIRFLOW-3365 and AIRFLOW-3366 URL: https://github.com/apache/incubator-airflow/pull/4211#issuecomment-440011679 Huh? The test is unstable? I expected that comparing OrderedDicts would be stable. This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] SamWildmo commented on issue #2206: [AIRFLOW-922] Update PrestoHook to enable synchronous execution
SamWildmo commented on issue #2206: [AIRFLOW-922] Update PrestoHook to enable synchronous execution URL: https://github.com/apache/incubator-airflow/pull/2206#issuecomment-440008306 @patrickmckenna are you still working on this PR? This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] SamWildmo commented on issue #2319: [AIRFLOW-1236] SlackPostOperator using Slack Incoming WebHook
SamWildmo commented on issue #2319: [AIRFLOW-1236] SlackPostOperator using Slack Incoming WebHook URL: https://github.com/apache/incubator-airflow/pull/2319#issuecomment-440007237 @forsberg are you still working on this PR? This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Commented] (AIRFLOW-3369) Un-pausing a DAG with catchup =False creates an extra DAG run (1.10)
[
https://issues.apache.org/jira/browse/AIRFLOW-3369?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16692131#comment-16692131
]
Andrew Harmon commented on AIRFLOW-3369:
a workaround until this gets fixed. If you set the start date = to the most
recent interval it will only schedule 1 DAG run. for example, if deploying your
dag on 11/19 and it should run daily. Set your start_date to 11/18. It will not
schedule the 11/17 DAG run in this scenario.
> Un-pausing a DAG with catchup =False creates an extra DAG run (1.10)
>
>
> Key: AIRFLOW-3369
> URL: https://issues.apache.org/jira/browse/AIRFLOW-3369
> Project: Apache Airflow
> Issue Type: Bug
>Affects Versions: 1.10.0
>Reporter: Andrew Harmon
>Priority: Major
> Attachments: image.png
>
>
> If you create a DAG with catchup=False, when it is un-paused, it creates 2
> dag runs. One for the most recent scheduled interval (expected) and one for
> the interval before that (unexpected).
> *Sample DAG*
> {code:java}
> from airflow import DAG
> from datetime import datetime
> from airflow.operators.dummy_operator import DummyOperator
> dag = DAG(
> dag_id='DummyTest',
> start_date=datetime(2018,1,1),
> catchup=False
> )
> do = DummyOperator(
> task_id='dummy_task',
> dag=dag
> )
> {code}
> *Result:*
> 2 DAG runs are created. 2018-11-18 and 108-11-17
> *Expected Result:*
> Only 1 DAG run should have been created (2018-11-18)
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Created] (AIRFLOW-3370) Enhance current ES handler with stdout capability and more output options
Robert Hwang created AIRFLOW-3370: - Summary: Enhance current ES handler with stdout capability and more output options Key: AIRFLOW-3370 URL: https://issues.apache.org/jira/browse/AIRFLOW-3370 Project: Apache Airflow Issue Type: Improvement Components: celery, logging, worker Affects Versions: 1.10.0 Reporter: Robert Hwang Currently, the ES handler in 1.10 can only search from ES, and it's not. Two possible enhancements are to allow the handler a "write to standard out" functionality, and make log messages more remote-storage friendly, with an optional JSON-formatted message, and the regular pretty-print log format as the default -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (AIRFLOW-3369) Un-pausing a DAG with catchup =False creates an extra DAG run (1.10)
[
https://issues.apache.org/jira/browse/AIRFLOW-3369?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Andrew Harmon updated AIRFLOW-3369:
---
Description:
If you create a DAG with catchup=False, when it is un-paused, it creates 2 dag
runs. One for the most recent scheduled interval (expected) and one for the
interval before that (unexpected).
*Sample DAG*
{code:java}
from airflow import DAG
from datetime import datetime
from airflow.operators.dummy_operator import DummyOperator
dag = DAG(
dag_id='DummyTest',
start_date=datetime(2018,1,1),
catchup=False
)
do = DummyOperator(
task_id='dummy_task',
dag=dag
)
{code}
*Result:*
2 DAG runs are created. 2018-11-18 and 108-11-17
*Expected Result:*
Only 1 DAG run should have been created (2018-11-18)
was:
If you create a DAG with catchup=False, when it is un-paused, it creates 2 dag
runs. One for the most recent scheduled interval (expected) and one for the
interval before that (unexpected).
*Sample DAG*
{code:java}
from airflow import DAG
from datetime import datetime
from airflow.operators.dummy_operator import DummyOperator
dag = DAG(
dag_id='DummyTest',
start_date=datetime(2018,1,1),
catchup=False
)
do = DummyOperator(
task_id='dummy_task',
dag=dag
)
{code}
*Result:*
.
*Expected Result:*
Only 1 DAG run should have been created (2018-11-18)
> Un-pausing a DAG with catchup =False creates an extra DAG run (1.10)
>
>
> Key: AIRFLOW-3369
> URL: https://issues.apache.org/jira/browse/AIRFLOW-3369
> Project: Apache Airflow
> Issue Type: Bug
>Affects Versions: 1.10.0
>Reporter: Andrew Harmon
>Priority: Major
> Attachments: image.png
>
>
> If you create a DAG with catchup=False, when it is un-paused, it creates 2
> dag runs. One for the most recent scheduled interval (expected) and one for
> the interval before that (unexpected).
> *Sample DAG*
> {code:java}
> from airflow import DAG
> from datetime import datetime
> from airflow.operators.dummy_operator import DummyOperator
> dag = DAG(
> dag_id='DummyTest',
> start_date=datetime(2018,1,1),
> catchup=False
> )
> do = DummyOperator(
> task_id='dummy_task',
> dag=dag
> )
> {code}
> *Result:*
> 2 DAG runs are created. 2018-11-18 and 108-11-17
> *Expected Result:*
> Only 1 DAG run should have been created (2018-11-18)
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (AIRFLOW-3369) Un-pausing a DAG with catchup =False creates an extra DAG run (1.10)
[
https://issues.apache.org/jira/browse/AIRFLOW-3369?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Andrew Harmon updated AIRFLOW-3369:
---
Attachment: image.png
Description:
If you create a DAG with catchup=False, when it is un-paused, it creates 2 dag
runs. One for the most recent scheduled interval (expected) and one for the
interval before that (unexpected).
*Sample DAG*
{code:java}
from airflow import DAG
from datetime import datetime
from airflow.operators.dummy_operator import DummyOperator
dag = DAG(
dag_id='DummyTest',
start_date=datetime(2018,1,1),
catchup=False
)
do = DummyOperator(
task_id='dummy_task',
dag=dag
)
{code}
*Result:*
.
*Expected Result:*
Only 1 DAG run should have been created (2018-11-18)
was:
If you create a DAG with catchup=False, when it is un-paused, it creates 2 dag
runs. One for the most recent scheduled interval (expected) and one for the
interval before that (unexpected).
*Sample DAG*
{code:java}
from airflow import DAG
from datetime import datetime
from airflow.operators.dummy_operator import DummyOperator
dag = DAG(
dag_id='DummyTest',
start_date=datetime(2018,1,1),
catchup=False
)
do = DummyOperator(
task_id='dummy_task',
dag=dag
)
{code}
*Result:*
!image-2018-11-19-13-41-49-961.png!
*Expected Result:*
Only 1 DAG run should have been created (2018-11-18)
> Un-pausing a DAG with catchup =False creates an extra DAG run (1.10)
>
>
> Key: AIRFLOW-3369
> URL: https://issues.apache.org/jira/browse/AIRFLOW-3369
> Project: Apache Airflow
> Issue Type: Bug
>Affects Versions: 1.10.0
>Reporter: Andrew Harmon
>Priority: Major
> Attachments: image.png
>
>
> If you create a DAG with catchup=False, when it is un-paused, it creates 2
> dag runs. One for the most recent scheduled interval (expected) and one for
> the interval before that (unexpected).
> *Sample DAG*
> {code:java}
> from airflow import DAG
> from datetime import datetime
> from airflow.operators.dummy_operator import DummyOperator
> dag = DAG(
> dag_id='DummyTest',
> start_date=datetime(2018,1,1),
> catchup=False
> )
> do = DummyOperator(
> task_id='dummy_task',
> dag=dag
> )
> {code}
> *Result:*
> .
> *Expected Result:*
> Only 1 DAG run should have been created (2018-11-18)
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Created] (AIRFLOW-3369) Un-pausing a DAG with catchup =False creates an extra DAG run (1.10)
Andrew Harmon created AIRFLOW-3369:
--
Summary: Un-pausing a DAG with catchup =False creates an extra DAG
run (1.10)
Key: AIRFLOW-3369
URL: https://issues.apache.org/jira/browse/AIRFLOW-3369
Project: Apache Airflow
Issue Type: Bug
Affects Versions: 1.10.0
Reporter: Andrew Harmon
If you create a DAG with catchup=False, when it is un-paused, it creates 2 dag
runs. One for the most recent scheduled interval (expected) and one for the
interval before that (unexpected).
*Sample DAG*
{code:java}
from airflow import DAG
from datetime import datetime
from airflow.operators.dummy_operator import DummyOperator
dag = DAG(
dag_id='DummyTest',
start_date=datetime(2018,1,1),
catchup=False
)
do = DummyOperator(
task_id='dummy_task',
dag=dag
)
{code}
*Result:*
!image-2018-11-19-13-41-49-961.png!
*Expected Result:*
Only 1 DAG run should have been created (2018-11-18)
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[GitHub] feng-tao commented on issue #4210: [AIRFLOW-XXX] Don't publish md5 sigs as part of release
feng-tao commented on issue #4210: [AIRFLOW-XXX] Don't publish md5 sigs as part of release URL: https://github.com/apache/incubator-airflow/pull/4210#issuecomment-439993607 lgtm This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] feng-tao closed pull request #4210: [AIRFLOW-XXX] Don't publish md5 sigs as part of release
feng-tao closed pull request #4210: [AIRFLOW-XXX] Don't publish md5 sigs as
part of release
URL: https://github.com/apache/incubator-airflow/pull/4210
This is a PR merged from a forked repository.
As GitHub hides the original diff on merge, it is displayed below for
the sake of provenance:
As this is a foreign pull request (from a fork), the diff is supplied
below (as it won't show otherwise due to GitHub magic):
diff --git a/dev/sign.sh b/dev/sign.sh
index 66427a7e87..33054a175e 100755
--- a/dev/sign.sh
+++ b/dev/sign.sh
@@ -26,4 +26,3 @@ NAME=${1}
gpg --armor --output ${NAME}.asc --detach-sig ${NAME}
gpg --print-md SHA512 ${NAME} > ${NAME}.sha512
-gpg --print-md MD5 ${NAME} > ${NAME}.md5
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] villasv edited a comment on issue #4211: AIRFLOW-3365 and AIRFLOW-3366
villasv edited a comment on issue #4211: AIRFLOW-3365 and AIRFLOW-3366 URL: https://github.com/apache/incubator-airflow/pull/4211#issuecomment-439974046 I'll look into modifying the `test_getsection` and `test_broker_transport_options` to cover the new functionality. Meanwhile, let me just link back this PR where I need it :-) https://github.com/villasv/aws-airflow-stack/issues/51 This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] villasv commented on issue #4211: AIRFLOW-3365 and AIRFLOW-3366
villasv commented on issue #4211: AIRFLOW-3365 and AIRFLOW-3366 URL: https://github.com/apache/incubator-airflow/pull/4211#issuecomment-439974046 I'll look into modifying the `test_getsection` and `test_broker_transport_options` to cover the new functionality, This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] codecov-io edited a comment on issue #4090: [AIRFLOW-3250] Fix for Redis Hook for not authorised connection calls
codecov-io edited a comment on issue #4090: [AIRFLOW-3250] Fix for Redis Hook for not authorised connection calls URL: https://github.com/apache/incubator-airflow/pull/4090#issuecomment-437704538 # [Codecov](https://codecov.io/gh/apache/incubator-airflow/pull/4090?src=pr=h1) Report > Merging [#4090](https://codecov.io/gh/apache/incubator-airflow/pull/4090?src=pr=desc) into [master](https://codecov.io/gh/apache/incubator-airflow/commit/94d19707d7f3563bb48868c2d6442c3da923da20?src=pr=desc) will **decrease** coverage by `0.04%`. > The diff coverage is `n/a`. [](https://codecov.io/gh/apache/incubator-airflow/pull/4090?src=pr=tree) ```diff @@Coverage Diff @@ ## master#4090 +/- ## == - Coverage77.7% 77.66% -0.05% == Files 199 199 Lines 1631716273 -44 == - Hits1267912638 -41 + Misses 3638 3635 -3 ``` | [Impacted Files](https://codecov.io/gh/apache/incubator-airflow/pull/4090?src=pr=tree) | Coverage Δ | | |---|---|---| | [airflow/www\_rbac/security.py](https://codecov.io/gh/apache/incubator-airflow/pull/4090/diff?src=pr=tree#diff-YWlyZmxvdy93d3dfcmJhYy9zZWN1cml0eS5weQ==) | `92.61% <0%> (-0.15%)` | :arrow_down: | | [airflow/models.py](https://codecov.io/gh/apache/incubator-airflow/pull/4090/diff?src=pr=tree#diff-YWlyZmxvdy9tb2RlbHMucHk=) | `92.19% <0%> (-0.11%)` | :arrow_down: | | [airflow/www\_rbac/views.py](https://codecov.io/gh/apache/incubator-airflow/pull/4090/diff?src=pr=tree#diff-YWlyZmxvdy93d3dfcmJhYy92aWV3cy5weQ==) | `72.32% <0%> (-0.07%)` | :arrow_down: | | [airflow/www\_rbac/app.py](https://codecov.io/gh/apache/incubator-airflow/pull/4090/diff?src=pr=tree#diff-YWlyZmxvdy93d3dfcmJhYy9hcHAucHk=) | `97.05% <0%> (-0.03%)` | :arrow_down: | | [airflow/api/common/experimental/trigger\_dag.py](https://codecov.io/gh/apache/incubator-airflow/pull/4090/diff?src=pr=tree#diff-YWlyZmxvdy9hcGkvY29tbW9uL2V4cGVyaW1lbnRhbC90cmlnZ2VyX2RhZy5weQ==) | `100% <0%> (ø)` | :arrow_up: | | [airflow/operators/bash\_operator.py](https://codecov.io/gh/apache/incubator-airflow/pull/4090/diff?src=pr=tree#diff-YWlyZmxvdy9vcGVyYXRvcnMvYmFzaF9vcGVyYXRvci5weQ==) | `91.37% <0%> (ø)` | :arrow_up: | | [airflow/operators/docker\_operator.py](https://codecov.io/gh/apache/incubator-airflow/pull/4090/diff?src=pr=tree#diff-YWlyZmxvdy9vcGVyYXRvcnMvZG9ja2VyX29wZXJhdG9yLnB5) | `97.67% <0%> (ø)` | :arrow_up: | | [airflow/bin/cli.py](https://codecov.io/gh/apache/incubator-airflow/pull/4090/diff?src=pr=tree#diff-YWlyZmxvdy9iaW4vY2xpLnB5) | `64.82% <0%> (+0.23%)` | :arrow_up: | -- [Continue to review full report at Codecov](https://codecov.io/gh/apache/incubator-airflow/pull/4090?src=pr=continue). > **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta) > `Δ = absolute (impact)`, `ø = not affected`, `? = missing data` > Powered by [Codecov](https://codecov.io/gh/apache/incubator-airflow/pull/4090?src=pr=footer). Last update [94d1970...99768a4](https://codecov.io/gh/apache/incubator-airflow/pull/4090?src=pr=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments). This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] codecov-io edited a comment on issue #4090: [AIRFLOW-3250] Fix for Redis Hook for not authorised connection calls
codecov-io edited a comment on issue #4090: [AIRFLOW-3250] Fix for Redis Hook for not authorised connection calls URL: https://github.com/apache/incubator-airflow/pull/4090#issuecomment-437704538 # [Codecov](https://codecov.io/gh/apache/incubator-airflow/pull/4090?src=pr=h1) Report > Merging [#4090](https://codecov.io/gh/apache/incubator-airflow/pull/4090?src=pr=desc) into [master](https://codecov.io/gh/apache/incubator-airflow/commit/94d19707d7f3563bb48868c2d6442c3da923da20?src=pr=desc) will **decrease** coverage by `0.04%`. > The diff coverage is `n/a`. [](https://codecov.io/gh/apache/incubator-airflow/pull/4090?src=pr=tree) ```diff @@Coverage Diff @@ ## master#4090 +/- ## == - Coverage77.7% 77.66% -0.05% == Files 199 199 Lines 1631716273 -44 == - Hits1267912638 -41 + Misses 3638 3635 -3 ``` | [Impacted Files](https://codecov.io/gh/apache/incubator-airflow/pull/4090?src=pr=tree) | Coverage Δ | | |---|---|---| | [airflow/www\_rbac/security.py](https://codecov.io/gh/apache/incubator-airflow/pull/4090/diff?src=pr=tree#diff-YWlyZmxvdy93d3dfcmJhYy9zZWN1cml0eS5weQ==) | `92.61% <0%> (-0.15%)` | :arrow_down: | | [airflow/models.py](https://codecov.io/gh/apache/incubator-airflow/pull/4090/diff?src=pr=tree#diff-YWlyZmxvdy9tb2RlbHMucHk=) | `92.19% <0%> (-0.11%)` | :arrow_down: | | [airflow/www\_rbac/views.py](https://codecov.io/gh/apache/incubator-airflow/pull/4090/diff?src=pr=tree#diff-YWlyZmxvdy93d3dfcmJhYy92aWV3cy5weQ==) | `72.32% <0%> (-0.07%)` | :arrow_down: | | [airflow/www\_rbac/app.py](https://codecov.io/gh/apache/incubator-airflow/pull/4090/diff?src=pr=tree#diff-YWlyZmxvdy93d3dfcmJhYy9hcHAucHk=) | `97.05% <0%> (-0.03%)` | :arrow_down: | | [airflow/api/common/experimental/trigger\_dag.py](https://codecov.io/gh/apache/incubator-airflow/pull/4090/diff?src=pr=tree#diff-YWlyZmxvdy9hcGkvY29tbW9uL2V4cGVyaW1lbnRhbC90cmlnZ2VyX2RhZy5weQ==) | `100% <0%> (ø)` | :arrow_up: | | [airflow/operators/bash\_operator.py](https://codecov.io/gh/apache/incubator-airflow/pull/4090/diff?src=pr=tree#diff-YWlyZmxvdy9vcGVyYXRvcnMvYmFzaF9vcGVyYXRvci5weQ==) | `91.37% <0%> (ø)` | :arrow_up: | | [airflow/operators/docker\_operator.py](https://codecov.io/gh/apache/incubator-airflow/pull/4090/diff?src=pr=tree#diff-YWlyZmxvdy9vcGVyYXRvcnMvZG9ja2VyX29wZXJhdG9yLnB5) | `97.67% <0%> (ø)` | :arrow_up: | | [airflow/bin/cli.py](https://codecov.io/gh/apache/incubator-airflow/pull/4090/diff?src=pr=tree#diff-YWlyZmxvdy9iaW4vY2xpLnB5) | `64.82% <0%> (+0.23%)` | :arrow_up: | -- [Continue to review full report at Codecov](https://codecov.io/gh/apache/incubator-airflow/pull/4090?src=pr=continue). > **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta) > `Δ = absolute (impact)`, `ø = not affected`, `? = missing data` > Powered by [Codecov](https://codecov.io/gh/apache/incubator-airflow/pull/4090?src=pr=footer). Last update [94d1970...99768a4](https://codecov.io/gh/apache/incubator-airflow/pull/4090?src=pr=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments). This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] czarinacatambing commented on issue #4190: [AIRFLOW-3368] Remove {table} parameter on COPY command
czarinacatambing commented on issue #4190: [AIRFLOW-3368] Remove {table}
parameter on COPY command
URL:
https://github.com/apache/incubator-airflow/pull/4190#issuecomment-439936923
Hello, I just filed a jira ticket. Sorry, I overlooked that step.
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] codecov-io edited a comment on issue #4207: Run celery integration test with redis broker.
codecov-io edited a comment on issue #4207: Run celery integration test with redis broker. URL: https://github.com/apache/incubator-airflow/pull/4207#issuecomment-439668923 # [Codecov](https://codecov.io/gh/apache/incubator-airflow/pull/4207?src=pr=h1) Report > Merging [#4207](https://codecov.io/gh/apache/incubator-airflow/pull/4207?src=pr=desc) into [master](https://codecov.io/gh/apache/incubator-airflow/commit/94d19707d7f3563bb48868c2d6442c3da923da20?src=pr=desc) will **increase** coverage by `<.01%`. > The diff coverage is `n/a`. [](https://codecov.io/gh/apache/incubator-airflow/pull/4207?src=pr=tree) ```diff @@Coverage Diff @@ ## master#4207 +/- ## == + Coverage77.7% 77.71% +<.01% == Files 199 199 Lines 1631716317 == + Hits1267912680 +1 + Misses 3638 3637 -1 ``` | [Impacted Files](https://codecov.io/gh/apache/incubator-airflow/pull/4207?src=pr=tree) | Coverage Δ | | |---|---|---| | [airflow/models.py](https://codecov.io/gh/apache/incubator-airflow/pull/4207/diff?src=pr=tree#diff-YWlyZmxvdy9tb2RlbHMucHk=) | `92.33% <0%> (+0.04%)` | :arrow_up: | -- [Continue to review full report at Codecov](https://codecov.io/gh/apache/incubator-airflow/pull/4207?src=pr=continue). > **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta) > `Δ = absolute (impact)`, `ø = not affected`, `? = missing data` > Powered by [Codecov](https://codecov.io/gh/apache/incubator-airflow/pull/4207?src=pr=footer). Last update [94d1970...bd7f9cf](https://codecov.io/gh/apache/incubator-airflow/pull/4207?src=pr=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments). This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] jmcarp commented on issue #4207: Run celery integration test with redis broker.
jmcarp commented on issue #4207: Run celery integration test with redis broker. URL: https://github.com/apache/incubator-airflow/pull/4207#issuecomment-439925572 cc @ashb This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Created] (AIRFLOW-3367) Test celery with redis broker
Josh Carp created AIRFLOW-3367: -- Summary: Test celery with redis broker Key: AIRFLOW-3367 URL: https://issues.apache.org/jira/browse/AIRFLOW-3367 Project: Apache Airflow Issue Type: Test Reporter: Josh Carp Current integration tests use celery with the rabbitmq broker, but not the redis broker. We should test with both brokers to avoid regressions. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[GitHub] smentek commented on a change in pull request #4090: [AIRFLOW-3250] Fix for Redis Hook for not authorised connection calls
smentek commented on a change in pull request #4090: [AIRFLOW-3250] Fix for
Redis Hook for not authorised connection calls
URL: https://github.com/apache/incubator-airflow/pull/4090#discussion_r234651267
##
File path: tests/contrib/hooks/test_redis_hook.py
##
@@ -19,32 +19,116 @@
import unittest
-from mock import patch
-
+from mock import patch, MagicMock
from airflow import configuration
from airflow.contrib.hooks.redis_hook import RedisHook
class TestRedisHook(unittest.TestCase):
+
def setUp(self):
configuration.load_test_config()
-def test_get_conn(self):
+@patch('airflow.contrib.hooks.redis_hook.StrictRedis')
+@patch('airflow.contrib.hooks.redis_hook.RedisHook.get_connection')
+def test_get_conn(self, redis_hook_get_connection_mock, StrictRedisMock):
+HOST = 'localhost'
+PORT = 6379
+PASSWORD = 's3cret!'
+DB = 0
+
+extra_dejson_mock = MagicMock()
+extra_dejson_mock.get.return_value = DB
+connection_parameters = MagicMock()
+connection_parameters.configure_mock(
+host=HOST,
+port=PORT,
+password=PASSWORD,
+extra_dejson=extra_dejson_mock)
+redis_hook_get_connection_mock.return_value = connection_parameters
+
+hook = RedisHook(redis_conn_id='redis_default')
+self.assertEqual(hook.redis, None)
+
+self.assertEqual(hook.host, None, 'host initialised as None.')
+self.assertEqual(hook.port, None, 'port initialised as None.')
+self.assertEqual(hook.password, None, 'password initialised as None.')
+self.assertEqual(hook.db, None, 'db initialised as None.')
+
+self.assertIs(hook.get_conn(), hook.get_conn(), 'Connection
initialized only if None.')
+
+StrictRedisMock.assert_called_once_with(
+host=HOST,
+port=PORT,
+password=PASSWORD,
+db=DB)
+
+@patch('airflow.contrib.hooks.redis_hook.StrictRedis')
+@patch('airflow.contrib.hooks.redis_hook.RedisHook.get_connection')
Review comment:
Done.
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] codecov-io edited a comment on issue #4090: [AIRFLOW-3250] Fix for Redis Hook for not authorised connection calls
codecov-io edited a comment on issue #4090: [AIRFLOW-3250] Fix for Redis Hook for not authorised connection calls URL: https://github.com/apache/incubator-airflow/pull/4090#issuecomment-437704538 # [Codecov](https://codecov.io/gh/apache/incubator-airflow/pull/4090?src=pr=h1) Report > Merging [#4090](https://codecov.io/gh/apache/incubator-airflow/pull/4090?src=pr=desc) into [master](https://codecov.io/gh/apache/incubator-airflow/commit/94d19707d7f3563bb48868c2d6442c3da923da20?src=pr=desc) will **decrease** coverage by `0.04%`. > The diff coverage is `n/a`. [](https://codecov.io/gh/apache/incubator-airflow/pull/4090?src=pr=tree) ```diff @@Coverage Diff @@ ## master#4090 +/- ## == - Coverage77.7% 77.66% -0.05% == Files 199 199 Lines 1631716273 -44 == - Hits1267912638 -41 + Misses 3638 3635 -3 ``` | [Impacted Files](https://codecov.io/gh/apache/incubator-airflow/pull/4090?src=pr=tree) | Coverage Δ | | |---|---|---| | [airflow/www\_rbac/security.py](https://codecov.io/gh/apache/incubator-airflow/pull/4090/diff?src=pr=tree#diff-YWlyZmxvdy93d3dfcmJhYy9zZWN1cml0eS5weQ==) | `92.61% <0%> (-0.15%)` | :arrow_down: | | [airflow/models.py](https://codecov.io/gh/apache/incubator-airflow/pull/4090/diff?src=pr=tree#diff-YWlyZmxvdy9tb2RlbHMucHk=) | `92.19% <0%> (-0.11%)` | :arrow_down: | | [airflow/www\_rbac/views.py](https://codecov.io/gh/apache/incubator-airflow/pull/4090/diff?src=pr=tree#diff-YWlyZmxvdy93d3dfcmJhYy92aWV3cy5weQ==) | `72.32% <0%> (-0.07%)` | :arrow_down: | | [airflow/www\_rbac/app.py](https://codecov.io/gh/apache/incubator-airflow/pull/4090/diff?src=pr=tree#diff-YWlyZmxvdy93d3dfcmJhYy9hcHAucHk=) | `97.05% <0%> (-0.03%)` | :arrow_down: | | [airflow/api/common/experimental/trigger\_dag.py](https://codecov.io/gh/apache/incubator-airflow/pull/4090/diff?src=pr=tree#diff-YWlyZmxvdy9hcGkvY29tbW9uL2V4cGVyaW1lbnRhbC90cmlnZ2VyX2RhZy5weQ==) | `100% <0%> (ø)` | :arrow_up: | | [airflow/operators/bash\_operator.py](https://codecov.io/gh/apache/incubator-airflow/pull/4090/diff?src=pr=tree#diff-YWlyZmxvdy9vcGVyYXRvcnMvYmFzaF9vcGVyYXRvci5weQ==) | `91.37% <0%> (ø)` | :arrow_up: | | [airflow/operators/docker\_operator.py](https://codecov.io/gh/apache/incubator-airflow/pull/4090/diff?src=pr=tree#diff-YWlyZmxvdy9vcGVyYXRvcnMvZG9ja2VyX29wZXJhdG9yLnB5) | `97.67% <0%> (ø)` | :arrow_up: | | [airflow/bin/cli.py](https://codecov.io/gh/apache/incubator-airflow/pull/4090/diff?src=pr=tree#diff-YWlyZmxvdy9iaW4vY2xpLnB5) | `64.82% <0%> (+0.23%)` | :arrow_up: | -- [Continue to review full report at Codecov](https://codecov.io/gh/apache/incubator-airflow/pull/4090?src=pr=continue). > **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta) > `Δ = absolute (impact)`, `ø = not affected`, `? = missing data` > Powered by [Codecov](https://codecov.io/gh/apache/incubator-airflow/pull/4090?src=pr=footer). Last update [94d1970...ceeca05](https://codecov.io/gh/apache/incubator-airflow/pull/4090?src=pr=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments). This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] codecov-io commented on issue #4211: AIRFLOW-3365 and AIRFLOW-3366
codecov-io commented on issue #4211: AIRFLOW-3365 and AIRFLOW-3366 URL: https://github.com/apache/incubator-airflow/pull/4211#issuecomment-439907380 # [Codecov](https://codecov.io/gh/apache/incubator-airflow/pull/4211?src=pr=h1) Report > Merging [#4211](https://codecov.io/gh/apache/incubator-airflow/pull/4211?src=pr=desc) into [master](https://codecov.io/gh/apache/incubator-airflow/commit/94d19707d7f3563bb48868c2d6442c3da923da20?src=pr=desc) will **increase** coverage by `<.01%`. > The diff coverage is `50%`. [](https://codecov.io/gh/apache/incubator-airflow/pull/4211?src=pr=tree) ```diff @@Coverage Diff@@ ## master #4211 +/- ## = + Coverage77.7% 77.7% +<.01% = Files 199 199 Lines 16317 16322 +5 = + Hits12679 12683 +4 - Misses 36383639 +1 ``` | [Impacted Files](https://codecov.io/gh/apache/incubator-airflow/pull/4211?src=pr=tree) | Coverage Δ | | |---|---|---| | [airflow/config\_templates/default\_celery.py](https://codecov.io/gh/apache/incubator-airflow/pull/4211/diff?src=pr=tree#diff-YWlyZmxvdy9jb25maWdfdGVtcGxhdGVzL2RlZmF1bHRfY2VsZXJ5LnB5) | `58.62% <0%> (ø)` | :arrow_up: | | [airflow/configuration.py](https://codecov.io/gh/apache/incubator-airflow/pull/4211/diff?src=pr=tree#diff-YWlyZmxvdy9jb25maWd1cmF0aW9uLnB5) | `88.53% <60%> (-0.53%)` | :arrow_down: | | [airflow/models.py](https://codecov.io/gh/apache/incubator-airflow/pull/4211/diff?src=pr=tree#diff-YWlyZmxvdy9tb2RlbHMucHk=) | `92.33% <0%> (+0.04%)` | :arrow_up: | -- [Continue to review full report at Codecov](https://codecov.io/gh/apache/incubator-airflow/pull/4211?src=pr=continue). > **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta) > `Δ = absolute (impact)`, `ø = not affected`, `? = missing data` > Powered by [Codecov](https://codecov.io/gh/apache/incubator-airflow/pull/4211?src=pr=footer). Last update [94d1970...951a93a](https://codecov.io/gh/apache/incubator-airflow/pull/4211?src=pr=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments). This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Commented] (AIRFLOW-3365) Missing visibility_timeout will overwrite celery broker transport options
[
https://issues.apache.org/jira/browse/AIRFLOW-3365?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16691715#comment-16691715
]
ASF GitHub Bot commented on AIRFLOW-3365:
-
villasv opened a new pull request #4211: AIRFLOW-3365 and AIRFLOW-3366
URL: https://github.com/apache/incubator-airflow/pull/4211
### Jira
- [x] My PR addresses the following [Airflow
Jira](https://issues.apache.org/jira/browse/AIRFLOW/) issues and references
them in the PR title. For example, "\[AIRFLOW-XXX\] My Airflow PR"
- https://issues.apache.org/jira/browse/AIRFLOW-3365
- https://issues.apache.org/jira/browse/AIRFLOW-3366
### Description
A small change on how the `visibility_timeout` option is enforced and a
significant change on how the `getsection()` works, making it look for
environment variables as well. This allows the use of environment variables to
override `celery_broker_transport_options` and possible other options in
sections that are configured as a group.
### Tests
- [ ] My PR adds the following unit tests __OR__ does not need testing for
this extremely good reason:
### Commits
- [X] My commits all reference Jira issues in their subject lines, and I
have squashed multiple commits if they address the same issue. In addition, my
commits follow the guidelines from "[How to write a good git commit
message](http://chris.beams.io/posts/git-commit/)":
1. Subject is separated from body by a blank line
1. Subject is limited to 50 characters (not including Jira issue reference)
1. Subject does not end with a period
1. Subject uses the imperative mood ("add", not "adding")
1. Body wraps at 72 characters
1. Body explains "what" and "why", not "how"
### Documentation
- [X] In case of new functionality, my PR adds documentation that describes
how to use it.
- When adding new operators/hooks/sensors, the autoclass documentation
generation needs to be added.
### Code Quality
- [ ] Passes `flake8`
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
> Missing visibility_timeout will overwrite celery broker transport options
> -
>
> Key: AIRFLOW-3365
> URL: https://issues.apache.org/jira/browse/AIRFLOW-3365
> Project: Apache Airflow
> Issue Type: Bug
> Components: configuration
>Affects Versions: 1.10.0
>Reporter: Victor Villas Bôas Chaves
>Priority: Major
>
> If you specify the region in your airflow.cfg like this:
> {{[celery_broker_transport_options] }}
> {{region = us-east-2 }}
> it gets ignored.
> This is ignored because Airflow checks to see if the {{visibility_timeout}}
> parameter is set in the transport options and, if it's not set, replaces the
> transport options completely with {{{'visibility_timeout': 21600}}} (at
> [https://github.com/apache/incubator-airflow/blob/master/airflow/config_templates/default_celery.py#L40]).
> If you add {{visibility_timeout}}, the region is correctly set:
> {{[celery_broker_transport_options] }}
> {{region = us-east-2 }}
> {{visibility_timeout = 21600 }}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[GitHub] villasv opened a new pull request #4211: AIRFLOW-3365 and AIRFLOW-3366
villasv opened a new pull request #4211: AIRFLOW-3365 and AIRFLOW-3366
URL: https://github.com/apache/incubator-airflow/pull/4211
### Jira
- [x] My PR addresses the following [Airflow
Jira](https://issues.apache.org/jira/browse/AIRFLOW/) issues and references
them in the PR title. For example, "\[AIRFLOW-XXX\] My Airflow PR"
- https://issues.apache.org/jira/browse/AIRFLOW-3365
- https://issues.apache.org/jira/browse/AIRFLOW-3366
### Description
A small change on how the `visibility_timeout` option is enforced and a
significant change on how the `getsection()` works, making it look for
environment variables as well. This allows the use of environment variables to
override `celery_broker_transport_options` and possible other options in
sections that are configured as a group.
### Tests
- [ ] My PR adds the following unit tests __OR__ does not need testing for
this extremely good reason:
### Commits
- [X] My commits all reference Jira issues in their subject lines, and I
have squashed multiple commits if they address the same issue. In addition, my
commits follow the guidelines from "[How to write a good git commit
message](http://chris.beams.io/posts/git-commit/)":
1. Subject is separated from body by a blank line
1. Subject is limited to 50 characters (not including Jira issue reference)
1. Subject does not end with a period
1. Subject uses the imperative mood ("add", not "adding")
1. Body wraps at 72 characters
1. Body explains "what" and "why", not "how"
### Documentation
- [X] In case of new functionality, my PR adds documentation that describes
how to use it.
- When adding new operators/hooks/sensors, the autoclass documentation
generation needs to be added.
### Code Quality
- [ ] Passes `flake8`
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[jira] [Created] (AIRFLOW-3365) Missing visibility_timeout will overwrite celery broker transport options
Victor Villas Bôas Chaves created AIRFLOW-3365:
--
Summary: Missing visibility_timeout will overwrite celery broker
transport options
Key: AIRFLOW-3365
URL: https://issues.apache.org/jira/browse/AIRFLOW-3365
Project: Apache Airflow
Issue Type: Bug
Components: configuration
Affects Versions: 1.10.0
Reporter: Victor Villas Bôas Chaves
If you specify the region in your airflow.cfg like this:
{{[celery_broker_transport_options] }}
{{region = us-east-2 }}
it gets ignored.
This is ignored because Airflow checks to see if the {{visibility_timeout}}
parameter is set in the transport options and, if it's not set, replaces the
transport options completely with {{{'visibility_timeout': 21600}}} (at
[https://github.com/apache/incubator-airflow/blob/master/airflow/config_templates/default_celery.py#L40]).
If you add {{visibility_timeout}}, the region is correctly set:
{{[celery_broker_transport_options] }}
{{region = us-east-2 }}
{{visibility_timeout = 21600 }}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Created] (AIRFLOW-3366) AirflowConfigParser getsection() doesn't pick environment overrides
Victor Villas Bôas Chaves created AIRFLOW-3366: -- Summary: AirflowConfigParser getsection() doesn't pick environment overrides Key: AIRFLOW-3366 URL: https://issues.apache.org/jira/browse/AIRFLOW-3366 Project: Apache Airflow Issue Type: Bug Components: configuration Affects Versions: 1.10.0 Reporter: Victor Villas Bôas Chaves -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[GitHub] smentek commented on a change in pull request #4090: [AIRFLOW-3250] Fix for Redis Hook for not authorised connection calls
smentek commented on a change in pull request #4090: [AIRFLOW-3250] Fix for
Redis Hook for not authorised connection calls
URL: https://github.com/apache/incubator-airflow/pull/4090#discussion_r234617813
##
File path: tests/contrib/hooks/test_redis_hook.py
##
@@ -19,32 +19,116 @@
import unittest
-from mock import patch
-
+from mock import patch, MagicMock
from airflow import configuration
from airflow.contrib.hooks.redis_hook import RedisHook
class TestRedisHook(unittest.TestCase):
+
def setUp(self):
configuration.load_test_config()
-def test_get_conn(self):
+@patch('airflow.contrib.hooks.redis_hook.StrictRedis')
+@patch('airflow.contrib.hooks.redis_hook.RedisHook.get_connection')
+def test_get_conn(self, redis_hook_get_connection_mock, StrictRedisMock):
+HOST = 'localhost'
+PORT = 6379
+PASSWORD = 's3cret!'
+DB = 0
+
+extra_dejson_mock = MagicMock()
+extra_dejson_mock.get.return_value = DB
+connection_parameters = MagicMock()
+connection_parameters.configure_mock(
+host=HOST,
+port=PORT,
+password=PASSWORD,
+extra_dejson=extra_dejson_mock)
+redis_hook_get_connection_mock.return_value = connection_parameters
+
+hook = RedisHook(redis_conn_id='redis_default')
+self.assertEqual(hook.redis, None)
+
+self.assertEqual(hook.host, None, 'host initialised as None.')
+self.assertEqual(hook.port, None, 'port initialised as None.')
+self.assertEqual(hook.password, None, 'password initialised as None.')
+self.assertEqual(hook.db, None, 'db initialised as None.')
+
+self.assertIs(hook.get_conn(), hook.get_conn(), 'Connection
initialized only if None.')
+
+StrictRedisMock.assert_called_once_with(
+host=HOST,
+port=PORT,
+password=PASSWORD,
+db=DB)
+
+@patch('airflow.contrib.hooks.redis_hook.StrictRedis')
+@patch('airflow.contrib.hooks.redis_hook.RedisHook.get_connection')
Review comment:
Done.
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] codecov-io edited a comment on issue #4090: [AIRFLOW-3250] Fix for Redis Hook for not authorised connection calls
codecov-io edited a comment on issue #4090: [AIRFLOW-3250] Fix for Redis Hook for not authorised connection calls URL: https://github.com/apache/incubator-airflow/pull/4090#issuecomment-437704538 # [Codecov](https://codecov.io/gh/apache/incubator-airflow/pull/4090?src=pr=h1) Report > Merging [#4090](https://codecov.io/gh/apache/incubator-airflow/pull/4090?src=pr=desc) into [master](https://codecov.io/gh/apache/incubator-airflow/commit/94d19707d7f3563bb48868c2d6442c3da923da20?src=pr=desc) will **decrease** coverage by `0.04%`. > The diff coverage is `n/a`. [](https://codecov.io/gh/apache/incubator-airflow/pull/4090?src=pr=tree) ```diff @@Coverage Diff @@ ## master#4090 +/- ## == - Coverage77.7% 77.66% -0.05% == Files 199 199 Lines 1631716273 -44 == - Hits1267912638 -41 + Misses 3638 3635 -3 ``` | [Impacted Files](https://codecov.io/gh/apache/incubator-airflow/pull/4090?src=pr=tree) | Coverage Δ | | |---|---|---| | [airflow/www\_rbac/security.py](https://codecov.io/gh/apache/incubator-airflow/pull/4090/diff?src=pr=tree#diff-YWlyZmxvdy93d3dfcmJhYy9zZWN1cml0eS5weQ==) | `92.61% <0%> (-0.15%)` | :arrow_down: | | [airflow/models.py](https://codecov.io/gh/apache/incubator-airflow/pull/4090/diff?src=pr=tree#diff-YWlyZmxvdy9tb2RlbHMucHk=) | `92.19% <0%> (-0.11%)` | :arrow_down: | | [airflow/www\_rbac/views.py](https://codecov.io/gh/apache/incubator-airflow/pull/4090/diff?src=pr=tree#diff-YWlyZmxvdy93d3dfcmJhYy92aWV3cy5weQ==) | `72.32% <0%> (-0.07%)` | :arrow_down: | | [airflow/www\_rbac/app.py](https://codecov.io/gh/apache/incubator-airflow/pull/4090/diff?src=pr=tree#diff-YWlyZmxvdy93d3dfcmJhYy9hcHAucHk=) | `97.05% <0%> (-0.03%)` | :arrow_down: | | [airflow/api/common/experimental/trigger\_dag.py](https://codecov.io/gh/apache/incubator-airflow/pull/4090/diff?src=pr=tree#diff-YWlyZmxvdy9hcGkvY29tbW9uL2V4cGVyaW1lbnRhbC90cmlnZ2VyX2RhZy5weQ==) | `100% <0%> (ø)` | :arrow_up: | | [airflow/operators/bash\_operator.py](https://codecov.io/gh/apache/incubator-airflow/pull/4090/diff?src=pr=tree#diff-YWlyZmxvdy9vcGVyYXRvcnMvYmFzaF9vcGVyYXRvci5weQ==) | `91.37% <0%> (ø)` | :arrow_up: | | [airflow/operators/docker\_operator.py](https://codecov.io/gh/apache/incubator-airflow/pull/4090/diff?src=pr=tree#diff-YWlyZmxvdy9vcGVyYXRvcnMvZG9ja2VyX29wZXJhdG9yLnB5) | `97.67% <0%> (ø)` | :arrow_up: | | [airflow/bin/cli.py](https://codecov.io/gh/apache/incubator-airflow/pull/4090/diff?src=pr=tree#diff-YWlyZmxvdy9iaW4vY2xpLnB5) | `64.82% <0%> (+0.23%)` | :arrow_up: | -- [Continue to review full report at Codecov](https://codecov.io/gh/apache/incubator-airflow/pull/4090?src=pr=continue). > **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta) > `Δ = absolute (impact)`, `ø = not affected`, `? = missing data` > Powered by [Codecov](https://codecov.io/gh/apache/incubator-airflow/pull/4090?src=pr=footer). Last update [94d1970...71b860c](https://codecov.io/gh/apache/incubator-airflow/pull/4090?src=pr=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments). This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] ckljohn commented on a change in pull request #4153: [AIRFLOW-3308] Fix plugins import
ckljohn commented on a change in pull request #4153: [AIRFLOW-3308] Fix plugins import URL: https://github.com/apache/incubator-airflow/pull/4153#discussion_r234604230 ## File path: airflow/__init__.py ## @@ -79,3 +79,16 @@ class AirflowViewPlugin(BaseView): class AirflowMacroPlugin(object): def __init__(self, namespace): self.namespace = namespace + + +from airflow import operators # noqa: E402 +from airflow import sensors # noqa: E402 +from airflow import hooks # noqa: E402 +from airflow import executors # noqa: E402 +from airflow import macros # noqa: E402 + +operators._integrate_plugins() +sensors._integrate_plugins() # noqa: E402 Review comment: I just follow the original code. https://github.com/apache/incubator-airflow/pull/3906/files This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] codecov-io commented on issue #4210: [AIRFLOW-XXX] Don't publish md5 sigs as part of release
codecov-io commented on issue #4210: [AIRFLOW-XXX] Don't publish md5 sigs as part of release URL: https://github.com/apache/incubator-airflow/pull/4210#issuecomment-439866424 # [Codecov](https://codecov.io/gh/apache/incubator-airflow/pull/4210?src=pr=h1) Report > Merging [#4210](https://codecov.io/gh/apache/incubator-airflow/pull/4210?src=pr=desc) into [master](https://codecov.io/gh/apache/incubator-airflow/commit/94d19707d7f3563bb48868c2d6442c3da923da20?src=pr=desc) will **not change** coverage. > The diff coverage is `n/a`. [](https://codecov.io/gh/apache/incubator-airflow/pull/4210?src=pr=tree) ```diff @@ Coverage Diff @@ ## master #4210 +/- ## == Coverage77.7% 77.7% == Files 199 199 Lines 16317 16317 == Hits12679 12679 Misses 36383638 ``` -- [Continue to review full report at Codecov](https://codecov.io/gh/apache/incubator-airflow/pull/4210?src=pr=continue). > **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta) > `Δ = absolute (impact)`, `ø = not affected`, `? = missing data` > Powered by [Codecov](https://codecov.io/gh/apache/incubator-airflow/pull/4210?src=pr=footer). Last update [94d1970...bf446c8](https://codecov.io/gh/apache/incubator-airflow/pull/4210?src=pr=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments). This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Resolved] (AIRFLOW-3354) Scheduler compares offset-naive and offset-aware dates
[
https://issues.apache.org/jira/browse/AIRFLOW-3354?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ash Berlin-Taylor resolved AIRFLOW-3354.
Resolution: Not A Bug
> Scheduler compares offset-naive and offset-aware dates
> --
>
> Key: AIRFLOW-3354
> URL: https://issues.apache.org/jira/browse/AIRFLOW-3354
> Project: Apache Airflow
> Issue Type: Bug
>Affects Versions: 1.10.1
>Reporter: Jakub Powierza
>Priority: Major
>
> New version of Airflow (1.10.1rc1 and 1.10.1rc2) tries to compare
> offset-naive and offset-aware dates in Scheduler. I've tested a simple case
> with schedule set to "*/10 * * * *". I've tried to clean my developer
> instance with `airflow resetdb` and start from scratch but it does not help
> at all. This issue does not occur on stable version 1.10.0.
> My setup: Python 3.6 on Ubuntu 14.04 with Airflow Scheduler based on Celery
> with RabbitMQ backend.
> Exception found in Scheduler logs:
> {code:java}
> 2018-11-15 14:41:23,194:ERROR:airflow.processor:[CT=None] Got an exception!
> Propagating...
> Traceback (most recent call last):
> File
> "/home/jpowierz/my_project/venv/lib/python3.6/site-packages/airflow/jobs.py",
> line 389, in helper
> pickle_dags)
> File
> "/home/jpowierz/my_project/venv/lib/python3.6/site-packages/airflow/utils/db.py",
> line 74, in wrapper
> return func(*args, **kwargs)
> File
> "/home/jpowierz/my_project/venv/lib/python3.6/site-packages/airflow/jobs.py",
> line 1846, in process_file
> self._process_dags(dagbag, dags, ti_keys_to_schedule)
> File
> "/home/jpowierz/my_project/venv/lib/python3.6/site-packages/airflow/jobs.py",
> line 1426, in _process_dags
> dag_run = self.create_dag_run(dag)
> File
> "/home/jpowierz/my_project/venv/lib/python3.6/site-packages/airflow/utils/db.py",
> line 74, in wrapper
> return func(*args, **kwargs)
> File
> "/home/jpowierz/my_project/venv/lib/python3.6/site-packages/airflow/jobs.py",
> line 909, in create_dag_run
> external_trigger=False
> File
> "/home/jpowierz/my_project/venv/lib/python3.6/site-packages/airflow/utils/db.py",
> line 74, in wrapper
> return func(*args, **kwargs)
> File
> "/home/jpowierz/my_project/venv/lib/python3.6/site-packages/airflow/models.py",
> line 4270, in create_dagrun
> run.verify_integrity(session=session)
> File
> "/home/jpowierz/my_project/venv/lib/python3.6/site-packages/airflow/utils/db.py",
> line 70, in wrapper
> return func(*args, **kwargs)
> File
> "/home/jpowierz/my_project/venv/lib/python3.6/site-packages/airflow/models.py",
> line 5215, in verify_integrity
> if task.start_date > self.execution_date and not self.is_backfill:
> TypeError: can't compare offset-naive and offset-aware datetimes
> Process DagFileProcessor40-Process:
> Traceback (most recent call last):
> File "/usr/lib/python3.6/multiprocessing/process.py", line 249, in _bootstrap
> self.run()
> File "/usr/lib/python3.6/multiprocessing/process.py", line 93, in run
> self._target(*self._args, **self._kwargs)
> File
> "/home/jpowierz/my_project/venv/lib/python3.6/site-packages/airflow/jobs.py",
> line 389, in helper
> pickle_dags)
> File
> "/home/jpowierz/my_project/venv/lib/python3.6/site-packages/airflow/utils/db.py",
> line 74, in wrapper
> return func(*args, **kwargs)
> File
> "/home/jpowierz/my_project/venv/lib/python3.6/site-packages/airflow/jobs.py",
> line 1846, in process_file
> self._process_dags(dagbag, dags, ti_keys_to_schedule)
> File
> "/home/jpowierz/my_project/venv/lib/python3.6/site-packages/airflow/jobs.py",
> line 1426, in _process_dags
> dag_run = self.create_dag_run(dag)
> File
> "/home/jpowierz/my_project/venv/lib/python3.6/site-packages/airflow/utils/db.py",
> line 74, in wrapper
> return func(*args, **kwargs)
> File
> "/home/jpowierz/my_project/venv/lib/python3.6/site-packages/airflow/jobs.py",
> line 909, in create_dag_run
> external_trigger=False
> File
> "/home/jpowierz/my_project/venv/lib/python3.6/site-packages/airflow/utils/db.py",
> line 74, in wrapper
> return func(*args, **kwargs)
> File
> "/home/jpowierz/my_project/venv/lib/python3.6/site-packages/airflow/models.py",
> line 4270, in create_dagrun
> run.verify_integrity(session=session)
> File
> "/home/jpowierz/my_project/venv/lib/python3.6/site-packages/airflow/utils/db.py",
> line 70, in wrapper
> return func(*args, **kwargs)
> File
> "/home/jpowierz/my_project/venv/lib/python3.6/site-packages/airflow/models.py",
> line 5215, in verify_integrity
> if task.start_date > self.execution_date and not self.is_backfill:
> TypeError: can't compare offset-naive and offset-aware datetimes
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[GitHub] ashb commented on a change in pull request #4153: [AIRFLOW-3308] Fix plugins import
ashb commented on a change in pull request #4153: [AIRFLOW-3308] Fix plugins import URL: https://github.com/apache/incubator-airflow/pull/4153#discussion_r234585552 ## File path: airflow/__init__.py ## @@ -79,3 +79,16 @@ class AirflowViewPlugin(BaseView): class AirflowMacroPlugin(object): def __init__(self, namespace): self.namespace = namespace + + +from airflow import operators # noqa: E402 +from airflow import sensors # noqa: E402 +from airflow import hooks # noqa: E402 +from airflow import executors # noqa: E402 +from airflow import macros # noqa: E402 + +operators._integrate_plugins() +sensors._integrate_plugins() # noqa: E402 Review comment: This one shouldn't need a noqa should it? This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] ashb opened a new pull request #4210: [AIRFLOW-XXX] Don't publish md5 sigs as part of release
ashb opened a new pull request #4210: [AIRFLOW-XXX] Don't publish md5 sigs as
part of release
URL: https://github.com/apache/incubator-airflow/pull/4210
Make sure you have checked _all_ steps below.
### Jira
- [x] No Jira
### Description
- [x] Apache recommend against publishing MD5 files now as they are
relatively
easy to collide and shouldn't be trusted anymore
### Tests
- [x] My PR adds the following unit tests __OR__ does not need testing for
this extremely good reason:
### Commits
- [x] My commits all reference Jira issues in their subject lines, and I
have squashed multiple commits if they address the same issue. In addition, my
commits follow the guidelines from "[How to write a good git commit
message](http://chris.beams.io/posts/git-commit/)":
1. Subject is separated from body by a blank line
1. Subject is limited to 50 characters (not including Jira issue reference)
1. Subject does not end with a period
1. Subject uses the imperative mood ("add", not "adding")
1. Body wraps at 72 characters
1. Body explains "what" and "why", not "how"
### Documentation
- [x] In case of new functionality, my PR adds documentation that describes
how to use it.
- When adding new operators/hooks/sensors, the autoclass documentation
generation needs to be added.
### Code Quality
- [x] Passes `flake8`
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[jira] [Commented] (AIRFLOW-3354) Scheduler compares offset-naive and offset-aware dates
[
https://issues.apache.org/jira/browse/AIRFLOW-3354?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16691572#comment-16691572
]
Jakub Powierza commented on AIRFLOW-3354:
-
You're right - I'll double check why we do so. I found it while upgrading
Airflow to the newest version and thought it may be useful for you.
Feel free to close this issue if you think that it is not a bug on your side!
Thanks for your time and sorry for false alarm!
> Scheduler compares offset-naive and offset-aware dates
> --
>
> Key: AIRFLOW-3354
> URL: https://issues.apache.org/jira/browse/AIRFLOW-3354
> Project: Apache Airflow
> Issue Type: Bug
>Affects Versions: 1.10.1
>Reporter: Jakub Powierza
>Priority: Major
>
> New version of Airflow (1.10.1rc1 and 1.10.1rc2) tries to compare
> offset-naive and offset-aware dates in Scheduler. I've tested a simple case
> with schedule set to "*/10 * * * *". I've tried to clean my developer
> instance with `airflow resetdb` and start from scratch but it does not help
> at all. This issue does not occur on stable version 1.10.0.
> My setup: Python 3.6 on Ubuntu 14.04 with Airflow Scheduler based on Celery
> with RabbitMQ backend.
> Exception found in Scheduler logs:
> {code:java}
> 2018-11-15 14:41:23,194:ERROR:airflow.processor:[CT=None] Got an exception!
> Propagating...
> Traceback (most recent call last):
> File
> "/home/jpowierz/my_project/venv/lib/python3.6/site-packages/airflow/jobs.py",
> line 389, in helper
> pickle_dags)
> File
> "/home/jpowierz/my_project/venv/lib/python3.6/site-packages/airflow/utils/db.py",
> line 74, in wrapper
> return func(*args, **kwargs)
> File
> "/home/jpowierz/my_project/venv/lib/python3.6/site-packages/airflow/jobs.py",
> line 1846, in process_file
> self._process_dags(dagbag, dags, ti_keys_to_schedule)
> File
> "/home/jpowierz/my_project/venv/lib/python3.6/site-packages/airflow/jobs.py",
> line 1426, in _process_dags
> dag_run = self.create_dag_run(dag)
> File
> "/home/jpowierz/my_project/venv/lib/python3.6/site-packages/airflow/utils/db.py",
> line 74, in wrapper
> return func(*args, **kwargs)
> File
> "/home/jpowierz/my_project/venv/lib/python3.6/site-packages/airflow/jobs.py",
> line 909, in create_dag_run
> external_trigger=False
> File
> "/home/jpowierz/my_project/venv/lib/python3.6/site-packages/airflow/utils/db.py",
> line 74, in wrapper
> return func(*args, **kwargs)
> File
> "/home/jpowierz/my_project/venv/lib/python3.6/site-packages/airflow/models.py",
> line 4270, in create_dagrun
> run.verify_integrity(session=session)
> File
> "/home/jpowierz/my_project/venv/lib/python3.6/site-packages/airflow/utils/db.py",
> line 70, in wrapper
> return func(*args, **kwargs)
> File
> "/home/jpowierz/my_project/venv/lib/python3.6/site-packages/airflow/models.py",
> line 5215, in verify_integrity
> if task.start_date > self.execution_date and not self.is_backfill:
> TypeError: can't compare offset-naive and offset-aware datetimes
> Process DagFileProcessor40-Process:
> Traceback (most recent call last):
> File "/usr/lib/python3.6/multiprocessing/process.py", line 249, in _bootstrap
> self.run()
> File "/usr/lib/python3.6/multiprocessing/process.py", line 93, in run
> self._target(*self._args, **self._kwargs)
> File
> "/home/jpowierz/my_project/venv/lib/python3.6/site-packages/airflow/jobs.py",
> line 389, in helper
> pickle_dags)
> File
> "/home/jpowierz/my_project/venv/lib/python3.6/site-packages/airflow/utils/db.py",
> line 74, in wrapper
> return func(*args, **kwargs)
> File
> "/home/jpowierz/my_project/venv/lib/python3.6/site-packages/airflow/jobs.py",
> line 1846, in process_file
> self._process_dags(dagbag, dags, ti_keys_to_schedule)
> File
> "/home/jpowierz/my_project/venv/lib/python3.6/site-packages/airflow/jobs.py",
> line 1426, in _process_dags
> dag_run = self.create_dag_run(dag)
> File
> "/home/jpowierz/my_project/venv/lib/python3.6/site-packages/airflow/utils/db.py",
> line 74, in wrapper
> return func(*args, **kwargs)
> File
> "/home/jpowierz/my_project/venv/lib/python3.6/site-packages/airflow/jobs.py",
> line 909, in create_dag_run
> external_trigger=False
> File
> "/home/jpowierz/my_project/venv/lib/python3.6/site-packages/airflow/utils/db.py",
> line 74, in wrapper
> return func(*args, **kwargs)
> File
> "/home/jpowierz/my_project/venv/lib/python3.6/site-packages/airflow/models.py",
> line 4270, in create_dagrun
> run.verify_integrity(session=session)
> File
> "/home/jpowierz/my_project/venv/lib/python3.6/site-packages/airflow/utils/db.py",
> line 70, in wrapper
> return func(*args, **kwargs)
> File
> "/home/jpowierz/my_project/venv/lib/python3.6/site-packages/airflow/models.py",
> line 5215, in verify_integrity
> if task.start_date >
svn commit: r30972 - in /dev/incubator/airflow/1.10.1rc2: apache-airflow-1.10.1rc2+incubating-bin.tar.gz.md5 apache-airflow-1.10.1rc2+incubating-source.tar.gz.md5
Author: ash Date: Mon Nov 19 11:14:06 2018 New Revision: 30972 Log: Remove md5 files MD5 files should not be published anymore - as per instruction of Henk P. Penning Removed: dev/incubator/airflow/1.10.1rc2/apache-airflow-1.10.1rc2+incubating-bin.tar.gz.md5 dev/incubator/airflow/1.10.1rc2/apache-airflow-1.10.1rc2+incubating-source.tar.gz.md5
[jira] [Updated] (AIRFLOW-3363) Plugin templates not rendered by Flask appbuilder's baseview
[
https://issues.apache.org/jira/browse/AIRFLOW-3363?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ran Zvi updated AIRFLOW-3363:
-
Description:
Hello, we're having issues with the new F.A.B plugins (rbac), the documentation
lacks the import for *{{AppBuilderBaseView}}*{{:}}
{code:java}
# Creating a flask appbuilder BaseView
class TestAppBuilderBaseView(AppBuilderBaseView):
@expose("/")
def test(self):
return self.render("test_plugin/test.html", content="Hello galaxy!"){code}
However after a bit of digging I found out it is probably imported from F.A.B
{code:java}
from flask_appbuilder import BaseView as AppBuilderBaseView{code}
The next issue is that the class lacks a *{{render}}* function and F.A.B only
provides a *{{render_template}}* function which ignores the
*{{plugins_folder}}* definition in *{{airflow.cfg}}* and the Blueprint's
*{{template_folder}}* definition in the plugin's code.
We've found out that in F.A.B's *{{Environment}}* Object created by Airflow the
only location defined for template location is under *{{www_rbac}}*{{.}}
We're unable to make any progress due to this Bug, does anyone know how to
Thanks in advance!
was:
Hello, I'm having issues with the new F.A.B plugins, the documentation lacks
the import for `AppBuilderBaseView`, However after a bit of digging I found out
it is probably `from flask_appbuilder import BaseView as AppBuilderBaseView`.
The next issue is that the class lacks a `render` function and F.A.B only
provides a `render_template` function which uses a preconfigured path for the
`template` folder which isn't under the airflow `plugins` folder.
Thabks in advance!
> Plugin templates not rendered by Flask appbuilder's baseview
>
>
> Key: AIRFLOW-3363
> URL: https://issues.apache.org/jira/browse/AIRFLOW-3363
> Project: Apache Airflow
> Issue Type: Bug
> Components: plugins
>Affects Versions: 1.10.0
> Environment: Docker
>Reporter: Ran Zvi
>Priority: Major
>
> Hello, we're having issues with the new F.A.B plugins (rbac), the
> documentation lacks the import for *{{AppBuilderBaseView}}*{{:}}
>
> {code:java}
> # Creating a flask appbuilder BaseView
> class TestAppBuilderBaseView(AppBuilderBaseView):
> @expose("/")
> def test(self):
> return self.render("test_plugin/test.html", content="Hello galaxy!"){code}
> However after a bit of digging I found out it is probably imported from F.A.B
> {code:java}
> from flask_appbuilder import BaseView as AppBuilderBaseView{code}
>
> The next issue is that the class lacks a *{{render}}* function and F.A.B only
> provides a *{{render_template}}* function which ignores the
> *{{plugins_folder}}* definition in *{{airflow.cfg}}* and the Blueprint's
> *{{template_folder}}* definition in the plugin's code.
> We've found out that in F.A.B's *{{Environment}}* Object created by Airflow
> the only location defined for template location is under *{{www_rbac}}*{{.}}
> We're unable to make any progress due to this Bug, does anyone know how to
> Thanks in advance!
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[GitHub] ron819 commented on issue #2709: [AIRFLOW-1739] Resolve TestDbApiHook naming ambiguity
ron819 commented on issue #2709: [AIRFLOW-1739] Resolve TestDbApiHook naming ambiguity URL: https://github.com/apache/incubator-airflow/pull/2709#issuecomment-439830013 @andyxhadji can you rebase? This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] codecov-io edited a comment on issue #4121: [AIRFLOW-2568] Azure Container Instances operator
codecov-io edited a comment on issue #4121: [AIRFLOW-2568] Azure Container Instances operator URL: https://github.com/apache/incubator-airflow/pull/4121#issuecomment-436818600 # [Codecov](https://codecov.io/gh/apache/incubator-airflow/pull/4121?src=pr=h1) Report > Merging [#4121](https://codecov.io/gh/apache/incubator-airflow/pull/4121?src=pr=desc) into [master](https://codecov.io/gh/apache/incubator-airflow/commit/94d19707d7f3563bb48868c2d6442c3da923da20?src=pr=desc) will **increase** coverage by `<.01%`. > The diff coverage is `75%`. [](https://codecov.io/gh/apache/incubator-airflow/pull/4121?src=pr=tree) ```diff @@Coverage Diff@@ ## master #4121 +/- ## = + Coverage77.7% 77.7% +<.01% = Files 199 199 Lines 16317 16321 +4 = + Hits12679 12683 +4 Misses 36383638 ``` | [Impacted Files](https://codecov.io/gh/apache/incubator-airflow/pull/4121?src=pr=tree) | Coverage Δ | | |---|---|---| | [airflow/utils/db.py](https://codecov.io/gh/apache/incubator-airflow/pull/4121/diff?src=pr=tree#diff-YWlyZmxvdy91dGlscy9kYi5weQ==) | `33.33% <0%> (-0.27%)` | :arrow_down: | | [airflow/models.py](https://codecov.io/gh/apache/incubator-airflow/pull/4121/diff?src=pr=tree#diff-YWlyZmxvdy9tb2RlbHMucHk=) | `92.34% <100%> (+0.04%)` | :arrow_up: | -- [Continue to review full report at Codecov](https://codecov.io/gh/apache/incubator-airflow/pull/4121?src=pr=continue). > **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta) > `Δ = absolute (impact)`, `ø = not affected`, `? = missing data` > Powered by [Codecov](https://codecov.io/gh/apache/incubator-airflow/pull/4121?src=pr=footer). Last update [94d1970...5464a86](https://codecov.io/gh/apache/incubator-airflow/pull/4121?src=pr=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments). This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] codecov-io edited a comment on issue #4121: [AIRFLOW-2568] Azure Container Instances operator
codecov-io edited a comment on issue #4121: [AIRFLOW-2568] Azure Container Instances operator URL: https://github.com/apache/incubator-airflow/pull/4121#issuecomment-436818600 # [Codecov](https://codecov.io/gh/apache/incubator-airflow/pull/4121?src=pr=h1) Report > Merging [#4121](https://codecov.io/gh/apache/incubator-airflow/pull/4121?src=pr=desc) into [master](https://codecov.io/gh/apache/incubator-airflow/commit/94d19707d7f3563bb48868c2d6442c3da923da20?src=pr=desc) will **increase** coverage by `<.01%`. > The diff coverage is `75%`. [](https://codecov.io/gh/apache/incubator-airflow/pull/4121?src=pr=tree) ```diff @@Coverage Diff@@ ## master #4121 +/- ## = + Coverage77.7% 77.7% +<.01% = Files 199 199 Lines 16317 16321 +4 = + Hits12679 12683 +4 Misses 36383638 ``` | [Impacted Files](https://codecov.io/gh/apache/incubator-airflow/pull/4121?src=pr=tree) | Coverage Δ | | |---|---|---| | [airflow/utils/db.py](https://codecov.io/gh/apache/incubator-airflow/pull/4121/diff?src=pr=tree#diff-YWlyZmxvdy91dGlscy9kYi5weQ==) | `33.33% <0%> (-0.27%)` | :arrow_down: | | [airflow/models.py](https://codecov.io/gh/apache/incubator-airflow/pull/4121/diff?src=pr=tree#diff-YWlyZmxvdy9tb2RlbHMucHk=) | `92.34% <100%> (+0.04%)` | :arrow_up: | -- [Continue to review full report at Codecov](https://codecov.io/gh/apache/incubator-airflow/pull/4121?src=pr=continue). > **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta) > `Δ = absolute (impact)`, `ø = not affected`, `? = missing data` > Powered by [Codecov](https://codecov.io/gh/apache/incubator-airflow/pull/4121?src=pr=footer). Last update [94d1970...5464a86](https://codecov.io/gh/apache/incubator-airflow/pull/4121?src=pr=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments). This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] codecov-io edited a comment on issue #4121: [AIRFLOW-2568] Azure Container Instances operator
codecov-io edited a comment on issue #4121: [AIRFLOW-2568] Azure Container Instances operator URL: https://github.com/apache/incubator-airflow/pull/4121#issuecomment-436818600 # [Codecov](https://codecov.io/gh/apache/incubator-airflow/pull/4121?src=pr=h1) Report > Merging [#4121](https://codecov.io/gh/apache/incubator-airflow/pull/4121?src=pr=desc) into [master](https://codecov.io/gh/apache/incubator-airflow/commit/94d19707d7f3563bb48868c2d6442c3da923da20?src=pr=desc) will **increase** coverage by `<.01%`. > The diff coverage is `75%`. [](https://codecov.io/gh/apache/incubator-airflow/pull/4121?src=pr=tree) ```diff @@Coverage Diff@@ ## master #4121 +/- ## = + Coverage77.7% 77.7% +<.01% = Files 199 199 Lines 16317 16321 +4 = + Hits12679 12683 +4 Misses 36383638 ``` | [Impacted Files](https://codecov.io/gh/apache/incubator-airflow/pull/4121?src=pr=tree) | Coverage Δ | | |---|---|---| | [airflow/utils/db.py](https://codecov.io/gh/apache/incubator-airflow/pull/4121/diff?src=pr=tree#diff-YWlyZmxvdy91dGlscy9kYi5weQ==) | `33.33% <0%> (-0.27%)` | :arrow_down: | | [airflow/models.py](https://codecov.io/gh/apache/incubator-airflow/pull/4121/diff?src=pr=tree#diff-YWlyZmxvdy9tb2RlbHMucHk=) | `92.34% <100%> (+0.04%)` | :arrow_up: | -- [Continue to review full report at Codecov](https://codecov.io/gh/apache/incubator-airflow/pull/4121?src=pr=continue). > **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta) > `Δ = absolute (impact)`, `ø = not affected`, `? = missing data` > Powered by [Codecov](https://codecov.io/gh/apache/incubator-airflow/pull/4121?src=pr=footer). Last update [94d1970...5464a86](https://codecov.io/gh/apache/incubator-airflow/pull/4121?src=pr=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments). This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Comment Edited] (AIRFLOW-987) `airflow kerberos` ignores --keytab and --principal arguments
[
https://issues.apache.org/jira/browse/AIRFLOW-987?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16691407#comment-16691407
]
Pratap20 edited comment on AIRFLOW-987 at 11/19/18 8:50 AM:
[~xnuinside] yes I am talking about command line issue , and for reference I
have sent settings in config file.
was (Author: pratap20):
[~xnuinside] yes I am talking about commandline issue , and for refernce I have
sent settings in config file.
> `airflow kerberos` ignores --keytab and --principal arguments
> -
>
> Key: AIRFLOW-987
> URL: https://issues.apache.org/jira/browse/AIRFLOW-987
> Project: Apache Airflow
> Issue Type: Bug
> Components: security
>Affects Versions: 1.8.0
> Environment: 1.8-rc5
>Reporter: Ruslan Dautkhanov
>Assignee: Pratap20
>Priority: Major
> Labels: easyfix, kerberos, security
>
> No matter which arguments I pass to `airflow kerberos`,
> it always executes as `kinit -r 3600m -k -t airflow.keytab -c
> /tmp/airflow_krb5_ccache airflow`
> So it failes with expected "kinit: Keytab contains no suitable keys for
> airf...@corp.some.com while getting initial credentials"
> Tried different arguments, -kt and --keytab, here's one of the runs (some
> lines wrapped for readability):
> {noformat}
> $ airflow kerberos -kt /home/rdautkha/.keytab rdautkha...@corp.some.com
> [2017-03-14 23:50:11,523] {__init__.py:57} INFO - Using executor LocalExecutor
> [2017-03-14 23:50:12,069] {kerberos.py:43} INFO - Reinitting kerberos from
> keytab:
> kinit -r 3600m -k -t airflow.keytab -c /tmp/airflow_krb5_ccache airflow
> [2017-03-14 23:50:12,080] {kerberos.py:55} ERROR -
> Couldn't reinit from keytab! `kinit' exited with 1.
> kinit: Keytab contains no suitable keys for airf...@corp.some.com
> while getting initial credentials
> {noformat}
> 1.8-rc5
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (AIRFLOW-987) `airflow kerberos` ignores --keytab and --principal arguments
[
https://issues.apache.org/jira/browse/AIRFLOW-987?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16691407#comment-16691407
]
Pratap20 commented on AIRFLOW-987:
--
[~xnuinside] yes I am talking about commandline issue , and for refernce I have
sent settings in config file.
> `airflow kerberos` ignores --keytab and --principal arguments
> -
>
> Key: AIRFLOW-987
> URL: https://issues.apache.org/jira/browse/AIRFLOW-987
> Project: Apache Airflow
> Issue Type: Bug
> Components: security
>Affects Versions: 1.8.0
> Environment: 1.8-rc5
>Reporter: Ruslan Dautkhanov
>Assignee: Pratap20
>Priority: Major
> Labels: easyfix, kerberos, security
>
> No matter which arguments I pass to `airflow kerberos`,
> it always executes as `kinit -r 3600m -k -t airflow.keytab -c
> /tmp/airflow_krb5_ccache airflow`
> So it failes with expected "kinit: Keytab contains no suitable keys for
> airf...@corp.some.com while getting initial credentials"
> Tried different arguments, -kt and --keytab, here's one of the runs (some
> lines wrapped for readability):
> {noformat}
> $ airflow kerberos -kt /home/rdautkha/.keytab rdautkha...@corp.some.com
> [2017-03-14 23:50:11,523] {__init__.py:57} INFO - Using executor LocalExecutor
> [2017-03-14 23:50:12,069] {kerberos.py:43} INFO - Reinitting kerberos from
> keytab:
> kinit -r 3600m -k -t airflow.keytab -c /tmp/airflow_krb5_ccache airflow
> [2017-03-14 23:50:12,080] {kerberos.py:55} ERROR -
> Couldn't reinit from keytab! `kinit' exited with 1.
> kinit: Keytab contains no suitable keys for airf...@corp.some.com
> while getting initial credentials
> {noformat}
> 1.8-rc5
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[GitHub] omusavi commented on issue #4121: [AIRFLOW-2568] Azure Container Instances operator
omusavi commented on issue #4121: [AIRFLOW-2568] Azure Container Instances operator URL: https://github.com/apache/incubator-airflow/pull/4121#issuecomment-439809982 Apologies for the bad iteration after splitting out the files @Fokko . Everything should be up to date now and I have verified the docs are updated and verified the scenario works end-to-end. Let me know if there is anything else (particularly with the one remaining unresolved comment) This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Created] (AIRFLOW-3364) Change Ooops Exception to something meanningfull
jack created AIRFLOW-3364:
-
Summary: Change Ooops Exception to something meanningfull
Key: AIRFLOW-3364
URL: https://issues.apache.org/jira/browse/AIRFLOW-3364
Project: Apache Airflow
Issue Type: Improvement
Affects Versions: 1.9.0
Reporter: jack
Attachments: oops.PNG
When I try to manually change tasks state (from up_for_retry) to success or
whatever.
I often get:
h1.
{code:java}
Ooops. Traceback (most recent call last): File
"/usr/local/lib/python2.7/dist-packages/flask/app.py", line 1988, in wsgi_app
response = self.full_dispatch_request() File
"/usr/local/lib/python2.7/dist-packages/flask/app.py", line 1641, in
full_dispatch_request rv = self.handle_user_exception(e) File
"/usr/local/lib/python2.7/dist-packages/flask/app.py", line 1544, in
handle_user_exception reraise(exc_type, exc_value, tb) File
"/usr/local/lib/python2.7/dist-packages/flask/app.py", line 1639, in
full_dispatch_request rv = self.dispatch_request() File
"/usr/local/lib/python2.7/dist-packages/flask/app.py", line 1625, in
dispatch_request return self.view_functions[rule.endpoint](**req.view_args)
File "/usr/local/lib/python2.7/dist-packages/flask_admin/base.py", line 69, in
inner return self._run_view(f, *args, **kwargs) File
"/usr/local/lib/python2.7/dist-packages/flask_admin/base.py", line 368, in
_run_view return fn(self, *args, **kwargs) File
"/usr/local/lib/python2.7/dist-packages/flask_admin/model/base.py", line 2068,
in action_view return self.handle_action() File
"/usr/local/lib/python2.7/dist-packages/flask_admin/actions.py", line 113, in
handle_action response = handler[0](ids) File
"/usr/local/lib/python2.7/dist-packages/airflow/www/views.py", line 2502, in
action_set_success self.set_task_instance_state(ids, State.SUCCESS) File
"/usr/local/lib/python2.7/dist-packages/airflow/utils/db.py", line 50, in
wrapper result = func(*args, **kwargs) File
"/usr/local/lib/python2.7/dist-packages/airflow/www/views.py", line 2540, in
set_task_instance_state raise Exception("Ooops") Exception: Ooops{code}
It's unclear what this oops is and why it's needed.
If there is exception it should shown the actual error.
Usually I simply delete the task to solve this.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
