[jira] [Commented] (AIRFLOW-6527) Error sending Celery task:Timeout in send_task_to_executor
[
https://issues.apache.org/jira/browse/AIRFLOW-6527?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17013669#comment-17013669
]
ASF GitHub Bot commented on AIRFLOW-6527:
-
yuqian90 commented on pull request #7143: [AIRFLOW-6527] Make
send_task_to_executor timeout configurable
URL: https://github.com/apache/airflow/pull/7143
This change makes the timeout for sending tasks to Celery worker
configurable. If people encounter the same Celery timeout error described in
AIRFLOW-6527, they may increase `[celery]send_task_timeout` to make the timeout
less likely to happen.
---
Issue link: WILL BE INSERTED BY
[boring-cyborg](https://github.com/kaxil/boring-cyborg)
- [ ] Description above provides context of the change
- [ ] Commit message/PR title starts with `[AIRFLOW-]`. AIRFLOW- =
JIRA ID*
- [ ] Unit tests coverage for changes (not needed for documentation changes)
- [ ] Commits follow "[How to write a good git commit
message](http://chris.beams.io/posts/git-commit/)"
- [ ] Relevant documentation is updated including usage instructions.
- [ ] I will engage committers as explained in [Contribution Workflow
Example](https://github.com/apache/airflow/blob/master/CONTRIBUTING.rst#contribution-workflow-example).
* For document-only changes commit message can start with
`[AIRFLOW-]`.
---
In case of fundamental code change, Airflow Improvement Proposal
([AIP](https://cwiki.apache.org/confluence/display/AIRFLOW/Airflow+Improvements+Proposals))
is needed.
In case of a new dependency, check compliance with the [ASF 3rd Party
License Policy](https://www.apache.org/legal/resolved.html#category-x).
In case of backwards incompatible changes please leave a note in
[UPDATING.md](https://github.com/apache/airflow/blob/master/UPDATING.md).
Read the [Pull Request
Guidelines](https://github.com/apache/airflow/blob/master/CONTRIBUTING.rst#pull-request-guidelines)
for more information.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
> Error sending Celery task:Timeout in send_task_to_executor
> --
>
> Key: AIRFLOW-6527
> URL: https://issues.apache.org/jira/browse/AIRFLOW-6527
> Project: Apache Airflow
> Issue Type: Bug
> Components: scheduler
>Affects Versions: 1.10.7
>Reporter: Qian Yu
>Priority: Major
>
> We use Airflow with CeleryExecutor and redis broker. Our airflow scheduler
> often encounters this \{{AirflowTaskTimeout}} error.
> - This happens in \{{send_task_to_executor()}}.
> - It only happens occasionally.
> - Retrying the failed task a few times always works.
> - This affects at least 1.10.6 and 1.10.7 and possibly other versions too.
> Possible cause:
> Our airflow venv and dags_folder are on an NFS mount because we want to keep
> the various pieces of Airflow services in sync.
> The NFS mount can be slow sometimes. This causes the import to be slow and
> causes \{{send_task_to_executor()}} to take more than 2 seconds.
> Other people with similar looking problems:
> The following issue is now closed. It's not clear to me whether or how the
> user resolved this issue.
> https://github.com/bitnami/bitnami-docker-airflow-scheduler/issues/1
> Another user asked a question in the mailing list. It's not answered.
> https://www.mail-archive.com/dev@airflow.apache.org/msg01093.html
> Proposed workaround:
> - Make this `timeout(seconds=2)` configurable. E.g adding a
> [celery]send_task_timeout to airflow.cfg. Since 2 seconds seems too short, we
> can configure it to something like 15 seconds to make it much less likely to
> happen.
> - Move airflow venv to the local disk. This makes it inconvenient to sync
> airflow installation across multiple hosts though.
> {code}
> Jan 09 22:46:59 scheduler_host airflow[18882]: [2020-01-09 22:46:59,763]
> \{celery_executor.py:224} ERROR - Error sending Celery task:Timeout, PID:
> 27724
> Jan 09 22:46:59 scheduler_host airflow[18882]: Celery Task ID:
> ('example_daily', 'example_sensor1', datetime.datetime(2020, 1, 9, 0, 0,
> tzinfo=), 1)
> Jan 09 22:46:59 scheduler_host airflow[18882]: Traceback (most recent call
> last):
> Jan 09 22:46:59 scheduler_host airflow[18882]: File
> "/mnt/nfs1/airflow_venv/lib/python3.6/site-packages/kombu/utils/objects.py",
> line 42, in __get__
> Jan 09 22:46:59 scheduler_host airflow[18882]: return
> obj.__dict__[self.__name__]
> Jan 09 22:46:59 scheduler_host airflow[18882]: KeyError: 'amqp'
> Jan 09 22:46:59 scheduler_host airflow[18882]: During handling of the
[GitHub] [airflow] yuqian90 opened a new pull request #7143: [AIRFLOW-6527] Make send_task_to_executor timeout configurable
yuqian90 opened a new pull request #7143: [AIRFLOW-6527] Make send_task_to_executor timeout configurable URL: https://github.com/apache/airflow/pull/7143 This change makes the timeout for sending tasks to Celery worker configurable. If people encounter the same Celery timeout error described in AIRFLOW-6527, they may increase `[celery]send_task_timeout` to make the timeout less likely to happen. --- Issue link: WILL BE INSERTED BY [boring-cyborg](https://github.com/kaxil/boring-cyborg) - [ ] Description above provides context of the change - [ ] Commit message/PR title starts with `[AIRFLOW-]`. AIRFLOW- = JIRA ID* - [ ] Unit tests coverage for changes (not needed for documentation changes) - [ ] Commits follow "[How to write a good git commit message](http://chris.beams.io/posts/git-commit/)" - [ ] Relevant documentation is updated including usage instructions. - [ ] I will engage committers as explained in [Contribution Workflow Example](https://github.com/apache/airflow/blob/master/CONTRIBUTING.rst#contribution-workflow-example). * For document-only changes commit message can start with `[AIRFLOW-]`. --- In case of fundamental code change, Airflow Improvement Proposal ([AIP](https://cwiki.apache.org/confluence/display/AIRFLOW/Airflow+Improvements+Proposals)) is needed. In case of a new dependency, check compliance with the [ASF 3rd Party License Policy](https://www.apache.org/legal/resolved.html#category-x). In case of backwards incompatible changes please leave a note in [UPDATING.md](https://github.com/apache/airflow/blob/master/UPDATING.md). Read the [Pull Request Guidelines](https://github.com/apache/airflow/blob/master/CONTRIBUTING.rst#pull-request-guidelines) for more information. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [airflow] yuqian90 commented on issue #7139: [AIRFLOW-2279] Clear tasks across DAGs if marked by ExternalTaskMarker
yuqian90 commented on issue #7139: [AIRFLOW-2279] Clear tasks across DAGs if marked by ExternalTaskMarker URL: https://github.com/apache/airflow/pull/7139#issuecomment-573390871 @kaxil, this is the same change that you already merged. Since it's a bit of work to merge against v1-10-test, I put up this PR to do that. (e.g. www vs www_rbac are two separate packages, and there's a test which counts the number of example DAGs that need to be updated). However, I noticed some tests time out. It doesn't look related to my change, e.g. test_cli.py timesout. test_external_task_sensors.py times out in one of the test profiles (Tests sqlite python 3.5), again I don't think it's due to my change. Is there some known issue with the tests in v1-10-test? The same tests in master seem to work just fine. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [airflow] mik-laj commented on a change in pull request #6993: [AIRFLOW-4502] new cli command - task_states_for_dag_run
mik-laj commented on a change in pull request #6993: [AIRFLOW-4502] new cli
command - task_states_for_dag_run
URL: https://github.com/apache/airflow/pull/6993#discussion_r365563318
##
File path: tests/www/test_views.py
##
@@ -2200,6 +2200,7 @@ def test_action_logging_post(self):
only_failed="false",
)
resp = self.client.post("clear", data=form)
+# sometimes fail due to shared db?
Review comment:
Can you remove this commend? Flaky tests are a problem and we try to limit
it in many ways.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [airflow] mik-laj edited a comment on issue #7142: [AIRFLOW-6539][AIP-21] Move Apache classes to providers.apache package
mik-laj edited a comment on issue #7142: [AIRFLOW-6539][AIP-21] Move Apache classes to providers.apache package URL: https://github.com/apache/airflow/pull/7142#issuecomment-573389953 Can you check if full compatibility has been maintained? For example: ``` # pylint: disable=unused-import from airflow.providers.apache.pinot.hooks.pinot import PinotAdminHook, PinotDbApiHook # noqa ``` If `airflow.providers.apache.pinot.hooks.pinot` package contains two classess, then two classes should be imported and two classes should be added in `test_core_to_contrib.py` file. The selection of classes was manual, because I did not want to maintain compatibility with BaseHook or other not important stuff. It's not WI now. I waited for last Travis's build. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [airflow] mik-laj edited a comment on issue #7142: [AIRFLOW-6539][AIP-21] Move Apache classes to providers.apache package
mik-laj edited a comment on issue #7142: [AIRFLOW-6539][AIP-21] Move Apache classes to providers.apache package URL: https://github.com/apache/airflow/pull/7142#issuecomment-573389953 Can you check if full compatibility has been maintained? For example: ``` # pylint: disable=unused-import from airflow.providers.apache.pinot.hooks.pinot import PinotAdminHook, PinotDbApiHook # noqa ``` If `airflow.providers.apache.pinot.hooks.pinot` package contains two classess, then two classes should be imported and two classes should be added in `test_core_to_contrib.py` file. The selection of classes was manual, because I did not want to maintain compatibility with BaseHook or other not important stuff. It's not WIP.. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [airflow] mik-laj edited a comment on issue #7142: [AIRFLOW-6539][AIP-21] Move Apache classes to providers.apache package
mik-laj edited a comment on issue #7142: [AIRFLOW-6539][AIP-21] Move Apache classes to providers.apache package URL: https://github.com/apache/airflow/pull/7142#issuecomment-573389953 Can you check if full compatibility has been maintained? For example: ``` # pylint: disable=unused-import from airflow.providers.apache.pinot.hooks.pinot import PinotAdminHook, PinotDbApiHook # noqa ``` If `airflow.providers.apache.pinot.hooks.pinot` package contains two classess, then two classes should be imported and two classes should be added in `test_core_to_contrib.py` file. The selection of classes was manual, because I did not want to maintain compatibility with BaseHook or other not important stuff. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [airflow] mik-laj commented on issue #7142: [AIRFLOW-6539][AIP-21][WIP] Move Apache classes to providers.apache package
mik-laj commented on issue #7142: [AIRFLOW-6539][AIP-21][WIP] Move Apache classes to providers.apache package URL: https://github.com/apache/airflow/pull/7142#issuecomment-573389953 Can you check if full compatibility has been maintained? For example: ``` # pylint: disable=unused-import from airflow.providers.apache.pinot.hooks.pinot import PinotAdminHook, PinotDbApiHook # noqa ``` If `airflow.providers.apache.pinot.hooks.pinot` package contains two classess, then two classes should be imported. The selection of classes was manual, because I did not want to maintain compatibility with BaseHook or other not important stuff. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [airflow] potiuk commented on issue #6900: [AIRFLOW-6346] Enhance dag default_view and orientation
potiuk commented on issue #6900: [AIRFLOW-6346] Enhance dag default_view and orientation URL: https://github.com/apache/airflow/pull/6900#issuecomment-573388572 I am not as good/knowledgable in the UI/Flask part. MAybe someone else can review this one? This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [airflow] potiuk commented on issue #6696: [AIRFLOW-6128] Simplify plugins_manager and webserver plugin code
potiuk commented on issue #6696: [AIRFLOW-6128] Simplify plugins_manager and webserver plugin code URL: https://github.com/apache/airflow/pull/6696#issuecomment-573388358 Thanks @feluelle ! I will come back to it soon-ish (or late-is maybe) This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [airflow] potiuk edited a comment on issue #6696: [AIRFLOW-6128] Simplify plugins_manager and webserver plugin code
potiuk edited a comment on issue #6696: [AIRFLOW-6128] Simplify plugins_manager and webserver plugin code URL: https://github.com/apache/airflow/pull/6696#issuecomment-573388358 Thanks @feluelle ! I will come back to it soon-ish (or late-ish maybe) This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [airflow] potiuk commented on a change in pull request #7088: [WIP][AIRFLOW-6494] SSH host_proxy has to be fresh
potiuk commented on a change in pull request #7088: [WIP][AIRFLOW-6494] SSH host_proxy has to be fresh URL: https://github.com/apache/airflow/pull/7088#discussion_r365561814 ## File path: airflow/contrib/hooks/ssh_hook.py ## @@ -90,14 +90,10 @@ def __init__(self, # Use connection to override defaults if self.ssh_conn_id is not None: conn = self.get_connection(self.ssh_conn_id) -if self.username is None: -self.username = conn.login -if self.password is None: -self.password = conn.password -if self.remote_host is None: -self.remote_host = conn.host -if self.port is None: -self.port = conn.port +self.username = conn.login Review comment: I think this should not be changed (and BTW the comment above is wrong). It's the username/password/host/port if specified override the connection, not the other way round. And that's intended behaviour. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [airflow] potiuk commented on a change in pull request #7128: [AIRFLOW-6529] Pickle error occurs when the scheduler tries to run on macOS.
potiuk commented on a change in pull request #7128: [AIRFLOW-6529] Pickle error
occurs when the scheduler tries to run on macOS.
URL: https://github.com/apache/airflow/pull/7128#discussion_r365559172
##
File path: tests/test_utils/mock_executor.py
##
@@ -36,10 +36,17 @@ def __init__(self, do_update=True, *args, **kwargs):
self.history = []
# All the tasks, in a stable sort order
self.sorted_tasks = []
-self.mock_task_results = defaultdict(lambda: State.SUCCESS)
+
+# If multiprocessing runs in spawn mode,
+# arguments are to be pickled but lambda is not picclable.
+# So we should pass self.success instead of lambda.
Review comment:
I was wondering if we could add an automated test to make sure it all stays
picklable.
We could have a test that executes all the multiprocessing-relevant code
from those Airflow classes and runs them in "spawn" mode also on Linux.
https://docs.python.org/3/library/multiprocessing.html#contexts-and-start-methods
I think the easiest way we could do is to simply add a dedicated environment
variable in the airflow startup code (and call `mp.set_start_method('spawn')`
if the variable is set) and process one of the example Dags with that
environment variable set.
That would be a nice protection for the future.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [airflow] potiuk commented on issue #7142: [AIRFLOW-6539][AIP-21][WIP] Move Apache classes to providers.apache package
potiuk commented on issue #7142: [AIRFLOW-6539][AIP-21][WIP] Move Apache classes to providers.apache package URL: https://github.com/apache/airflow/pull/7142#issuecomment-573383213 Is it still [WIP] ? Or should we merge it @mik-laj ? This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [airflow] codecov-io commented on issue #7142: [AIRFLOW-6539][AIP-21][WIP] Move Apache classes to providers.apache package
codecov-io commented on issue #7142: [AIRFLOW-6539][AIP-21][WIP] Move Apache classes to providers.apache package URL: https://github.com/apache/airflow/pull/7142#issuecomment-573381395 # [Codecov](https://codecov.io/gh/apache/airflow/pull/7142?src=pr=h1) Report > Merging [#7142](https://codecov.io/gh/apache/airflow/pull/7142?src=pr=desc) into [master](https://codecov.io/gh/apache/airflow/commit/d654d69d7794a57c5c51685a8a96f1d7c38c2302?src=pr=desc) will **increase** coverage by `0.02%`. > The diff coverage is `85.11%`. [](https://codecov.io/gh/apache/airflow/pull/7142?src=pr=tree) ```diff @@Coverage Diff @@ ## master#7142 +/- ## == + Coverage 85.24% 85.27% +0.02% == Files 683 707 +24 Lines 3915539251 +96 == + Hits3337833470 +92 - Misses 5777 5781 +4 ``` | [Impacted Files](https://codecov.io/gh/apache/airflow/pull/7142?src=pr=tree) | Coverage Δ | | |---|---|---| | [.../providers/apache/spark/hooks/spark\_jdbc\_script.py](https://codecov.io/gh/apache/airflow/pull/7142/diff?src=pr=tree#diff-YWlyZmxvdy9wcm92aWRlcnMvYXBhY2hlL3NwYXJrL2hvb2tzL3NwYXJrX2pkYmNfc2NyaXB0LnB5) | `0% <ø> (ø)` | | | [airflow/macros/hive.py](https://codecov.io/gh/apache/airflow/pull/7142/diff?src=pr=tree#diff-YWlyZmxvdy9tYWNyb3MvaGl2ZS5weQ==) | `38.7% <0%> (ø)` | :arrow_up: | | [airflow/models/connection.py](https://codecov.io/gh/apache/airflow/pull/7142/diff?src=pr=tree#diff-YWlyZmxvdy9tb2RlbHMvY29ubmVjdGlvbi5weQ==) | `68.78% <0%> (ø)` | :arrow_up: | | [airflow/contrib/operators/vertica\_to\_hive.py](https://codecov.io/gh/apache/airflow/pull/7142/diff?src=pr=tree#diff-YWlyZmxvdy9jb250cmliL29wZXJhdG9ycy92ZXJ0aWNhX3RvX2hpdmUucHk=) | `0% <0%> (ø)` | :arrow_up: | | [airflow/contrib/operators/spark\_submit\_operator.py](https://codecov.io/gh/apache/airflow/pull/7142/diff?src=pr=tree#diff-YWlyZmxvdy9jb250cmliL29wZXJhdG9ycy9zcGFya19zdWJtaXRfb3BlcmF0b3IucHk=) | `100% <100%> (+7.14%)` | :arrow_up: | | [airflow/hooks/hive\_hooks.py](https://codecov.io/gh/apache/airflow/pull/7142/diff?src=pr=tree#diff-YWlyZmxvdy9ob29rcy9oaXZlX2hvb2tzLnB5) | `100% <100%> (+22.44%)` | :arrow_up: | | [...flow/providers/apache/hive/operators/hive\_stats.py](https://codecov.io/gh/apache/airflow/pull/7142/diff?src=pr=tree#diff-YWlyZmxvdy9wcm92aWRlcnMvYXBhY2hlL2hpdmUvb3BlcmF0b3JzL2hpdmVfc3RhdHMucHk=) | `100% <100%> (ø)` | | | [airflow/operators/s3\_to\_hive\_operator.py](https://codecov.io/gh/apache/airflow/pull/7142/diff?src=pr=tree#diff-YWlyZmxvdy9vcGVyYXRvcnMvczNfdG9faGl2ZV9vcGVyYXRvci5weQ==) | `93.96% <100%> (ø)` | :arrow_up: | | [airflow/operators/hive\_stats\_operator.py](https://codecov.io/gh/apache/airflow/pull/7142/diff?src=pr=tree#diff-YWlyZmxvdy9vcGVyYXRvcnMvaGl2ZV9zdGF0c19vcGVyYXRvci5weQ==) | `100% <100%> (ø)` | :arrow_up: | | [airflow/sensors/web\_hdfs\_sensor.py](https://codecov.io/gh/apache/airflow/pull/7142/diff?src=pr=tree#diff-YWlyZmxvdy9zZW5zb3JzL3dlYl9oZGZzX3NlbnNvci5weQ==) | `100% <100%> (+50%)` | :arrow_up: | | ... and [77 more](https://codecov.io/gh/apache/airflow/pull/7142/diff?src=pr=tree-more) | | -- [Continue to review full report at Codecov](https://codecov.io/gh/apache/airflow/pull/7142?src=pr=continue). > **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta) > `Δ = absolute (impact)`, `ø = not affected`, `? = missing data` > Powered by [Codecov](https://codecov.io/gh/apache/airflow/pull/7142?src=pr=footer). Last update [d654d69...5be5384](https://codecov.io/gh/apache/airflow/pull/7142?src=pr=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments). This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [airflow] codecov-io edited a comment on issue #7142: [AIRFLOW-6539][AIP-21][WIP] Move Apache classes to providers.apache package
codecov-io edited a comment on issue #7142: [AIRFLOW-6539][AIP-21][WIP] Move Apache classes to providers.apache package URL: https://github.com/apache/airflow/pull/7142#issuecomment-573381395 # [Codecov](https://codecov.io/gh/apache/airflow/pull/7142?src=pr=h1) Report > Merging [#7142](https://codecov.io/gh/apache/airflow/pull/7142?src=pr=desc) into [master](https://codecov.io/gh/apache/airflow/commit/d654d69d7794a57c5c51685a8a96f1d7c38c2302?src=pr=desc) will **increase** coverage by `0.02%`. > The diff coverage is `85.11%`. [](https://codecov.io/gh/apache/airflow/pull/7142?src=pr=tree) ```diff @@Coverage Diff @@ ## master#7142 +/- ## == + Coverage 85.24% 85.27% +0.02% == Files 683 707 +24 Lines 3915539251 +96 == + Hits3337833470 +92 - Misses 5777 5781 +4 ``` | [Impacted Files](https://codecov.io/gh/apache/airflow/pull/7142?src=pr=tree) | Coverage Δ | | |---|---|---| | [.../providers/apache/spark/hooks/spark\_jdbc\_script.py](https://codecov.io/gh/apache/airflow/pull/7142/diff?src=pr=tree#diff-YWlyZmxvdy9wcm92aWRlcnMvYXBhY2hlL3NwYXJrL2hvb2tzL3NwYXJrX2pkYmNfc2NyaXB0LnB5) | `0% <ø> (ø)` | | | [airflow/macros/hive.py](https://codecov.io/gh/apache/airflow/pull/7142/diff?src=pr=tree#diff-YWlyZmxvdy9tYWNyb3MvaGl2ZS5weQ==) | `38.7% <0%> (ø)` | :arrow_up: | | [airflow/models/connection.py](https://codecov.io/gh/apache/airflow/pull/7142/diff?src=pr=tree#diff-YWlyZmxvdy9tb2RlbHMvY29ubmVjdGlvbi5weQ==) | `68.78% <0%> (ø)` | :arrow_up: | | [airflow/contrib/operators/vertica\_to\_hive.py](https://codecov.io/gh/apache/airflow/pull/7142/diff?src=pr=tree#diff-YWlyZmxvdy9jb250cmliL29wZXJhdG9ycy92ZXJ0aWNhX3RvX2hpdmUucHk=) | `0% <0%> (ø)` | :arrow_up: | | [airflow/contrib/operators/spark\_submit\_operator.py](https://codecov.io/gh/apache/airflow/pull/7142/diff?src=pr=tree#diff-YWlyZmxvdy9jb250cmliL29wZXJhdG9ycy9zcGFya19zdWJtaXRfb3BlcmF0b3IucHk=) | `100% <100%> (+7.14%)` | :arrow_up: | | [airflow/hooks/hive\_hooks.py](https://codecov.io/gh/apache/airflow/pull/7142/diff?src=pr=tree#diff-YWlyZmxvdy9ob29rcy9oaXZlX2hvb2tzLnB5) | `100% <100%> (+22.44%)` | :arrow_up: | | [...flow/providers/apache/hive/operators/hive\_stats.py](https://codecov.io/gh/apache/airflow/pull/7142/diff?src=pr=tree#diff-YWlyZmxvdy9wcm92aWRlcnMvYXBhY2hlL2hpdmUvb3BlcmF0b3JzL2hpdmVfc3RhdHMucHk=) | `100% <100%> (ø)` | | | [airflow/operators/s3\_to\_hive\_operator.py](https://codecov.io/gh/apache/airflow/pull/7142/diff?src=pr=tree#diff-YWlyZmxvdy9vcGVyYXRvcnMvczNfdG9faGl2ZV9vcGVyYXRvci5weQ==) | `93.96% <100%> (ø)` | :arrow_up: | | [airflow/operators/hive\_stats\_operator.py](https://codecov.io/gh/apache/airflow/pull/7142/diff?src=pr=tree#diff-YWlyZmxvdy9vcGVyYXRvcnMvaGl2ZV9zdGF0c19vcGVyYXRvci5weQ==) | `100% <100%> (ø)` | :arrow_up: | | [airflow/sensors/web\_hdfs\_sensor.py](https://codecov.io/gh/apache/airflow/pull/7142/diff?src=pr=tree#diff-YWlyZmxvdy9zZW5zb3JzL3dlYl9oZGZzX3NlbnNvci5weQ==) | `100% <100%> (+50%)` | :arrow_up: | | ... and [77 more](https://codecov.io/gh/apache/airflow/pull/7142/diff?src=pr=tree-more) | | -- [Continue to review full report at Codecov](https://codecov.io/gh/apache/airflow/pull/7142?src=pr=continue). > **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta) > `Δ = absolute (impact)`, `ø = not affected`, `? = missing data` > Powered by [Codecov](https://codecov.io/gh/apache/airflow/pull/7142?src=pr=footer). Last update [d654d69...5be5384](https://codecov.io/gh/apache/airflow/pull/7142?src=pr=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments). This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [airflow] codecov-io edited a comment on issue #7142: [AIRFLOW-6539][AIP-21][WIP] Move Apache classes to providers.apache package
codecov-io edited a comment on issue #7142: [AIRFLOW-6539][AIP-21][WIP] Move Apache classes to providers.apache package URL: https://github.com/apache/airflow/pull/7142#issuecomment-573381395 # [Codecov](https://codecov.io/gh/apache/airflow/pull/7142?src=pr=h1) Report > Merging [#7142](https://codecov.io/gh/apache/airflow/pull/7142?src=pr=desc) into [master](https://codecov.io/gh/apache/airflow/commit/d654d69d7794a57c5c51685a8a96f1d7c38c2302?src=pr=desc) will **increase** coverage by `0.02%`. > The diff coverage is `85.11%`. [](https://codecov.io/gh/apache/airflow/pull/7142?src=pr=tree) ```diff @@Coverage Diff @@ ## master#7142 +/- ## == + Coverage 85.24% 85.27% +0.02% == Files 683 707 +24 Lines 3915539251 +96 == + Hits3337833470 +92 - Misses 5777 5781 +4 ``` | [Impacted Files](https://codecov.io/gh/apache/airflow/pull/7142?src=pr=tree) | Coverage Δ | | |---|---|---| | [.../providers/apache/spark/hooks/spark\_jdbc\_script.py](https://codecov.io/gh/apache/airflow/pull/7142/diff?src=pr=tree#diff-YWlyZmxvdy9wcm92aWRlcnMvYXBhY2hlL3NwYXJrL2hvb2tzL3NwYXJrX2pkYmNfc2NyaXB0LnB5) | `0% <ø> (ø)` | | | [airflow/macros/hive.py](https://codecov.io/gh/apache/airflow/pull/7142/diff?src=pr=tree#diff-YWlyZmxvdy9tYWNyb3MvaGl2ZS5weQ==) | `38.7% <0%> (ø)` | :arrow_up: | | [airflow/models/connection.py](https://codecov.io/gh/apache/airflow/pull/7142/diff?src=pr=tree#diff-YWlyZmxvdy9tb2RlbHMvY29ubmVjdGlvbi5weQ==) | `68.78% <0%> (ø)` | :arrow_up: | | [airflow/contrib/operators/vertica\_to\_hive.py](https://codecov.io/gh/apache/airflow/pull/7142/diff?src=pr=tree#diff-YWlyZmxvdy9jb250cmliL29wZXJhdG9ycy92ZXJ0aWNhX3RvX2hpdmUucHk=) | `0% <0%> (ø)` | :arrow_up: | | [airflow/contrib/operators/spark\_submit\_operator.py](https://codecov.io/gh/apache/airflow/pull/7142/diff?src=pr=tree#diff-YWlyZmxvdy9jb250cmliL29wZXJhdG9ycy9zcGFya19zdWJtaXRfb3BlcmF0b3IucHk=) | `100% <100%> (+7.14%)` | :arrow_up: | | [airflow/hooks/hive\_hooks.py](https://codecov.io/gh/apache/airflow/pull/7142/diff?src=pr=tree#diff-YWlyZmxvdy9ob29rcy9oaXZlX2hvb2tzLnB5) | `100% <100%> (+22.44%)` | :arrow_up: | | [...flow/providers/apache/hive/operators/hive\_stats.py](https://codecov.io/gh/apache/airflow/pull/7142/diff?src=pr=tree#diff-YWlyZmxvdy9wcm92aWRlcnMvYXBhY2hlL2hpdmUvb3BlcmF0b3JzL2hpdmVfc3RhdHMucHk=) | `100% <100%> (ø)` | | | [airflow/operators/s3\_to\_hive\_operator.py](https://codecov.io/gh/apache/airflow/pull/7142/diff?src=pr=tree#diff-YWlyZmxvdy9vcGVyYXRvcnMvczNfdG9faGl2ZV9vcGVyYXRvci5weQ==) | `93.96% <100%> (ø)` | :arrow_up: | | [airflow/operators/hive\_stats\_operator.py](https://codecov.io/gh/apache/airflow/pull/7142/diff?src=pr=tree#diff-YWlyZmxvdy9vcGVyYXRvcnMvaGl2ZV9zdGF0c19vcGVyYXRvci5weQ==) | `100% <100%> (ø)` | :arrow_up: | | [airflow/sensors/web\_hdfs\_sensor.py](https://codecov.io/gh/apache/airflow/pull/7142/diff?src=pr=tree#diff-YWlyZmxvdy9zZW5zb3JzL3dlYl9oZGZzX3NlbnNvci5weQ==) | `100% <100%> (+50%)` | :arrow_up: | | ... and [77 more](https://codecov.io/gh/apache/airflow/pull/7142/diff?src=pr=tree-more) | | -- [Continue to review full report at Codecov](https://codecov.io/gh/apache/airflow/pull/7142?src=pr=continue). > **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta) > `Δ = absolute (impact)`, `ø = not affected`, `? = missing data` > Powered by [Codecov](https://codecov.io/gh/apache/airflow/pull/7142?src=pr=footer). Last update [d654d69...5be5384](https://codecov.io/gh/apache/airflow/pull/7142?src=pr=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments). This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [airflow] codecov-io edited a comment on issue #7142: [AIRFLOW-6539][AIP-21][WIP] Move Apache classes to providers.apache package
codecov-io edited a comment on issue #7142: [AIRFLOW-6539][AIP-21][WIP] Move Apache classes to providers.apache package URL: https://github.com/apache/airflow/pull/7142#issuecomment-573381395 # [Codecov](https://codecov.io/gh/apache/airflow/pull/7142?src=pr=h1) Report > Merging [#7142](https://codecov.io/gh/apache/airflow/pull/7142?src=pr=desc) into [master](https://codecov.io/gh/apache/airflow/commit/d654d69d7794a57c5c51685a8a96f1d7c38c2302?src=pr=desc) will **increase** coverage by `0.02%`. > The diff coverage is `85.11%`. [](https://codecov.io/gh/apache/airflow/pull/7142?src=pr=tree) ```diff @@Coverage Diff @@ ## master#7142 +/- ## == + Coverage 85.24% 85.27% +0.02% == Files 683 707 +24 Lines 3915539251 +96 == + Hits3337833470 +92 - Misses 5777 5781 +4 ``` | [Impacted Files](https://codecov.io/gh/apache/airflow/pull/7142?src=pr=tree) | Coverage Δ | | |---|---|---| | [.../providers/apache/spark/hooks/spark\_jdbc\_script.py](https://codecov.io/gh/apache/airflow/pull/7142/diff?src=pr=tree#diff-YWlyZmxvdy9wcm92aWRlcnMvYXBhY2hlL3NwYXJrL2hvb2tzL3NwYXJrX2pkYmNfc2NyaXB0LnB5) | `0% <ø> (ø)` | | | [airflow/macros/hive.py](https://codecov.io/gh/apache/airflow/pull/7142/diff?src=pr=tree#diff-YWlyZmxvdy9tYWNyb3MvaGl2ZS5weQ==) | `38.7% <0%> (ø)` | :arrow_up: | | [airflow/models/connection.py](https://codecov.io/gh/apache/airflow/pull/7142/diff?src=pr=tree#diff-YWlyZmxvdy9tb2RlbHMvY29ubmVjdGlvbi5weQ==) | `68.78% <0%> (ø)` | :arrow_up: | | [airflow/contrib/operators/vertica\_to\_hive.py](https://codecov.io/gh/apache/airflow/pull/7142/diff?src=pr=tree#diff-YWlyZmxvdy9jb250cmliL29wZXJhdG9ycy92ZXJ0aWNhX3RvX2hpdmUucHk=) | `0% <0%> (ø)` | :arrow_up: | | [airflow/contrib/operators/spark\_submit\_operator.py](https://codecov.io/gh/apache/airflow/pull/7142/diff?src=pr=tree#diff-YWlyZmxvdy9jb250cmliL29wZXJhdG9ycy9zcGFya19zdWJtaXRfb3BlcmF0b3IucHk=) | `100% <100%> (+7.14%)` | :arrow_up: | | [airflow/hooks/hive\_hooks.py](https://codecov.io/gh/apache/airflow/pull/7142/diff?src=pr=tree#diff-YWlyZmxvdy9ob29rcy9oaXZlX2hvb2tzLnB5) | `100% <100%> (+22.44%)` | :arrow_up: | | [...flow/providers/apache/hive/operators/hive\_stats.py](https://codecov.io/gh/apache/airflow/pull/7142/diff?src=pr=tree#diff-YWlyZmxvdy9wcm92aWRlcnMvYXBhY2hlL2hpdmUvb3BlcmF0b3JzL2hpdmVfc3RhdHMucHk=) | `100% <100%> (ø)` | | | [airflow/operators/s3\_to\_hive\_operator.py](https://codecov.io/gh/apache/airflow/pull/7142/diff?src=pr=tree#diff-YWlyZmxvdy9vcGVyYXRvcnMvczNfdG9faGl2ZV9vcGVyYXRvci5weQ==) | `93.96% <100%> (ø)` | :arrow_up: | | [airflow/operators/hive\_stats\_operator.py](https://codecov.io/gh/apache/airflow/pull/7142/diff?src=pr=tree#diff-YWlyZmxvdy9vcGVyYXRvcnMvaGl2ZV9zdGF0c19vcGVyYXRvci5weQ==) | `100% <100%> (ø)` | :arrow_up: | | [airflow/sensors/web\_hdfs\_sensor.py](https://codecov.io/gh/apache/airflow/pull/7142/diff?src=pr=tree#diff-YWlyZmxvdy9zZW5zb3JzL3dlYl9oZGZzX3NlbnNvci5weQ==) | `100% <100%> (+50%)` | :arrow_up: | | ... and [77 more](https://codecov.io/gh/apache/airflow/pull/7142/diff?src=pr=tree-more) | | -- [Continue to review full report at Codecov](https://codecov.io/gh/apache/airflow/pull/7142?src=pr=continue). > **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta) > `Δ = absolute (impact)`, `ø = not affected`, `? = missing data` > Powered by [Codecov](https://codecov.io/gh/apache/airflow/pull/7142?src=pr=footer). Last update [d654d69...5be5384](https://codecov.io/gh/apache/airflow/pull/7142?src=pr=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments). This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [airflow] sarutak commented on a change in pull request #7128: [AIRFLOW-6529] Pickle error occurs when the scheduler tries to run on macOS.
sarutak commented on a change in pull request #7128: [AIRFLOW-6529] Pickle error occurs when the scheduler tries to run on macOS. URL: https://github.com/apache/airflow/pull/7128#discussion_r36136 ## File path: tests/test_utils/mock_executor.py ## @@ -36,10 +36,17 @@ def __init__(self, do_update=True, *args, **kwargs): self.history = [] # All the tasks, in a stable sort order self.sorted_tasks = [] -self.mock_task_results = defaultdict(lambda: State.SUCCESS) + +# If multiprocessing runs in spawn mode, +# arguments are to be pickled but lambda is not picclable. +# So we should pass self.success instead of lambda. Review comment: If we run `TestSchedulerJob.test_retry_still_in_executor` on macOS, we will get following stack-trace. ``` File "/Users/kou/work/oss/airflow/airflow/jobs/scheduler_job.py", line 1499, in _execute self._execute_helper() File "/Users/kou/work/oss/airflow/airflow/jobs/scheduler_job.py", line 1532, in _execute_helper self.processor_agent.start() File "/Users/kou/work/oss/airflow/airflow/utils/dag_processing.py", line 353, in start self._process.start() File "/opt/python3/3.8.1/lib/python3.8/multiprocessing/process.py", line 121, in start self._popen = self._Popen(self) File "/opt/python3/3.8.1/lib/python3.8/multiprocessing/context.py", line 224, in _Popen return _default_context.get_context().Process._Popen(process_obj) File "/opt/python3/3.8.1/lib/python3.8/multiprocessing/context.py", line 283, in _Popen return Popen(process_obj) File "/opt/python3/3.8.1/lib/python3.8/multiprocessing/popen_spawn_posix.py", line 32, in __init__ super().__init__(process_obj) File "/opt/python3/3.8.1/lib/python3.8/multiprocessing/popen_fork.py", line 19, in __init__ self._launch(process_obj) File "/opt/python3/3.8.1/lib/python3.8/multiprocessing/popen_spawn_posix.py", line 47, in _launch reduction.dump(process_obj, fp) File "/opt/python3/3.8.1/lib/python3.8/multiprocessing/reduction.py", line 60, in dump ForkingPickler(file, protocol).dump(obj) AttributeError: Can't pickle local object 'MockExecutor.__init__..' ``` This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [airflow] mik-laj commented on issue #7131: [AIRFLOW-6534] - BigQuery - move methods from BigQueryBaseCursor to BigQueryHook
mik-laj commented on issue #7131: [AIRFLOW-6534] - BigQuery - move methods from BigQueryBaseCursor to BigQueryHook URL: https://github.com/apache/airflow/pull/7131#issuecomment-573368833 Very good work! This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Commented] (AIRFLOW-1423) Enhance scheduler logs to better explain DAG runs decisions
[ https://issues.apache.org/jira/browse/AIRFLOW-1423?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17013626#comment-17013626 ] ASF GitHub Bot commented on AIRFLOW-1423: - stale[bot] commented on pull request #2455: [AIRFLOW-1423] Add logs to the scheduler DAG run decision logic URL: https://github.com/apache/airflow/pull/2455 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org > Enhance scheduler logs to better explain DAG runs decisions > --- > > Key: AIRFLOW-1423 > URL: https://issues.apache.org/jira/browse/AIRFLOW-1423 > Project: Apache Airflow > Issue Type: Improvement > Components: scheduler >Reporter: Ultrabug >Priority: Major > Attachments: add_scheduler_logs.patch > > > One of the most frustrating topic for users is usually related to their > understanding on the scheduler decisions about running a DAG or not. > It would be wise to add more logs in the jobs creation decision so that it > gets more clear whether a DAG is run or not and why. > This patch adds such simple and useful logs. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[GitHub] [airflow] stale[bot] closed pull request #2455: [AIRFLOW-1423] Add logs to the scheduler DAG run decision logic
stale[bot] closed pull request #2455: [AIRFLOW-1423] Add logs to the scheduler DAG run decision logic URL: https://github.com/apache/airflow/pull/2455 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [airflow] mik-laj commented on issue #7131: [AIRFLOW-6534] - BigQuery - move methods from BigQueryBaseCursor to BigQueryHook
mik-laj commented on issue #7131: [AIRFLOW-6534] - BigQuery - move methods from BigQueryBaseCursor to BigQueryHook URL: https://github.com/apache/airflow/pull/7131#issuecomment-573368366 Can you also move test methods to `TestBigQueryHookMethods` class? This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Commented] (AIRFLOW-6539) Move Apache classes to providers.apache package
[ https://issues.apache.org/jira/browse/AIRFLOW-6539?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17013606#comment-17013606 ] ASF GitHub Bot commented on AIRFLOW-6539: - mik-laj commented on pull request #7142: [AIRFLOW-6539][AIP-21] Move Apache classes to providers.apache package URL: https://github.com/apache/airflow/pull/7142 Hello, I know that this is a very large PR, but unfortunately, it is not possible to divide it, because these classes are very close together on the lists in `test_core_to_contrib.py` file, documentation and others. If we don't do it in bulk, we will constantly fight merge conflicts and AIP-21 will not be finalized in the next six months (100+ integration * 1.5 day =150 days, 150 days * (7/ 5) [exclude weekend] = 210 days, 210 days = 6.89 month = 0.57 year). Best regards, Kamil --- Issue link: WILL BE INSERTED BY [boring-cyborg](https://github.com/kaxil/boring-cyborg) - [X] Description above provides context of the change - [X] Commit message/PR title starts with `[AIRFLOW-]`. AIRFLOW- = JIRA ID* - [X] Unit tests coverage for changes (not needed for documentation changes) - [X] Commits follow "[How to write a good git commit message](http://chris.beams.io/posts/git-commit/)" - [X] Relevant documentation is updated including usage instructions. - [X] I will engage committers as explained in [Contribution Workflow Example](https://github.com/apache/airflow/blob/master/CONTRIBUTING.rst#contribution-workflow-example). * For document-only changes commit message can start with `[AIRFLOW-]`. --- In case of fundamental code change, Airflow Improvement Proposal ([AIP](https://cwiki.apache.org/confluence/display/AIRFLOW/Airflow+Improvements+Proposals)) is needed. In case of a new dependency, check compliance with the [ASF 3rd Party License Policy](https://www.apache.org/legal/resolved.html#category-x). In case of backwards incompatible changes please leave a note in [UPDATING.md](https://github.com/apache/airflow/blob/master/UPDATING.md). Read the [Pull Request Guidelines](https://github.com/apache/airflow/blob/master/CONTRIBUTING.rst#pull-request-guidelines) for more information. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org > Move Apache classes to providers.apache package > --- > > Key: AIRFLOW-6539 > URL: https://issues.apache.org/jira/browse/AIRFLOW-6539 > Project: Apache Airflow > Issue Type: Improvement > Components: hooks, operators >Affects Versions: 1.10.7 >Reporter: Kamil Bregula >Priority: Major > > More information: > https://cwiki.apache.org/confluence/display/AIRFLOW/AIP-21%3A+Changes+in+import+paths -- This message was sent by Atlassian Jira (v8.3.4#803005)
[GitHub] [airflow] mik-laj opened a new pull request #7142: [AIRFLOW-6539][AIP-21] Move Apache classes to providers.apache package
mik-laj opened a new pull request #7142: [AIRFLOW-6539][AIP-21] Move Apache classes to providers.apache package URL: https://github.com/apache/airflow/pull/7142 Hello, I know that this is a very large PR, but unfortunately, it is not possible to divide it, because these classes are very close together on the lists in `test_core_to_contrib.py` file, documentation and others. If we don't do it in bulk, we will constantly fight merge conflicts and AIP-21 will not be finalized in the next six months (100+ integration * 1.5 day =150 days, 150 days * (7/ 5) [exclude weekend] = 210 days, 210 days = 6.89 month = 0.57 year). Best regards, Kamil --- Issue link: WILL BE INSERTED BY [boring-cyborg](https://github.com/kaxil/boring-cyborg) - [X] Description above provides context of the change - [X] Commit message/PR title starts with `[AIRFLOW-]`. AIRFLOW- = JIRA ID* - [X] Unit tests coverage for changes (not needed for documentation changes) - [X] Commits follow "[How to write a good git commit message](http://chris.beams.io/posts/git-commit/)" - [X] Relevant documentation is updated including usage instructions. - [X] I will engage committers as explained in [Contribution Workflow Example](https://github.com/apache/airflow/blob/master/CONTRIBUTING.rst#contribution-workflow-example). * For document-only changes commit message can start with `[AIRFLOW-]`. --- In case of fundamental code change, Airflow Improvement Proposal ([AIP](https://cwiki.apache.org/confluence/display/AIRFLOW/Airflow+Improvements+Proposals)) is needed. In case of a new dependency, check compliance with the [ASF 3rd Party License Policy](https://www.apache.org/legal/resolved.html#category-x). In case of backwards incompatible changes please leave a note in [UPDATING.md](https://github.com/apache/airflow/blob/master/UPDATING.md). Read the [Pull Request Guidelines](https://github.com/apache/airflow/blob/master/CONTRIBUTING.rst#pull-request-guidelines) for more information. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Created] (AIRFLOW-6539) Move Apache classes to providers.apache package
Kamil Bregula created AIRFLOW-6539: -- Summary: Move Apache classes to providers.apache package Key: AIRFLOW-6539 URL: https://issues.apache.org/jira/browse/AIRFLOW-6539 Project: Apache Airflow Issue Type: Improvement Components: hooks, operators Affects Versions: 1.10.7 Reporter: Kamil Bregula More information: https://cwiki.apache.org/confluence/display/AIRFLOW/AIP-21%3A+Changes+in+import+paths -- This message was sent by Atlassian Jira (v8.3.4#803005)
[GitHub] [airflow] larryzhu2018 opened a new pull request #7141: add log_id to end-of-file mark and also add an index config for logs
larryzhu2018 opened a new pull request #7141: add log_id to end-of-file mark and also add an index config for logs URL: https://github.com/apache/airflow/pull/7141 --- Issue link: WILL BE INSERTED BY [boring-cyborg](https://github.com/kaxil/boring-cyborg) - [ ] Description above provides context of the change The “end of log” marker does not include the aforementioned log_id. The issue is then airflow-web does not know when to stop tailing the logs. - [ ] Commit message/PR title starts with `[AIRFLOW-]`. AIRFLOW- = JIRA ID* - [ ] Unit tests coverage for changes (not needed for documentation changes) - [ ] Commits follow "[How to write a good git commit message](http://chris.beams.io/posts/git-commit/)" - [ ] Relevant documentation is updated including usage instructions. - [ ] I will engage committers as explained in [Contribution Workflow Example](https://github.com/apache/airflow/blob/master/CONTRIBUTING.rst#contribution-workflow-example). * For document-only changes commit message can start with `[AIRFLOW-]`. --- In case of fundamental code change, Airflow Improvement Proposal ([AIP](https://cwiki.apache.org/confluence/display/AIRFLOW/Airflow+Improvements+Proposals)) is needed. In case of a new dependency, check compliance with the [ASF 3rd Party License Policy](https://www.apache.org/legal/resolved.html#category-x). In case of backwards incompatible changes please leave a note in [UPDATING.md](https://github.com/apache/airflow/blob/master/UPDATING.md). Read the [Pull Request Guidelines](https://github.com/apache/airflow/blob/master/CONTRIBUTING.rst#pull-request-guidelines) for more information. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [airflow] boring-cyborg[bot] commented on issue #7141: add log_id to end-of-file mark and also add an index config for logs
boring-cyborg[bot] commented on issue #7141: add log_id to end-of-file mark and also add an index config for logs URL: https://github.com/apache/airflow/pull/7141#issuecomment-573364933 Congratulations on your first Pull Request and welcome to the Apache Airflow community! If you have any issues or are unsure about any anything please check our Contribution Guide (https://github.com/apache/airflow/blob/master/CONTRIBUTING.rst) Here are some useful points: - Pay attention to the quality of your code (flake8, pylint and type annotations). Our [pre-commits]( https://github.com/apache/airflow/blob/master/STATIC_CODE_CHECKS.rst#prerequisites-for-pre-commit-hooks) will help you with that. - In case of a new feature add useful documentation (in docstrings or in `docs/` directory). Adding a new operator? Consider adding an example DAG that shows how users should use it. - Consider using [Breeze environment](https://github.com/apache/airflow/blob/master/BREEZE.rst) for testing locally, it’s a heavy docker but it ships with a working Airflow and a lot of integrations. - Be patient and persistent. It might take some time to get a review or get the final approval from Committers. Apache Airflow is a community-driven project and together we are making it better . In case of doubts contact the developers at: Mailing List: d...@airflow.apache.org Slack: https://apache-airflow-slack.herokuapp.com/ This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [airflow] mik-laj commented on a change in pull request #7130: [AIRFLOW-6533] Add cli commands to manage config
mik-laj commented on a change in pull request #7130: [AIRFLOW-6533] Add cli
commands to manage config
URL: https://github.com/apache/airflow/pull/7130#discussion_r365344763
##
File path: airflow/bin/cli.py
##
@@ -130,6 +130,23 @@ class CLIFactory:
choices=tabulate_formats,
default="fancy_grid"),
+# config
+'section': Arg(
+("--section",),
+help="Section of the config"),
+'option': Arg(
+("--option",),
+help="Option in the given section"),
+'value': Arg(
+("--value",),
+help="Value for the option"),
+'executor': Arg(
+("--executor",),
+help="Use to change the executor"),
+'dagdir': Arg(
+("--dagdir",),
Review comment:
```suggestion
("--subdir",),
```
We are currently using the subdir option for this purpose.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[jira] [Created] (AIRFLOW-6538) prevent autocomplete of username in login UI
t oo created AIRFLOW-6538: - Summary: prevent autocomplete of username in login UI Key: AIRFLOW-6538 URL: https://issues.apache.org/jira/browse/AIRFLOW-6538 Project: Apache Airflow Issue Type: Improvement Components: security, ui Affects Versions: 1.10.7 Reporter: t oo Login page of the UI has autocomplete for username field. This should be disabled for security -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (AIRFLOW-6536) Make "job_id" parameter of the DatabricksRunNowOperator optional
[
https://issues.apache.org/jira/browse/AIRFLOW-6536?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17013583#comment-17013583
]
ASF subversion and git services commented on AIRFLOW-6536:
--
Commit d654d69d7794a57c5c51685a8a96f1d7c38c2302 in airflow's branch
refs/heads/master from Mustafa Gök
[ https://gitbox.apache.org/repos/asf?p=airflow.git;h=d654d69 ]
[AIRFLOW-6536] Make job_id parameter optional (#7136)
It can be passed in the json parameter
> Make "job_id" parameter of the DatabricksRunNowOperator optional
>
>
> Key: AIRFLOW-6536
> URL: https://issues.apache.org/jira/browse/AIRFLOW-6536
> Project: Apache Airflow
> Issue Type: Improvement
> Components: contrib
>Affects Versions: 1.10.7
>Reporter: Mustafa Gök
>Assignee: Mustafa Gök
>Priority: Minor
> Fix For: 2.0.0
>
>
> "job_id" parameter should be optional because it can be passed in json(dict)
> parameter.
> line 317 (example in docstring, but it gives error):
> {code:python}
> notebook_run = DatabricksRunNowOperator(task_id='notebook_run', json=json)
> {code}
> line 458:
> {code:python}
> if job_id is not None:
> self.json['job_id'] = job_id
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Resolved] (AIRFLOW-6536) Make "job_id" parameter of the DatabricksRunNowOperator optional
[
https://issues.apache.org/jira/browse/AIRFLOW-6536?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jarek Potiuk resolved AIRFLOW-6536.
---
Fix Version/s: 2.0.0
Resolution: Fixed
> Make "job_id" parameter of the DatabricksRunNowOperator optional
>
>
> Key: AIRFLOW-6536
> URL: https://issues.apache.org/jira/browse/AIRFLOW-6536
> Project: Apache Airflow
> Issue Type: Improvement
> Components: contrib
>Affects Versions: 1.10.7
>Reporter: Mustafa Gök
>Assignee: Mustafa Gök
>Priority: Minor
> Fix For: 2.0.0
>
>
> "job_id" parameter should be optional because it can be passed in json(dict)
> parameter.
> line 317 (example in docstring, but it gives error):
> {code:python}
> notebook_run = DatabricksRunNowOperator(task_id='notebook_run', json=json)
> {code}
> line 458:
> {code:python}
> if job_id is not None:
> self.json['job_id'] = job_id
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (AIRFLOW-6536) Make "job_id" parameter of the DatabricksRunNowOperator optional
[
https://issues.apache.org/jira/browse/AIRFLOW-6536?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17013582#comment-17013582
]
ASF GitHub Bot commented on AIRFLOW-6536:
-
potiuk commented on pull request #7136: [AIRFLOW-6536] Make job_id parameter
optional
URL: https://github.com/apache/airflow/pull/7136
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
> Make "job_id" parameter of the DatabricksRunNowOperator optional
>
>
> Key: AIRFLOW-6536
> URL: https://issues.apache.org/jira/browse/AIRFLOW-6536
> Project: Apache Airflow
> Issue Type: Improvement
> Components: contrib
>Affects Versions: 1.10.7
>Reporter: Mustafa Gök
>Assignee: Mustafa Gök
>Priority: Minor
>
> "job_id" parameter should be optional because it can be passed in json(dict)
> parameter.
> line 317 (example in docstring, but it gives error):
> {code:python}
> notebook_run = DatabricksRunNowOperator(task_id='notebook_run', json=json)
> {code}
> line 458:
> {code:python}
> if job_id is not None:
> self.json['job_id'] = job_id
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[GitHub] [airflow] potiuk commented on issue #7136: [AIRFLOW-6536] Make job_id parameter optional
potiuk commented on issue #7136: [AIRFLOW-6536] Make job_id parameter optional URL: https://github.com/apache/airflow/pull/7136#issuecomment-573357010 > Actually there are some tests in the test_databricks_operator.py file: > line 278: > `test_init_with_json()` method (already updated in this PR) the job_id is not passed directly, and it is in the json dict in this test. > Does it like the "one with the default param from json" or what kind of test should I implement other than this? > line 267: > `test_init_with_named_parameters()` job_id is not in the json, it passed directly. That's a very good point :). I missed this test entirely. It's good to go ! This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [airflow] potiuk merged pull request #7136: [AIRFLOW-6536] Make job_id parameter optional
potiuk merged pull request #7136: [AIRFLOW-6536] Make job_id parameter optional URL: https://github.com/apache/airflow/pull/7136 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [airflow] mustafagok commented on issue #7136: [AIRFLOW-6536] Make job_id parameter optional
mustafagok commented on issue #7136: [AIRFLOW-6536] Make job_id parameter optional URL: https://github.com/apache/airflow/pull/7136#issuecomment-573355737 Actually there are some tests in the test_databricks_operator.py file: line 278: `test_init_with_json()` method (already updated in this PR) the job_id is not passed directly, and it is in the json dict in this test. Does it like the "one with the default param from json" or what kind of test should I implement other than this? line 267: `test_init_with_named_parameters()` job_id is not in the json, it passed directly. Also, in the all of the other tests except `test_init_with_json()`, job_id parameter passed because it was a required paremeter until now. So, does it like "the other with the param passed directly" or what kind of test should I implement other than this? This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [airflow] davlum edited a comment on issue #6230: [AIRFLOW-5413] Allow K8S worker pod to be configured from JSON/YAML file
davlum edited a comment on issue #6230: [AIRFLOW-5413] Allow K8S worker pod to be configured from JSON/YAML file URL: https://github.com/apache/airflow/pull/6230#issuecomment-573354037 Adding an additional argument won't break backwards compatibility, but it will change some of the previously required arguments to optional, which is a backwards compatible change. I'm open to the separate operator. I'm noticing there is some validation for the fields on the `KubernetesPodOperator` that wouldn't be necessary if `pod_template_file` is passed, so that might motivate a separate operator. Would I also move the `full_pod_spec` arg to that operator as well? For the `executor_config`, should I add support for passing the `PodGenerator` directly in this PR, or make a ticket and new PR? This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [airflow] davlum commented on issue #6230: [AIRFLOW-5413] Allow K8S worker pod to be configured from JSON/YAML file
davlum commented on issue #6230: [AIRFLOW-5413] Allow K8S worker pod to be configured from JSON/YAML file URL: https://github.com/apache/airflow/pull/6230#issuecomment-573354037 I'm easy about the separate operator. I'm noticing there is some validation for the fields on the `KubernetesPodOperator` that wouldn't be necessary if `pod_template_file` is passed, so that might motivate a separate operator. Would I also move the `full_pod_spec` arg to that operator as well? For the `executor_config`, should I add support for passing the `PodGenerator` directly in this PR, or make a ticket and new PR? This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [airflow] davlum commented on a change in pull request #6230: [AIRFLOW-5413] Allow K8S worker pod to be configured from JSON/YAML file
davlum commented on a change in pull request #6230: [AIRFLOW-5413] Allow K8S worker pod to be configured from JSON/YAML file URL: https://github.com/apache/airflow/pull/6230#discussion_r365541701 ## File path: airflow/contrib/operators/kubernetes_pod_operator.py ## @@ -169,8 +171,9 @@ def __init__(self, # pylint: disable=too-many-arguments,too-many-locals dnspolicy: Optional[str] = None, schedulername: Optional[str] = None, full_pod_spec: Optional[k8s.V1Pod] = None, - init_containers: Optional[List[k8s.V1Container]] = None, do_xcom_push: bool = False, + init_containers: Optional[List[k8s.V1Container]] = None, + pod_template_file=None, Review comment: I like this idea. Wouldn't take too much of a change either. I previously had support for passing both strings and paths to `pod_template_file` (simple `os.path.exists`), [but it was discussed](https://github.com/apache/airflow/pull/6230#discussion_r355993872) that it might be easier to just accept paths in the `PodGenerator`. In the case of the `KubernetesPodOperator` using the `template_ext` would motivate passing a string to the `PodGenerator`, but in the case of configuring the pod from the `airflow.cfg`, paths seems to be better. Is the `executor_config` argument templated as well? I do see a use case for supporting both strings and paths, and this is not a totally unheard of pattern. `pandas.read_csv` supports strings or paths for example. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [airflow] codecov-io edited a comment on issue #7136: [AIRFLOW-6536] Make job_id parameter optional
codecov-io edited a comment on issue #7136: [AIRFLOW-6536] Make job_id parameter optional URL: https://github.com/apache/airflow/pull/7136#issuecomment-573326191 # [Codecov](https://codecov.io/gh/apache/airflow/pull/7136?src=pr=h1) Report > Merging [#7136](https://codecov.io/gh/apache/airflow/pull/7136?src=pr=desc) into [master](https://codecov.io/gh/apache/airflow/commit/a676f159d4fb01f6e35baa04efa4914a695cd788?src=pr=desc) will **decrease** coverage by `0.27%`. > The diff coverage is `n/a`. [](https://codecov.io/gh/apache/airflow/pull/7136?src=pr=tree) ```diff @@Coverage Diff @@ ## master#7136 +/- ## == - Coverage 85.24% 84.96% -0.28% == Files 683 683 Lines 3915539155 == - Hits3337833269 -109 - Misses 5777 5886 +109 ``` | [Impacted Files](https://codecov.io/gh/apache/airflow/pull/7136?src=pr=tree) | Coverage Δ | | |---|---|---| | [airflow/contrib/operators/databricks\_operator.py](https://codecov.io/gh/apache/airflow/pull/7136/diff?src=pr=tree#diff-YWlyZmxvdy9jb250cmliL29wZXJhdG9ycy9kYXRhYnJpY2tzX29wZXJhdG9yLnB5) | `92.24% <ø> (ø)` | :arrow_up: | | [airflow/kubernetes/volume\_mount.py](https://codecov.io/gh/apache/airflow/pull/7136/diff?src=pr=tree#diff-YWlyZmxvdy9rdWJlcm5ldGVzL3ZvbHVtZV9tb3VudC5weQ==) | `44.44% <0%> (-55.56%)` | :arrow_down: | | [airflow/kubernetes/volume.py](https://codecov.io/gh/apache/airflow/pull/7136/diff?src=pr=tree#diff-YWlyZmxvdy9rdWJlcm5ldGVzL3ZvbHVtZS5weQ==) | `52.94% <0%> (-47.06%)` | :arrow_down: | | [airflow/kubernetes/pod\_launcher.py](https://codecov.io/gh/apache/airflow/pull/7136/diff?src=pr=tree#diff-YWlyZmxvdy9rdWJlcm5ldGVzL3BvZF9sYXVuY2hlci5weQ==) | `45.25% <0%> (-46.72%)` | :arrow_down: | | [airflow/kubernetes/refresh\_config.py](https://codecov.io/gh/apache/airflow/pull/7136/diff?src=pr=tree#diff-YWlyZmxvdy9rdWJlcm5ldGVzL3JlZnJlc2hfY29uZmlnLnB5) | `50.98% <0%> (-23.53%)` | :arrow_down: | | [...rflow/contrib/operators/kubernetes\_pod\_operator.py](https://codecov.io/gh/apache/airflow/pull/7136/diff?src=pr=tree#diff-YWlyZmxvdy9jb250cmliL29wZXJhdG9ycy9rdWJlcm5ldGVzX3BvZF9vcGVyYXRvci5weQ==) | `78.31% <0%> (-20.49%)` | :arrow_down: | | [airflow/utils/dag\_processing.py](https://codecov.io/gh/apache/airflow/pull/7136/diff?src=pr=tree#diff-YWlyZmxvdy91dGlscy9kYWdfcHJvY2Vzc2luZy5weQ==) | `88.38% <0%> (+0.38%)` | :arrow_up: | -- [Continue to review full report at Codecov](https://codecov.io/gh/apache/airflow/pull/7136?src=pr=continue). > **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta) > `Δ = absolute (impact)`, `ø = not affected`, `? = missing data` > Powered by [Codecov](https://codecov.io/gh/apache/airflow/pull/7136?src=pr=footer). Last update [a676f15...f69ea52](https://codecov.io/gh/apache/airflow/pull/7136?src=pr=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments). This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [airflow] kaxil commented on issue #7137: [AIRFLOW-6393] Ensure rendering of all lineage items and record source
kaxil commented on issue #7137: [AIRFLOW-6393] Ensure rendering of all lineage items and record source URL: https://github.com/apache/airflow/pull/7137#issuecomment-573352528 @bolkedebruin This PR changes no file at all. These changes were already merge by https://github.com/apache/airflow/pull/6953 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Commented] (AIRFLOW-6506) KubernetesPodOperator defaulting do_xcom_push to True after 1.10.7 release
[
https://issues.apache.org/jira/browse/AIRFLOW-6506?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17013566#comment-17013566
]
ASF subversion and git services commented on AIRFLOW-6506:
--
Commit 9659398ce25137c1717b88f46a149eea88ba9ffe in airflow's branch
refs/heads/v1-10-test from Daniel Imberman
[ https://gitbox.apache.org/repos/asf?p=airflow.git;h=9659398 ]
[AIRFLOW-6506] Fix do_xcom_push defaulting to True in KubernetesPodOperator
(#7122)
(cherry-picked from 86f525f6527e45c9ff1b8f844569c4d9d22bccac)
> KubernetesPodOperator defaulting do_xcom_push to True after 1.10.7 release
> --
>
> Key: AIRFLOW-6506
> URL: https://issues.apache.org/jira/browse/AIRFLOW-6506
> Project: Apache Airflow
> Issue Type: Bug
> Components: executor-kubernetes, xcom
>Affects Versions: 1.10.7
>Reporter: Tim Blazina
>Assignee: Daniel Imberman
>Priority: Minor
> Fix For: 1.10.8
>
>
> {color:#1d1c1d}After updating to 1.10.7, I am getting
> {color}
> [2019-12-27 11:37:42,220] \{{logging_mixin.py:112}} INFO - [2019-12-27
> 11:37:42,219] \{{pod_launcher.py:217}} INFO - Running command... cat
> /airflow/xcom/return.json
> [2019-12-27 11:37:42,286] \{{logging_mixin.py:112}} INFO - [2019-12-27
> 11:37:42,286] \{{pod_launcher.py:224}} INFO - cat: can't open
> '/airflow/xcom/return.json': No such file or directory`
> for tasks run with the KubernetesPodOperator. {color:#1d1c1d}I see that there
> is a {color}{{do_xcom_push}}{color:#1d1c1d} attribute for the
> {color}{{KubernetesPodOperator}}{color:#1d1c1d} and if I set this to
> {color}{{False}}{color:#1d1c1d} I do not get this error{color}.
> When discussing issue in slack - [~dimberman] rightly pointed to -[
> https://github.com/apache/airflow/commit/d1664d2e507024489c565808ebe2e5b48a1797ef#diff-f9bc463ffc2cc8fd3967968549c5639fR122|https://github.com/apache/airflow/commit/d1664d2e507024489c565808ebe2e5b48a1797ef#diff-f9bc463ffc2cc8fd3967968549c5639fR122]
> - as the culprit and [~potiuk] commented that "{color:#1d1c1d}There was no
> UPDATING.md note with it, so I missed that there is a breaking change for
> do_xcom_push while cherry-picking{color}"
>
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (AIRFLOW-6504) KubernetesExecutor: Allow specifying configmap for Airflow Local Setting
[ https://issues.apache.org/jira/browse/AIRFLOW-6504?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17013564#comment-17013564 ] ASF subversion and git services commented on AIRFLOW-6504: -- Commit aca3dc2b9fad112dafbf882d4005bbebd6f0eb78 in airflow's branch refs/heads/v1-10-test from Kaxil Naik [ https://gitbox.apache.org/repos/asf?p=airflow.git;h=aca3dc2 ] [AIRFLOW-6504] Allow specifying configmap for Airflow Local Setting (#7097) > KubernetesExecutor: Allow specifying configmap for Airflow Local Setting > > > Key: AIRFLOW-6504 > URL: https://issues.apache.org/jira/browse/AIRFLOW-6504 > Project: Apache Airflow > Issue Type: Improvement > Components: core >Affects Versions: 1.10.7 >Reporter: Kaxil Naik >Assignee: Kaxil Naik >Priority: Major > Fix For: 1.10.8 > > > Currently airflow.cfg file can be passed via ConfigMap to `airflow_configmap` > setting. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (AIRFLOW-6516) Airflow Local Setting ConfigMap overrides airflow.cfg VolumeMount
[ https://issues.apache.org/jira/browse/AIRFLOW-6516?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17013565#comment-17013565 ] ASF subversion and git services commented on AIRFLOW-6516: -- Commit 69f16edeef34332564847f19b61523edbc053363 in airflow's branch refs/heads/v1-10-test from Kaxil Naik [ https://gitbox.apache.org/repos/asf?p=airflow.git;h=69f16ed ] [AIRFLOW-6516] BugFix: airflow.cfg does not exist in Volume Mounts (#7109) (cherry-picked from a0a02dfcd89a147c1b3a63af992bb6acba2b6b9f) > Airflow Local Setting ConfigMap overrides airflow.cfg VolumeMount > - > > Key: AIRFLOW-6516 > URL: https://issues.apache.org/jira/browse/AIRFLOW-6516 > Project: Apache Airflow > Issue Type: Bug > Components: core >Affects Versions: 1.10.8 >Reporter: Kaxil Naik >Assignee: Kaxil Naik >Priority: Major > Fix For: 1.10.8 > > > https://github.com/apache/airflow/commit/4748a518ced4ebd30caec2ac290c782ef023b0d7 > introduced a bug because we were re-assigning the "airflow-config" key in > volume_mounts/ > Because of this airflow.cfg was not mounted and only airflow_local_settings > was mounted if the `airflow_local_settings_configmap` is also set. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (AIRFLOW-6436) Create & Automate docs on Airflow Configs
[ https://issues.apache.org/jira/browse/AIRFLOW-6436?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17013562#comment-17013562 ] ASF subversion and git services commented on AIRFLOW-6436: -- Commit c6c20e188ff6956d412c04414c86f9f660ffd669 in airflow's branch refs/heads/v1-10-test from Kaxil Naik [ https://gitbox.apache.org/repos/asf?p=airflow.git;h=c6c20e1 ] [AIRFLOW-6436] Cleanup for Airflow configs doc generator code (#7036) (cherry-picked from 09c895ee72cc8d2cb2e73ffb28a2a2fe5cb32fef) > Create & Automate docs on Airflow Configs > - > > Key: AIRFLOW-6436 > URL: https://issues.apache.org/jira/browse/AIRFLOW-6436 > Project: Apache Airflow > Issue Type: Improvement > Components: documentation >Affects Versions: 2.0.0, 1.10.7 >Reporter: Kaxil Naik >Assignee: Kaxil Naik >Priority: Critical > Fix For: 2.0.0, 1.10.8 > > > Follow Up of https://issues.apache.org/jira/browse/AIRFLOW-6414 > This PR aims to automate the creation of Config docs and add pre-commit hooks > that would also serve as tests. > Also, add a structure to the documentation by adding the following sections: > * Description > * Example > * Type > * Default > for each config -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (AIRFLOW-6436) Create & Automate docs on Airflow Configs
[ https://issues.apache.org/jira/browse/AIRFLOW-6436?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17013561#comment-17013561 ] ASF subversion and git services commented on AIRFLOW-6436: -- Commit fec6c454e71c146c02b92eebb94667183559276d in airflow's branch refs/heads/v1-10-test from Kaxil Naik [ https://gitbox.apache.org/repos/asf?p=airflow.git;h=fec6c45 ] [AIRFLOW-6436] Add x_frame_enabled config in config.yml (#7024) (cherry picked from commit e7f5c83f4f0646ef6e404084e84a540de72e441c) > Create & Automate docs on Airflow Configs > - > > Key: AIRFLOW-6436 > URL: https://issues.apache.org/jira/browse/AIRFLOW-6436 > Project: Apache Airflow > Issue Type: Improvement > Components: documentation >Affects Versions: 2.0.0, 1.10.7 >Reporter: Kaxil Naik >Assignee: Kaxil Naik >Priority: Critical > Fix For: 2.0.0, 1.10.8 > > > Follow Up of https://issues.apache.org/jira/browse/AIRFLOW-6414 > This PR aims to automate the creation of Config docs and add pre-commit hooks > that would also serve as tests. > Also, add a structure to the documentation by adding the following sections: > * Description > * Example > * Type > * Default > for each config -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (AIRFLOW-6537) Fix backticks in rst file
[ https://issues.apache.org/jira/browse/AIRFLOW-6537?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17013558#comment-17013558 ] ASF subversion and git services commented on AIRFLOW-6537: -- Commit 934a2afcda8f408c61def4fae3f95e1b4b22ff9f in airflow's branch refs/heads/master from Kamil Breguła [ https://gitbox.apache.org/repos/asf?p=airflow.git;h=934a2af ] [AIRFLOW-6537] Fix backticks in rst files (#7140) > Fix backticks in rst file > - > > Key: AIRFLOW-6537 > URL: https://issues.apache.org/jira/browse/AIRFLOW-6537 > Project: Apache Airflow > Issue Type: Improvement > Components: documentation >Affects Versions: 1.10.7 >Reporter: Kamil Bregula >Priority: Major > Fix For: 2.0.0 > > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Resolved] (AIRFLOW-6537) Fix backticks in rst file
[ https://issues.apache.org/jira/browse/AIRFLOW-6537?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jarek Potiuk resolved AIRFLOW-6537. --- Fix Version/s: 2.0.0 Resolution: Fixed > Fix backticks in rst file > - > > Key: AIRFLOW-6537 > URL: https://issues.apache.org/jira/browse/AIRFLOW-6537 > Project: Apache Airflow > Issue Type: Improvement > Components: documentation >Affects Versions: 1.10.7 >Reporter: Kamil Bregula >Priority: Major > Fix For: 2.0.0 > > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[GitHub] [airflow] potiuk commented on issue #7140: [AIRFLOW-6537] Fix backticks in rst files
potiuk commented on issue #7140: [AIRFLOW-6537] Fix backticks in rst files URL: https://github.com/apache/airflow/pull/7140#issuecomment-573344379 Nice! This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [airflow] potiuk merged pull request #7140: [AIRFLOW-6537] Fix backticks in rst files
potiuk merged pull request #7140: [AIRFLOW-6537] Fix backticks in rst files URL: https://github.com/apache/airflow/pull/7140 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Commented] (AIRFLOW-6537) Fix backticks in rst file
[ https://issues.apache.org/jira/browse/AIRFLOW-6537?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17013557#comment-17013557 ] ASF GitHub Bot commented on AIRFLOW-6537: - potiuk commented on pull request #7140: [AIRFLOW-6537] Fix backticks in rst files URL: https://github.com/apache/airflow/pull/7140 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org > Fix backticks in rst file > - > > Key: AIRFLOW-6537 > URL: https://issues.apache.org/jira/browse/AIRFLOW-6537 > Project: Apache Airflow > Issue Type: Improvement > Components: documentation >Affects Versions: 1.10.7 >Reporter: Kamil Bregula >Priority: Major > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[GitHub] [airflow] stale[bot] commented on issue #6007: Improved and Fixed Apache Glue hook/operator/sensor from [AIRFLOW-2310]
stale[bot] commented on issue #6007: Improved and Fixed Apache Glue hook/operator/sensor from [AIRFLOW-2310] URL: https://github.com/apache/airflow/pull/6007#issuecomment-573343782 This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [airflow] codecov-io edited a comment on issue #7038: [AIRFLOW-4495] allow externally triggered dags to run for future exec dates
codecov-io edited a comment on issue #7038: [AIRFLOW-4495] allow externally triggered dags to run for future exec dates URL: https://github.com/apache/airflow/pull/7038#issuecomment-570935774 # [Codecov](https://codecov.io/gh/apache/airflow/pull/7038?src=pr=h1) Report > :exclamation: No coverage uploaded for pull request base (`master@b5bd9ab`). [Click here to learn what that means](https://docs.codecov.io/docs/error-reference#section-missing-base-commit). > The diff coverage is `96.9%`. [](https://codecov.io/gh/apache/airflow/pull/7038?src=pr=tree) ```diff @@Coverage Diff@@ ## master#7038 +/- ## = Coverage ? 84.96% = Files ? 683 Lines ?39159 Branches ?0 = Hits ?33271 Misses? 5888 Partials ?0 ``` | [Impacted Files](https://codecov.io/gh/apache/airflow/pull/7038?src=pr=tree) | Coverage Δ | | |---|---|---| | [...irflow/contrib/operators/emr\_add\_steps\_operator.py](https://codecov.io/gh/apache/airflow/pull/7038/diff?src=pr=tree#diff-YWlyZmxvdy9jb250cmliL29wZXJhdG9ycy9lbXJfYWRkX3N0ZXBzX29wZXJhdG9yLnB5) | `91.42% <ø> (ø)` | | | [airflow/sensors/base\_sensor\_operator.py](https://codecov.io/gh/apache/airflow/pull/7038/diff?src=pr=tree#diff-YWlyZmxvdy9zZW5zb3JzL2Jhc2Vfc2Vuc29yX29wZXJhdG9yLnB5) | `98.61% <ø> (ø)` | | | [airflow/sensors/external\_task\_sensor.py](https://codecov.io/gh/apache/airflow/pull/7038/diff?src=pr=tree#diff-YWlyZmxvdy9zZW5zb3JzL2V4dGVybmFsX3Rhc2tfc2Vuc29yLnB5) | `85.52% <ø> (ø)` | | | [...ib/operators/azure\_container\_instances\_operator.py](https://codecov.io/gh/apache/airflow/pull/7038/diff?src=pr=tree#diff-YWlyZmxvdy9jb250cmliL29wZXJhdG9ycy9henVyZV9jb250YWluZXJfaW5zdGFuY2VzX29wZXJhdG9yLnB5) | `78.14% <ø> (ø)` | | | [airflow/models/baseoperator.py](https://codecov.io/gh/apache/airflow/pull/7038/diff?src=pr=tree#diff-YWlyZmxvdy9tb2RlbHMvYmFzZW9wZXJhdG9yLnB5) | `96.25% <ø> (ø)` | | | [airflow/ti\_deps/deps/prev\_dagrun\_dep.py](https://codecov.io/gh/apache/airflow/pull/7038/diff?src=pr=tree#diff-YWlyZmxvdy90aV9kZXBzL2RlcHMvcHJldl9kYWdydW5fZGVwLnB5) | `80.55% <ø> (ø)` | | | [airflow/jobs/scheduler\_job.py](https://codecov.io/gh/apache/airflow/pull/7038/diff?src=pr=tree#diff-YWlyZmxvdy9qb2JzL3NjaGVkdWxlcl9qb2IucHk=) | `89.29% <0%> (ø)` | | | [airflow/cli/commands/celery\_command.py](https://codecov.io/gh/apache/airflow/pull/7038/diff?src=pr=tree#diff-YWlyZmxvdy9jbGkvY29tbWFuZHMvY2VsZXJ5X2NvbW1hbmQucHk=) | `52.5% <0%> (ø)` | | | [airflow/models/dag.py](https://codecov.io/gh/apache/airflow/pull/7038/diff?src=pr=tree#diff-YWlyZmxvdy9tb2RlbHMvZGFnLnB5) | `90.9% <100%> (ø)` | | | [airflow/jobs/local\_task\_job.py](https://codecov.io/gh/apache/airflow/pull/7038/diff?src=pr=tree#diff-YWlyZmxvdy9qb2JzL2xvY2FsX3Rhc2tfam9iLnB5) | `89.74% <100%> (ø)` | | | ... and [4 more](https://codecov.io/gh/apache/airflow/pull/7038/diff?src=pr=tree-more) | | -- [Continue to review full report at Codecov](https://codecov.io/gh/apache/airflow/pull/7038?src=pr=continue). > **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta) > `Δ = absolute (impact)`, `ø = not affected`, `? = missing data` > Powered by [Codecov](https://codecov.io/gh/apache/airflow/pull/7038?src=pr=footer). Last update [b5bd9ab...a93d509](https://codecov.io/gh/apache/airflow/pull/7038?src=pr=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments). This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [airflow] codecov-io edited a comment on issue #7038: [AIRFLOW-4495] allow externally triggered dags to run for future exec dates
codecov-io edited a comment on issue #7038: [AIRFLOW-4495] allow externally triggered dags to run for future exec dates URL: https://github.com/apache/airflow/pull/7038#issuecomment-570935774 # [Codecov](https://codecov.io/gh/apache/airflow/pull/7038?src=pr=h1) Report > :exclamation: No coverage uploaded for pull request base (`master@b5bd9ab`). [Click here to learn what that means](https://docs.codecov.io/docs/error-reference#section-missing-base-commit). > The diff coverage is `96.9%`. [](https://codecov.io/gh/apache/airflow/pull/7038?src=pr=tree) ```diff @@Coverage Diff@@ ## master#7038 +/- ## = Coverage ? 84.96% = Files ? 683 Lines ?39159 Branches ?0 = Hits ?33271 Misses? 5888 Partials ?0 ``` | [Impacted Files](https://codecov.io/gh/apache/airflow/pull/7038?src=pr=tree) | Coverage Δ | | |---|---|---| | [...irflow/contrib/operators/emr\_add\_steps\_operator.py](https://codecov.io/gh/apache/airflow/pull/7038/diff?src=pr=tree#diff-YWlyZmxvdy9jb250cmliL29wZXJhdG9ycy9lbXJfYWRkX3N0ZXBzX29wZXJhdG9yLnB5) | `91.42% <ø> (ø)` | | | [airflow/sensors/base\_sensor\_operator.py](https://codecov.io/gh/apache/airflow/pull/7038/diff?src=pr=tree#diff-YWlyZmxvdy9zZW5zb3JzL2Jhc2Vfc2Vuc29yX29wZXJhdG9yLnB5) | `98.61% <ø> (ø)` | | | [airflow/sensors/external\_task\_sensor.py](https://codecov.io/gh/apache/airflow/pull/7038/diff?src=pr=tree#diff-YWlyZmxvdy9zZW5zb3JzL2V4dGVybmFsX3Rhc2tfc2Vuc29yLnB5) | `85.52% <ø> (ø)` | | | [...ib/operators/azure\_container\_instances\_operator.py](https://codecov.io/gh/apache/airflow/pull/7038/diff?src=pr=tree#diff-YWlyZmxvdy9jb250cmliL29wZXJhdG9ycy9henVyZV9jb250YWluZXJfaW5zdGFuY2VzX29wZXJhdG9yLnB5) | `78.14% <ø> (ø)` | | | [airflow/models/baseoperator.py](https://codecov.io/gh/apache/airflow/pull/7038/diff?src=pr=tree#diff-YWlyZmxvdy9tb2RlbHMvYmFzZW9wZXJhdG9yLnB5) | `96.25% <ø> (ø)` | | | [airflow/ti\_deps/deps/prev\_dagrun\_dep.py](https://codecov.io/gh/apache/airflow/pull/7038/diff?src=pr=tree#diff-YWlyZmxvdy90aV9kZXBzL2RlcHMvcHJldl9kYWdydW5fZGVwLnB5) | `80.55% <ø> (ø)` | | | [airflow/jobs/scheduler\_job.py](https://codecov.io/gh/apache/airflow/pull/7038/diff?src=pr=tree#diff-YWlyZmxvdy9qb2JzL3NjaGVkdWxlcl9qb2IucHk=) | `89.29% <0%> (ø)` | | | [airflow/cli/commands/celery\_command.py](https://codecov.io/gh/apache/airflow/pull/7038/diff?src=pr=tree#diff-YWlyZmxvdy9jbGkvY29tbWFuZHMvY2VsZXJ5X2NvbW1hbmQucHk=) | `52.5% <0%> (ø)` | | | [airflow/models/dag.py](https://codecov.io/gh/apache/airflow/pull/7038/diff?src=pr=tree#diff-YWlyZmxvdy9tb2RlbHMvZGFnLnB5) | `90.9% <100%> (ø)` | | | [airflow/jobs/local\_task\_job.py](https://codecov.io/gh/apache/airflow/pull/7038/diff?src=pr=tree#diff-YWlyZmxvdy9qb2JzL2xvY2FsX3Rhc2tfam9iLnB5) | `89.74% <100%> (ø)` | | | ... and [4 more](https://codecov.io/gh/apache/airflow/pull/7038/diff?src=pr=tree-more) | | -- [Continue to review full report at Codecov](https://codecov.io/gh/apache/airflow/pull/7038?src=pr=continue). > **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta) > `Δ = absolute (impact)`, `ø = not affected`, `? = missing data` > Powered by [Codecov](https://codecov.io/gh/apache/airflow/pull/7038?src=pr=footer). Last update [b5bd9ab...a93d509](https://codecov.io/gh/apache/airflow/pull/7038?src=pr=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments). This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [airflow] mik-laj opened a new pull request #7140: [AIRFLOW-6537] Fix backticks in rst files
mik-laj opened a new pull request #7140: [AIRFLOW-6537] Fix backticks in rst files URL: https://github.com/apache/airflow/pull/7140 I fixed small bugs in the hook: https://github.com/pre-commit/pygrep-hooks/pull/24 Detect single backticks at the start of the line --- Issue link: WILL BE INSERTED BY [boring-cyborg](https://github.com/kaxil/boring-cyborg) - [X] Description above provides context of the change - [X] Commit message/PR title starts with `[AIRFLOW-]`. AIRFLOW- = JIRA ID* - [X] Unit tests coverage for changes (not needed for documentation changes) - [X] Commits follow "[How to write a good git commit message](http://chris.beams.io/posts/git-commit/)" - [X] Relevant documentation is updated including usage instructions. - [X] I will engage committers as explained in [Contribution Workflow Example](https://github.com/apache/airflow/blob/master/CONTRIBUTING.rst#contribution-workflow-example). * For document-only changes commit message can start with `[AIRFLOW-]`. --- In case of fundamental code change, Airflow Improvement Proposal ([AIP](https://cwiki.apache.org/confluence/display/AIRFLOW/Airflow+Improvements+Proposals)) is needed. In case of a new dependency, check compliance with the [ASF 3rd Party License Policy](https://www.apache.org/legal/resolved.html#category-x). In case of backwards incompatible changes please leave a note in [UPDATING.md](https://github.com/apache/airflow/blob/master/UPDATING.md). Read the [Pull Request Guidelines](https://github.com/apache/airflow/blob/master/CONTRIBUTING.rst#pull-request-guidelines) for more information. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Commented] (AIRFLOW-6537) Fix backticks in rst file
[ https://issues.apache.org/jira/browse/AIRFLOW-6537?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17013550#comment-17013550 ] ASF GitHub Bot commented on AIRFLOW-6537: - mik-laj commented on pull request #7140: [AIRFLOW-6537] Fix backticks in rst files URL: https://github.com/apache/airflow/pull/7140 I fixed small bugs in the hook: https://github.com/pre-commit/pygrep-hooks/pull/24 Detect single backticks at the start of the line --- Issue link: WILL BE INSERTED BY [boring-cyborg](https://github.com/kaxil/boring-cyborg) - [X] Description above provides context of the change - [X] Commit message/PR title starts with `[AIRFLOW-]`. AIRFLOW- = JIRA ID* - [X] Unit tests coverage for changes (not needed for documentation changes) - [X] Commits follow "[How to write a good git commit message](http://chris.beams.io/posts/git-commit/)" - [X] Relevant documentation is updated including usage instructions. - [X] I will engage committers as explained in [Contribution Workflow Example](https://github.com/apache/airflow/blob/master/CONTRIBUTING.rst#contribution-workflow-example). * For document-only changes commit message can start with `[AIRFLOW-]`. --- In case of fundamental code change, Airflow Improvement Proposal ([AIP](https://cwiki.apache.org/confluence/display/AIRFLOW/Airflow+Improvements+Proposals)) is needed. In case of a new dependency, check compliance with the [ASF 3rd Party License Policy](https://www.apache.org/legal/resolved.html#category-x). In case of backwards incompatible changes please leave a note in [UPDATING.md](https://github.com/apache/airflow/blob/master/UPDATING.md). Read the [Pull Request Guidelines](https://github.com/apache/airflow/blob/master/CONTRIBUTING.rst#pull-request-guidelines) for more information. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org > Fix backticks in rst file > - > > Key: AIRFLOW-6537 > URL: https://issues.apache.org/jira/browse/AIRFLOW-6537 > Project: Apache Airflow > Issue Type: Improvement > Components: documentation >Affects Versions: 1.10.7 >Reporter: Kamil Bregula >Priority: Major > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (AIRFLOW-6537) Fix backticks in rst file
Kamil Bregula created AIRFLOW-6537: -- Summary: Fix backticks in rst file Key: AIRFLOW-6537 URL: https://issues.apache.org/jira/browse/AIRFLOW-6537 Project: Apache Airflow Issue Type: Improvement Components: documentation Affects Versions: 1.10.7 Reporter: Kamil Bregula -- This message was sent by Atlassian Jira (v8.3.4#803005)
[GitHub] [airflow] tooptoop4 commented on issue #7038: [AIRFLOW-4495] allow externally triggered dags to run for future exec dates
tooptoop4 commented on issue #7038: [AIRFLOW-4495] allow externally triggered dags to run for future exec dates URL: https://github.com/apache/airflow/pull/7038#issuecomment-573337735 updates made This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [airflow] tooptoop4 commented on a change in pull request #7038: [AIRFLOW-4495] allow externally triggered dags to run for future exec dates
tooptoop4 commented on a change in pull request #7038: [AIRFLOW-4495] allow
externally triggered dags to run for future exec dates
URL: https://github.com/apache/airflow/pull/7038#discussion_r365431328
##
File path: tests/ti_deps/deps/test_runnable_exec_date_dep.py
##
@@ -37,14 +37,14 @@ def _get_task_instance(self, execution_date,
dag_end_date=None, task_end_date=No
@freeze_time('2016-01-01')
def test_exec_date_after_end_date(self):
"""
-If the dag's execution date is in the future this dep should fail
+If the dag's execution date is in the future this dep should succeed
"""
ti = self._get_task_instance(
dag_end_date=datetime(2016, 1, 3),
task_end_date=datetime(2016, 1, 3),
execution_date=datetime(2016, 1, 2),
)
-self.assertFalse(RunnableExecDateDep().is_met(ti=ti))
+self.assertTrue(RunnableExecDateDep().is_met(ti=ti))
Review comment:
@yuqian90 can u help to debug?
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [airflow] amichai07 commented on a change in pull request #4751: [AIRFLOW-3607] collected trigger rule dep check per dag run
amichai07 commented on a change in pull request #4751: [AIRFLOW-3607] collected trigger rule dep check per dag run URL: https://github.com/apache/airflow/pull/4751#discussion_r365532675 ## File path: tests/models/test_dagrun.py ## @@ -229,7 +229,7 @@ def test_dagrun_deadlock(self): start_date=now) ti_op1 = dr.get_task_instance(task_id=op1.task_id) -ti_op1.set_state(state=State.SUCCESS, session=session) +ti_op1.set_state(state=State.FAILED, session=session) Review comment: You are right, But it means the dr should have been 'failed' instead of 'running'. I will change it the task to 'success' and the state of the dagrun to 'failed' and then it will be as intended. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [airflow] codecov-io edited a comment on issue #7127: [AIRFLOW-6510] - Fix druid operator templating
codecov-io edited a comment on issue #7127: [AIRFLOW-6510] - Fix druid operator templating URL: https://github.com/apache/airflow/pull/7127#issuecomment-573027155 # [Codecov](https://codecov.io/gh/apache/airflow/pull/7127?src=pr=h1) Report > Merging [#7127](https://codecov.io/gh/apache/airflow/pull/7127?src=pr=desc) into [master](https://codecov.io/gh/apache/airflow/commit/86f525f6527e45c9ff1b8f844569c4d9d22bccac?src=pr=desc) will **decrease** coverage by `0.42%`. > The diff coverage is `50%`. [](https://codecov.io/gh/apache/airflow/pull/7127?src=pr=tree) ```diff @@Coverage Diff @@ ## master#7127 +/- ## == - Coverage 85.24% 84.82% -0.42% == Files 683 683 Lines 3915639153 -3 == - Hits3337933212 -167 - Misses 5777 5941 +164 ``` | [Impacted Files](https://codecov.io/gh/apache/airflow/pull/7127?src=pr=tree) | Coverage Δ | | |---|---|---| | [airflow/contrib/operators/druid\_operator.py](https://codecov.io/gh/apache/airflow/pull/7127/diff?src=pr=tree#diff-YWlyZmxvdy9jb250cmliL29wZXJhdG9ycy9kcnVpZF9vcGVyYXRvci5weQ==) | `82.35% <50%> (-1.86%)` | :arrow_down: | | [airflow/kubernetes/volume\_mount.py](https://codecov.io/gh/apache/airflow/pull/7127/diff?src=pr=tree#diff-YWlyZmxvdy9rdWJlcm5ldGVzL3ZvbHVtZV9tb3VudC5weQ==) | `44.44% <0%> (-55.56%)` | :arrow_down: | | [airflow/kubernetes/volume.py](https://codecov.io/gh/apache/airflow/pull/7127/diff?src=pr=tree#diff-YWlyZmxvdy9rdWJlcm5ldGVzL3ZvbHVtZS5weQ==) | `52.94% <0%> (-47.06%)` | :arrow_down: | | [airflow/kubernetes/pod\_launcher.py](https://codecov.io/gh/apache/airflow/pull/7127/diff?src=pr=tree#diff-YWlyZmxvdy9rdWJlcm5ldGVzL3BvZF9sYXVuY2hlci5weQ==) | `45.25% <0%> (-46.72%)` | :arrow_down: | | [airflow/executors/sequential\_executor.py](https://codecov.io/gh/apache/airflow/pull/7127/diff?src=pr=tree#diff-YWlyZmxvdy9leGVjdXRvcnMvc2VxdWVudGlhbF9leGVjdXRvci5weQ==) | `56% <0%> (-32%)` | :arrow_down: | | [airflow/kubernetes/refresh\_config.py](https://codecov.io/gh/apache/airflow/pull/7127/diff?src=pr=tree#diff-YWlyZmxvdy9rdWJlcm5ldGVzL3JlZnJlc2hfY29uZmlnLnB5) | `50.98% <0%> (-23.53%)` | :arrow_down: | | [...rflow/contrib/operators/kubernetes\_pod\_operator.py](https://codecov.io/gh/apache/airflow/pull/7127/diff?src=pr=tree#diff-YWlyZmxvdy9jb250cmliL29wZXJhdG9ycy9rdWJlcm5ldGVzX3BvZF9vcGVyYXRvci5weQ==) | `78.31% <0%> (-20.49%)` | :arrow_down: | | [airflow/utils/sqlalchemy.py](https://codecov.io/gh/apache/airflow/pull/7127/diff?src=pr=tree#diff-YWlyZmxvdy91dGlscy9zcWxhbGNoZW15LnB5) | `90% <0%> (-6.67%)` | :arrow_down: | | [airflow/utils/dag\_processing.py](https://codecov.io/gh/apache/airflow/pull/7127/diff?src=pr=tree#diff-YWlyZmxvdy91dGlscy9kYWdfcHJvY2Vzc2luZy5weQ==) | `81.71% <0%> (-6.29%)` | :arrow_down: | | [airflow/jobs/scheduler\_job.py](https://codecov.io/gh/apache/airflow/pull/7127/diff?src=pr=tree#diff-YWlyZmxvdy9qb2JzL3NjaGVkdWxlcl9qb2IucHk=) | `88.12% <0%> (-1.18%)` | :arrow_down: | | ... and [1 more](https://codecov.io/gh/apache/airflow/pull/7127/diff?src=pr=tree-more) | | -- [Continue to review full report at Codecov](https://codecov.io/gh/apache/airflow/pull/7127?src=pr=continue). > **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta) > `Δ = absolute (impact)`, `ø = not affected`, `? = missing data` > Powered by [Codecov](https://codecov.io/gh/apache/airflow/pull/7127?src=pr=footer). Last update [86f525f...c54ddbc](https://codecov.io/gh/apache/airflow/pull/7127?src=pr=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments). This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [airflow] codecov-io edited a comment on issue #7127: [AIRFLOW-6510] - Fix druid operator templating
codecov-io edited a comment on issue #7127: [AIRFLOW-6510] - Fix druid operator templating URL: https://github.com/apache/airflow/pull/7127#issuecomment-573027155 # [Codecov](https://codecov.io/gh/apache/airflow/pull/7127?src=pr=h1) Report > Merging [#7127](https://codecov.io/gh/apache/airflow/pull/7127?src=pr=desc) into [master](https://codecov.io/gh/apache/airflow/commit/86f525f6527e45c9ff1b8f844569c4d9d22bccac?src=pr=desc) will **decrease** coverage by `0.42%`. > The diff coverage is `50%`. [](https://codecov.io/gh/apache/airflow/pull/7127?src=pr=tree) ```diff @@Coverage Diff @@ ## master#7127 +/- ## == - Coverage 85.24% 84.82% -0.42% == Files 683 683 Lines 3915639153 -3 == - Hits3337933212 -167 - Misses 5777 5941 +164 ``` | [Impacted Files](https://codecov.io/gh/apache/airflow/pull/7127?src=pr=tree) | Coverage Δ | | |---|---|---| | [airflow/contrib/operators/druid\_operator.py](https://codecov.io/gh/apache/airflow/pull/7127/diff?src=pr=tree#diff-YWlyZmxvdy9jb250cmliL29wZXJhdG9ycy9kcnVpZF9vcGVyYXRvci5weQ==) | `82.35% <50%> (-1.86%)` | :arrow_down: | | [airflow/kubernetes/volume\_mount.py](https://codecov.io/gh/apache/airflow/pull/7127/diff?src=pr=tree#diff-YWlyZmxvdy9rdWJlcm5ldGVzL3ZvbHVtZV9tb3VudC5weQ==) | `44.44% <0%> (-55.56%)` | :arrow_down: | | [airflow/kubernetes/volume.py](https://codecov.io/gh/apache/airflow/pull/7127/diff?src=pr=tree#diff-YWlyZmxvdy9rdWJlcm5ldGVzL3ZvbHVtZS5weQ==) | `52.94% <0%> (-47.06%)` | :arrow_down: | | [airflow/kubernetes/pod\_launcher.py](https://codecov.io/gh/apache/airflow/pull/7127/diff?src=pr=tree#diff-YWlyZmxvdy9rdWJlcm5ldGVzL3BvZF9sYXVuY2hlci5weQ==) | `45.25% <0%> (-46.72%)` | :arrow_down: | | [airflow/executors/sequential\_executor.py](https://codecov.io/gh/apache/airflow/pull/7127/diff?src=pr=tree#diff-YWlyZmxvdy9leGVjdXRvcnMvc2VxdWVudGlhbF9leGVjdXRvci5weQ==) | `56% <0%> (-32%)` | :arrow_down: | | [airflow/kubernetes/refresh\_config.py](https://codecov.io/gh/apache/airflow/pull/7127/diff?src=pr=tree#diff-YWlyZmxvdy9rdWJlcm5ldGVzL3JlZnJlc2hfY29uZmlnLnB5) | `50.98% <0%> (-23.53%)` | :arrow_down: | | [...rflow/contrib/operators/kubernetes\_pod\_operator.py](https://codecov.io/gh/apache/airflow/pull/7127/diff?src=pr=tree#diff-YWlyZmxvdy9jb250cmliL29wZXJhdG9ycy9rdWJlcm5ldGVzX3BvZF9vcGVyYXRvci5weQ==) | `78.31% <0%> (-20.49%)` | :arrow_down: | | [airflow/utils/sqlalchemy.py](https://codecov.io/gh/apache/airflow/pull/7127/diff?src=pr=tree#diff-YWlyZmxvdy91dGlscy9zcWxhbGNoZW15LnB5) | `90% <0%> (-6.67%)` | :arrow_down: | | [airflow/utils/dag\_processing.py](https://codecov.io/gh/apache/airflow/pull/7127/diff?src=pr=tree#diff-YWlyZmxvdy91dGlscy9kYWdfcHJvY2Vzc2luZy5weQ==) | `81.71% <0%> (-6.29%)` | :arrow_down: | | [airflow/jobs/scheduler\_job.py](https://codecov.io/gh/apache/airflow/pull/7127/diff?src=pr=tree#diff-YWlyZmxvdy9qb2JzL3NjaGVkdWxlcl9qb2IucHk=) | `88.12% <0%> (-1.18%)` | :arrow_down: | | ... and [1 more](https://codecov.io/gh/apache/airflow/pull/7127/diff?src=pr=tree-more) | | -- [Continue to review full report at Codecov](https://codecov.io/gh/apache/airflow/pull/7127?src=pr=continue). > **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta) > `Δ = absolute (impact)`, `ø = not affected`, `? = missing data` > Powered by [Codecov](https://codecov.io/gh/apache/airflow/pull/7127?src=pr=footer). Last update [86f525f...c54ddbc](https://codecov.io/gh/apache/airflow/pull/7127?src=pr=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments). This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [airflow] codecov-io edited a comment on issue #7127: [AIRFLOW-6510] - Fix druid operator templating
codecov-io edited a comment on issue #7127: [AIRFLOW-6510] - Fix druid operator templating URL: https://github.com/apache/airflow/pull/7127#issuecomment-573027155 # [Codecov](https://codecov.io/gh/apache/airflow/pull/7127?src=pr=h1) Report > Merging [#7127](https://codecov.io/gh/apache/airflow/pull/7127?src=pr=desc) into [master](https://codecov.io/gh/apache/airflow/commit/86f525f6527e45c9ff1b8f844569c4d9d22bccac?src=pr=desc) will **decrease** coverage by `0.42%`. > The diff coverage is `50%`. [](https://codecov.io/gh/apache/airflow/pull/7127?src=pr=tree) ```diff @@Coverage Diff @@ ## master#7127 +/- ## == - Coverage 85.24% 84.82% -0.42% == Files 683 683 Lines 3915639153 -3 == - Hits3337933212 -167 - Misses 5777 5941 +164 ``` | [Impacted Files](https://codecov.io/gh/apache/airflow/pull/7127?src=pr=tree) | Coverage Δ | | |---|---|---| | [airflow/contrib/operators/druid\_operator.py](https://codecov.io/gh/apache/airflow/pull/7127/diff?src=pr=tree#diff-YWlyZmxvdy9jb250cmliL29wZXJhdG9ycy9kcnVpZF9vcGVyYXRvci5weQ==) | `82.35% <50%> (-1.86%)` | :arrow_down: | | [airflow/kubernetes/volume\_mount.py](https://codecov.io/gh/apache/airflow/pull/7127/diff?src=pr=tree#diff-YWlyZmxvdy9rdWJlcm5ldGVzL3ZvbHVtZV9tb3VudC5weQ==) | `44.44% <0%> (-55.56%)` | :arrow_down: | | [airflow/kubernetes/volume.py](https://codecov.io/gh/apache/airflow/pull/7127/diff?src=pr=tree#diff-YWlyZmxvdy9rdWJlcm5ldGVzL3ZvbHVtZS5weQ==) | `52.94% <0%> (-47.06%)` | :arrow_down: | | [airflow/kubernetes/pod\_launcher.py](https://codecov.io/gh/apache/airflow/pull/7127/diff?src=pr=tree#diff-YWlyZmxvdy9rdWJlcm5ldGVzL3BvZF9sYXVuY2hlci5weQ==) | `45.25% <0%> (-46.72%)` | :arrow_down: | | [airflow/executors/sequential\_executor.py](https://codecov.io/gh/apache/airflow/pull/7127/diff?src=pr=tree#diff-YWlyZmxvdy9leGVjdXRvcnMvc2VxdWVudGlhbF9leGVjdXRvci5weQ==) | `56% <0%> (-32%)` | :arrow_down: | | [airflow/kubernetes/refresh\_config.py](https://codecov.io/gh/apache/airflow/pull/7127/diff?src=pr=tree#diff-YWlyZmxvdy9rdWJlcm5ldGVzL3JlZnJlc2hfY29uZmlnLnB5) | `50.98% <0%> (-23.53%)` | :arrow_down: | | [...rflow/contrib/operators/kubernetes\_pod\_operator.py](https://codecov.io/gh/apache/airflow/pull/7127/diff?src=pr=tree#diff-YWlyZmxvdy9jb250cmliL29wZXJhdG9ycy9rdWJlcm5ldGVzX3BvZF9vcGVyYXRvci5weQ==) | `78.31% <0%> (-20.49%)` | :arrow_down: | | [airflow/utils/sqlalchemy.py](https://codecov.io/gh/apache/airflow/pull/7127/diff?src=pr=tree#diff-YWlyZmxvdy91dGlscy9zcWxhbGNoZW15LnB5) | `90% <0%> (-6.67%)` | :arrow_down: | | [airflow/utils/dag\_processing.py](https://codecov.io/gh/apache/airflow/pull/7127/diff?src=pr=tree#diff-YWlyZmxvdy91dGlscy9kYWdfcHJvY2Vzc2luZy5weQ==) | `81.71% <0%> (-6.29%)` | :arrow_down: | | [airflow/jobs/scheduler\_job.py](https://codecov.io/gh/apache/airflow/pull/7127/diff?src=pr=tree#diff-YWlyZmxvdy9qb2JzL3NjaGVkdWxlcl9qb2IucHk=) | `88.12% <0%> (-1.18%)` | :arrow_down: | | ... and [1 more](https://codecov.io/gh/apache/airflow/pull/7127/diff?src=pr=tree-more) | | -- [Continue to review full report at Codecov](https://codecov.io/gh/apache/airflow/pull/7127?src=pr=continue). > **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta) > `Δ = absolute (impact)`, `ø = not affected`, `? = missing data` > Powered by [Codecov](https://codecov.io/gh/apache/airflow/pull/7127?src=pr=footer). Last update [86f525f...c54ddbc](https://codecov.io/gh/apache/airflow/pull/7127?src=pr=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments). This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [airflow] codecov-io edited a comment on issue #6954: [AIRFLOW-4355] removed task should not lead to dagrun success
codecov-io edited a comment on issue #6954: [AIRFLOW-4355] removed task should not lead to dagrun success URL: https://github.com/apache/airflow/pull/6954#issuecomment-569545898 # [Codecov](https://codecov.io/gh/apache/airflow/pull/6954?src=pr=h1) Report > Merging [#6954](https://codecov.io/gh/apache/airflow/pull/6954?src=pr=desc) into [master](https://codecov.io/gh/apache/airflow/commit/3d82e8cf6a9b6b0af3fafef0769f77ced1974cbf?src=pr=desc) will **increase** coverage by `0.28%`. > The diff coverage is `100%`. [](https://codecov.io/gh/apache/airflow/pull/6954?src=pr=tree) ```diff @@Coverage Diff @@ ## master#6954 +/- ## == + Coverage 84.68% 84.96% +0.28% == Files 680 683 +3 Lines 3868639164 +478 == + Hits3276133276 +515 + Misses 5925 5888 -37 ``` | [Impacted Files](https://codecov.io/gh/apache/airflow/pull/6954?src=pr=tree) | Coverage Δ | | |---|---|---| | [airflow/models/dagrun.py](https://codecov.io/gh/apache/airflow/pull/6954/diff?src=pr=tree#diff-YWlyZmxvdy9tb2RlbHMvZGFncnVuLnB5) | `96.77% <100%> (+0.18%)` | :arrow_up: | | [airflow/kubernetes/volume\_mount.py](https://codecov.io/gh/apache/airflow/pull/6954/diff?src=pr=tree#diff-YWlyZmxvdy9rdWJlcm5ldGVzL3ZvbHVtZV9tb3VudC5weQ==) | `44.44% <0%> (-55.56%)` | :arrow_down: | | [airflow/kubernetes/volume.py](https://codecov.io/gh/apache/airflow/pull/6954/diff?src=pr=tree#diff-YWlyZmxvdy9rdWJlcm5ldGVzL3ZvbHVtZS5weQ==) | `52.94% <0%> (-47.06%)` | :arrow_down: | | [airflow/kubernetes/pod\_launcher.py](https://codecov.io/gh/apache/airflow/pull/6954/diff?src=pr=tree#diff-YWlyZmxvdy9rdWJlcm5ldGVzL3BvZF9sYXVuY2hlci5weQ==) | `45.25% <0%> (-46.72%)` | :arrow_down: | | [airflow/kubernetes/refresh\_config.py](https://codecov.io/gh/apache/airflow/pull/6954/diff?src=pr=tree#diff-YWlyZmxvdy9rdWJlcm5ldGVzL3JlZnJlc2hfY29uZmlnLnB5) | `50.98% <0%> (-23.53%)` | :arrow_down: | | [...rflow/contrib/operators/kubernetes\_pod\_operator.py](https://codecov.io/gh/apache/airflow/pull/6954/diff?src=pr=tree#diff-YWlyZmxvdy9jb250cmliL29wZXJhdG9ycy9rdWJlcm5ldGVzX3BvZF9vcGVyYXRvci5weQ==) | `78.31% <0%> (-20.44%)` | :arrow_down: | | [airflow/contrib/operators/gcp\_transfer\_operator.py](https://codecov.io/gh/apache/airflow/pull/6954/diff?src=pr=tree#diff-YWlyZmxvdy9jb250cmliL29wZXJhdG9ycy9nY3BfdHJhbnNmZXJfb3BlcmF0b3IucHk=) | `95.45% <0%> (-4.55%)` | :arrow_down: | | [airflow/providers/amazon/aws/operators/batch.py](https://codecov.io/gh/apache/airflow/pull/6954/diff?src=pr=tree#diff-YWlyZmxvdy9wcm92aWRlcnMvYW1hem9uL2F3cy9vcGVyYXRvcnMvYmF0Y2gucHk=) | `93.75% <0%> (-2.09%)` | :arrow_down: | | [airflow/serialization/serialized\_objects.py](https://codecov.io/gh/apache/airflow/pull/6954/diff?src=pr=tree#diff-YWlyZmxvdy9zZXJpYWxpemF0aW9uL3NlcmlhbGl6ZWRfb2JqZWN0cy5weQ==) | `90.03% <0%> (-1.1%)` | :arrow_down: | | [airflow/cli/commands/task\_command.py](https://codecov.io/gh/apache/airflow/pull/6954/diff?src=pr=tree#diff-YWlyZmxvdy9jbGkvY29tbWFuZHMvdGFza19jb21tYW5kLnB5) | `70.96% <0%> (-0.85%)` | :arrow_down: | | ... and [54 more](https://codecov.io/gh/apache/airflow/pull/6954/diff?src=pr=tree-more) | | -- [Continue to review full report at Codecov](https://codecov.io/gh/apache/airflow/pull/6954?src=pr=continue). > **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta) > `Δ = absolute (impact)`, `ø = not affected`, `? = missing data` > Powered by [Codecov](https://codecov.io/gh/apache/airflow/pull/6954?src=pr=footer). Last update [3d82e8c...3e2817c](https://codecov.io/gh/apache/airflow/pull/6954?src=pr=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments). This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [airflow] codecov-io edited a comment on issue #6954: [AIRFLOW-4355] removed task should not lead to dagrun success
codecov-io edited a comment on issue #6954: [AIRFLOW-4355] removed task should not lead to dagrun success URL: https://github.com/apache/airflow/pull/6954#issuecomment-569545898 # [Codecov](https://codecov.io/gh/apache/airflow/pull/6954?src=pr=h1) Report > Merging [#6954](https://codecov.io/gh/apache/airflow/pull/6954?src=pr=desc) into [master](https://codecov.io/gh/apache/airflow/commit/0614edbb59657e4998d80e7092214c8e657861a0?src=pr=desc) will **increase** coverage by `0.4%`. > The diff coverage is `78.75%`. [](https://codecov.io/gh/apache/airflow/pull/6954?src=pr=tree) ```diff @@Coverage Diff@@ ## master#6954 +/- ## = + Coverage 84.56% 84.96% +0.4% = Files 679 683 +4 Lines 3854739164+617 = + Hits3259633276+680 + Misses 5951 5888 -63 ``` | [Impacted Files](https://codecov.io/gh/apache/airflow/pull/6954?src=pr=tree) | Coverage Δ | | |---|---|---| | [airflow/gcp/operators/dataflow.py](https://codecov.io/gh/apache/airflow/pull/6954/diff?src=pr=tree#diff-YWlyZmxvdy9nY3Avb3BlcmF0b3JzL2RhdGFmbG93LnB5) | `99.07% <ø> (ø)` | :arrow_up: | | [airflow/gcp/hooks/dataflow.py](https://codecov.io/gh/apache/airflow/pull/6954/diff?src=pr=tree#diff-YWlyZmxvdy9nY3AvaG9va3MvZGF0YWZsb3cucHk=) | `89.28% <ø> (ø)` | :arrow_up: | | [airflow/utils/weight\_rule.py](https://codecov.io/gh/apache/airflow/pull/6954/diff?src=pr=tree#diff-YWlyZmxvdy91dGlscy93ZWlnaHRfcnVsZS5weQ==) | `100% <ø> (ø)` | :arrow_up: | | [airflow/example\_dags/docker\_copy\_data.py](https://codecov.io/gh/apache/airflow/pull/6954/diff?src=pr=tree#diff-YWlyZmxvdy9leGFtcGxlX2RhZ3MvZG9ja2VyX2NvcHlfZGF0YS5weQ==) | `100% <ø> (ø)` | :arrow_up: | | [airflow/example\_dags/example\_docker\_operator.py](https://codecov.io/gh/apache/airflow/pull/6954/diff?src=pr=tree#diff-YWlyZmxvdy9leGFtcGxlX2RhZ3MvZXhhbXBsZV9kb2NrZXJfb3BlcmF0b3IucHk=) | `100% <ø> (ø)` | :arrow_up: | | [airflow/cli/commands/db\_command.py](https://codecov.io/gh/apache/airflow/pull/6954/diff?src=pr=tree#diff-YWlyZmxvdy9jbGkvY29tbWFuZHMvZGJfY29tbWFuZC5weQ==) | `97.43% <ø> (ø)` | :arrow_up: | | [...flow/example\_dags/example\_docker\_swarm\_operator.py](https://codecov.io/gh/apache/airflow/pull/6954/diff?src=pr=tree#diff-YWlyZmxvdy9leGFtcGxlX2RhZ3MvZXhhbXBsZV9kb2NrZXJfc3dhcm1fb3BlcmF0b3IucHk=) | `100% <ø> (ø)` | :arrow_up: | | [airflow/gcp/utils/mlengine\_operator\_utils.py](https://codecov.io/gh/apache/airflow/pull/6954/diff?src=pr=tree#diff-YWlyZmxvdy9nY3AvdXRpbHMvbWxlbmdpbmVfb3BlcmF0b3JfdXRpbHMucHk=) | `95.34% <ø> (ø)` | :arrow_up: | | [airflow/utils/state.py](https://codecov.io/gh/apache/airflow/pull/6954/diff?src=pr=tree#diff-YWlyZmxvdy91dGlscy9zdGF0ZS5weQ==) | `96.29% <ø> (+3.19%)` | :arrow_up: | | [airflow/utils/operator\_resources.py](https://codecov.io/gh/apache/airflow/pull/6954/diff?src=pr=tree#diff-YWlyZmxvdy91dGlscy9vcGVyYXRvcl9yZXNvdXJjZXMucHk=) | `84.78% <ø> (ø)` | :arrow_up: | | ... and [204 more](https://codecov.io/gh/apache/airflow/pull/6954/diff?src=pr=tree-more) | | -- [Continue to review full report at Codecov](https://codecov.io/gh/apache/airflow/pull/6954?src=pr=continue). > **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta) > `Δ = absolute (impact)`, `ø = not affected`, `? = missing data` > Powered by [Codecov](https://codecov.io/gh/apache/airflow/pull/6954?src=pr=footer). Last update [0614edb...3e2817c](https://codecov.io/gh/apache/airflow/pull/6954?src=pr=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments). This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [airflow] codecov-io edited a comment on issue #6954: [AIRFLOW-4355] removed task should not lead to dagrun success
codecov-io edited a comment on issue #6954: [AIRFLOW-4355] removed task should not lead to dagrun success URL: https://github.com/apache/airflow/pull/6954#issuecomment-569545898 # [Codecov](https://codecov.io/gh/apache/airflow/pull/6954?src=pr=h1) Report > Merging [#6954](https://codecov.io/gh/apache/airflow/pull/6954?src=pr=desc) into [master](https://codecov.io/gh/apache/airflow/commit/0614edbb59657e4998d80e7092214c8e657861a0?src=pr=desc) will **increase** coverage by `0.4%`. > The diff coverage is `78.75%`. [](https://codecov.io/gh/apache/airflow/pull/6954?src=pr=tree) ```diff @@Coverage Diff@@ ## master#6954 +/- ## = + Coverage 84.56% 84.96% +0.4% = Files 679 683 +4 Lines 3854739164+617 = + Hits3259633276+680 + Misses 5951 5888 -63 ``` | [Impacted Files](https://codecov.io/gh/apache/airflow/pull/6954?src=pr=tree) | Coverage Δ | | |---|---|---| | [airflow/gcp/operators/dataflow.py](https://codecov.io/gh/apache/airflow/pull/6954/diff?src=pr=tree#diff-YWlyZmxvdy9nY3Avb3BlcmF0b3JzL2RhdGFmbG93LnB5) | `99.07% <ø> (ø)` | :arrow_up: | | [airflow/gcp/hooks/dataflow.py](https://codecov.io/gh/apache/airflow/pull/6954/diff?src=pr=tree#diff-YWlyZmxvdy9nY3AvaG9va3MvZGF0YWZsb3cucHk=) | `89.28% <ø> (ø)` | :arrow_up: | | [airflow/utils/weight\_rule.py](https://codecov.io/gh/apache/airflow/pull/6954/diff?src=pr=tree#diff-YWlyZmxvdy91dGlscy93ZWlnaHRfcnVsZS5weQ==) | `100% <ø> (ø)` | :arrow_up: | | [airflow/example\_dags/docker\_copy\_data.py](https://codecov.io/gh/apache/airflow/pull/6954/diff?src=pr=tree#diff-YWlyZmxvdy9leGFtcGxlX2RhZ3MvZG9ja2VyX2NvcHlfZGF0YS5weQ==) | `100% <ø> (ø)` | :arrow_up: | | [airflow/example\_dags/example\_docker\_operator.py](https://codecov.io/gh/apache/airflow/pull/6954/diff?src=pr=tree#diff-YWlyZmxvdy9leGFtcGxlX2RhZ3MvZXhhbXBsZV9kb2NrZXJfb3BlcmF0b3IucHk=) | `100% <ø> (ø)` | :arrow_up: | | [airflow/cli/commands/db\_command.py](https://codecov.io/gh/apache/airflow/pull/6954/diff?src=pr=tree#diff-YWlyZmxvdy9jbGkvY29tbWFuZHMvZGJfY29tbWFuZC5weQ==) | `97.43% <ø> (ø)` | :arrow_up: | | [...flow/example\_dags/example\_docker\_swarm\_operator.py](https://codecov.io/gh/apache/airflow/pull/6954/diff?src=pr=tree#diff-YWlyZmxvdy9leGFtcGxlX2RhZ3MvZXhhbXBsZV9kb2NrZXJfc3dhcm1fb3BlcmF0b3IucHk=) | `100% <ø> (ø)` | :arrow_up: | | [airflow/gcp/utils/mlengine\_operator\_utils.py](https://codecov.io/gh/apache/airflow/pull/6954/diff?src=pr=tree#diff-YWlyZmxvdy9nY3AvdXRpbHMvbWxlbmdpbmVfb3BlcmF0b3JfdXRpbHMucHk=) | `95.34% <ø> (ø)` | :arrow_up: | | [airflow/utils/state.py](https://codecov.io/gh/apache/airflow/pull/6954/diff?src=pr=tree#diff-YWlyZmxvdy91dGlscy9zdGF0ZS5weQ==) | `96.29% <ø> (+3.19%)` | :arrow_up: | | [airflow/utils/operator\_resources.py](https://codecov.io/gh/apache/airflow/pull/6954/diff?src=pr=tree#diff-YWlyZmxvdy91dGlscy9vcGVyYXRvcl9yZXNvdXJjZXMucHk=) | `84.78% <ø> (ø)` | :arrow_up: | | ... and [204 more](https://codecov.io/gh/apache/airflow/pull/6954/diff?src=pr=tree-more) | | -- [Continue to review full report at Codecov](https://codecov.io/gh/apache/airflow/pull/6954?src=pr=continue). > **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta) > `Δ = absolute (impact)`, `ø = not affected`, `? = missing data` > Powered by [Codecov](https://codecov.io/gh/apache/airflow/pull/6954?src=pr=footer). Last update [0614edb...3e2817c](https://codecov.io/gh/apache/airflow/pull/6954?src=pr=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments). This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [airflow] codecov-io edited a comment on issue #6954: [AIRFLOW-4355] removed task should not lead to dagrun success
codecov-io edited a comment on issue #6954: [AIRFLOW-4355] removed task should not lead to dagrun success URL: https://github.com/apache/airflow/pull/6954#issuecomment-569545898 # [Codecov](https://codecov.io/gh/apache/airflow/pull/6954?src=pr=h1) Report > Merging [#6954](https://codecov.io/gh/apache/airflow/pull/6954?src=pr=desc) into [master](https://codecov.io/gh/apache/airflow/commit/0614edbb59657e4998d80e7092214c8e657861a0?src=pr=desc) will **increase** coverage by `0.4%`. > The diff coverage is `78.75%`. [](https://codecov.io/gh/apache/airflow/pull/6954?src=pr=tree) ```diff @@Coverage Diff@@ ## master#6954 +/- ## = + Coverage 84.56% 84.96% +0.4% = Files 679 683 +4 Lines 3854739164+617 = + Hits3259633276+680 + Misses 5951 5888 -63 ``` | [Impacted Files](https://codecov.io/gh/apache/airflow/pull/6954?src=pr=tree) | Coverage Δ | | |---|---|---| | [airflow/gcp/operators/dataflow.py](https://codecov.io/gh/apache/airflow/pull/6954/diff?src=pr=tree#diff-YWlyZmxvdy9nY3Avb3BlcmF0b3JzL2RhdGFmbG93LnB5) | `99.07% <ø> (ø)` | :arrow_up: | | [airflow/gcp/hooks/dataflow.py](https://codecov.io/gh/apache/airflow/pull/6954/diff?src=pr=tree#diff-YWlyZmxvdy9nY3AvaG9va3MvZGF0YWZsb3cucHk=) | `89.28% <ø> (ø)` | :arrow_up: | | [airflow/utils/weight\_rule.py](https://codecov.io/gh/apache/airflow/pull/6954/diff?src=pr=tree#diff-YWlyZmxvdy91dGlscy93ZWlnaHRfcnVsZS5weQ==) | `100% <ø> (ø)` | :arrow_up: | | [airflow/example\_dags/docker\_copy\_data.py](https://codecov.io/gh/apache/airflow/pull/6954/diff?src=pr=tree#diff-YWlyZmxvdy9leGFtcGxlX2RhZ3MvZG9ja2VyX2NvcHlfZGF0YS5weQ==) | `100% <ø> (ø)` | :arrow_up: | | [airflow/example\_dags/example\_docker\_operator.py](https://codecov.io/gh/apache/airflow/pull/6954/diff?src=pr=tree#diff-YWlyZmxvdy9leGFtcGxlX2RhZ3MvZXhhbXBsZV9kb2NrZXJfb3BlcmF0b3IucHk=) | `100% <ø> (ø)` | :arrow_up: | | [airflow/cli/commands/db\_command.py](https://codecov.io/gh/apache/airflow/pull/6954/diff?src=pr=tree#diff-YWlyZmxvdy9jbGkvY29tbWFuZHMvZGJfY29tbWFuZC5weQ==) | `97.43% <ø> (ø)` | :arrow_up: | | [...flow/example\_dags/example\_docker\_swarm\_operator.py](https://codecov.io/gh/apache/airflow/pull/6954/diff?src=pr=tree#diff-YWlyZmxvdy9leGFtcGxlX2RhZ3MvZXhhbXBsZV9kb2NrZXJfc3dhcm1fb3BlcmF0b3IucHk=) | `100% <ø> (ø)` | :arrow_up: | | [airflow/gcp/utils/mlengine\_operator\_utils.py](https://codecov.io/gh/apache/airflow/pull/6954/diff?src=pr=tree#diff-YWlyZmxvdy9nY3AvdXRpbHMvbWxlbmdpbmVfb3BlcmF0b3JfdXRpbHMucHk=) | `95.34% <ø> (ø)` | :arrow_up: | | [airflow/utils/state.py](https://codecov.io/gh/apache/airflow/pull/6954/diff?src=pr=tree#diff-YWlyZmxvdy91dGlscy9zdGF0ZS5weQ==) | `96.29% <ø> (+3.19%)` | :arrow_up: | | [airflow/utils/operator\_resources.py](https://codecov.io/gh/apache/airflow/pull/6954/diff?src=pr=tree#diff-YWlyZmxvdy91dGlscy9vcGVyYXRvcl9yZXNvdXJjZXMucHk=) | `84.78% <ø> (ø)` | :arrow_up: | | ... and [204 more](https://codecov.io/gh/apache/airflow/pull/6954/diff?src=pr=tree-more) | | -- [Continue to review full report at Codecov](https://codecov.io/gh/apache/airflow/pull/6954?src=pr=continue). > **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta) > `Δ = absolute (impact)`, `ø = not affected`, `? = missing data` > Powered by [Codecov](https://codecov.io/gh/apache/airflow/pull/6954?src=pr=footer). Last update [0614edb...3e2817c](https://codecov.io/gh/apache/airflow/pull/6954?src=pr=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments). This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [airflow] codecov-io edited a comment on issue #6954: [AIRFLOW-4355] removed task should not lead to dagrun success
codecov-io edited a comment on issue #6954: [AIRFLOW-4355] removed task should not lead to dagrun success URL: https://github.com/apache/airflow/pull/6954#issuecomment-569545898 # [Codecov](https://codecov.io/gh/apache/airflow/pull/6954?src=pr=h1) Report > Merging [#6954](https://codecov.io/gh/apache/airflow/pull/6954?src=pr=desc) into [master](https://codecov.io/gh/apache/airflow/commit/0614edbb59657e4998d80e7092214c8e657861a0?src=pr=desc) will **increase** coverage by `0.4%`. > The diff coverage is `78.75%`. [](https://codecov.io/gh/apache/airflow/pull/6954?src=pr=tree) ```diff @@Coverage Diff@@ ## master#6954 +/- ## = + Coverage 84.56% 84.96% +0.4% = Files 679 683 +4 Lines 3854739164+617 = + Hits3259633276+680 + Misses 5951 5888 -63 ``` | [Impacted Files](https://codecov.io/gh/apache/airflow/pull/6954?src=pr=tree) | Coverage Δ | | |---|---|---| | [airflow/gcp/operators/dataflow.py](https://codecov.io/gh/apache/airflow/pull/6954/diff?src=pr=tree#diff-YWlyZmxvdy9nY3Avb3BlcmF0b3JzL2RhdGFmbG93LnB5) | `99.07% <ø> (ø)` | :arrow_up: | | [airflow/gcp/hooks/dataflow.py](https://codecov.io/gh/apache/airflow/pull/6954/diff?src=pr=tree#diff-YWlyZmxvdy9nY3AvaG9va3MvZGF0YWZsb3cucHk=) | `89.28% <ø> (ø)` | :arrow_up: | | [airflow/utils/weight\_rule.py](https://codecov.io/gh/apache/airflow/pull/6954/diff?src=pr=tree#diff-YWlyZmxvdy91dGlscy93ZWlnaHRfcnVsZS5weQ==) | `100% <ø> (ø)` | :arrow_up: | | [airflow/example\_dags/docker\_copy\_data.py](https://codecov.io/gh/apache/airflow/pull/6954/diff?src=pr=tree#diff-YWlyZmxvdy9leGFtcGxlX2RhZ3MvZG9ja2VyX2NvcHlfZGF0YS5weQ==) | `100% <ø> (ø)` | :arrow_up: | | [airflow/example\_dags/example\_docker\_operator.py](https://codecov.io/gh/apache/airflow/pull/6954/diff?src=pr=tree#diff-YWlyZmxvdy9leGFtcGxlX2RhZ3MvZXhhbXBsZV9kb2NrZXJfb3BlcmF0b3IucHk=) | `100% <ø> (ø)` | :arrow_up: | | [airflow/cli/commands/db\_command.py](https://codecov.io/gh/apache/airflow/pull/6954/diff?src=pr=tree#diff-YWlyZmxvdy9jbGkvY29tbWFuZHMvZGJfY29tbWFuZC5weQ==) | `97.43% <ø> (ø)` | :arrow_up: | | [...flow/example\_dags/example\_docker\_swarm\_operator.py](https://codecov.io/gh/apache/airflow/pull/6954/diff?src=pr=tree#diff-YWlyZmxvdy9leGFtcGxlX2RhZ3MvZXhhbXBsZV9kb2NrZXJfc3dhcm1fb3BlcmF0b3IucHk=) | `100% <ø> (ø)` | :arrow_up: | | [airflow/gcp/utils/mlengine\_operator\_utils.py](https://codecov.io/gh/apache/airflow/pull/6954/diff?src=pr=tree#diff-YWlyZmxvdy9nY3AvdXRpbHMvbWxlbmdpbmVfb3BlcmF0b3JfdXRpbHMucHk=) | `95.34% <ø> (ø)` | :arrow_up: | | [airflow/utils/state.py](https://codecov.io/gh/apache/airflow/pull/6954/diff?src=pr=tree#diff-YWlyZmxvdy91dGlscy9zdGF0ZS5weQ==) | `96.29% <ø> (+3.19%)` | :arrow_up: | | [airflow/utils/operator\_resources.py](https://codecov.io/gh/apache/airflow/pull/6954/diff?src=pr=tree#diff-YWlyZmxvdy91dGlscy9vcGVyYXRvcl9yZXNvdXJjZXMucHk=) | `84.78% <ø> (ø)` | :arrow_up: | | ... and [204 more](https://codecov.io/gh/apache/airflow/pull/6954/diff?src=pr=tree-more) | | -- [Continue to review full report at Codecov](https://codecov.io/gh/apache/airflow/pull/6954?src=pr=continue). > **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta) > `Δ = absolute (impact)`, `ø = not affected`, `? = missing data` > Powered by [Codecov](https://codecov.io/gh/apache/airflow/pull/6954?src=pr=footer). Last update [0614edb...3e2817c](https://codecov.io/gh/apache/airflow/pull/6954?src=pr=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments). This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [airflow] mik-laj commented on issue #7132: [AIRFLOW-6321] Revise operators and hooks documentation layout
mik-laj commented on issue #7132: [AIRFLOW-6321] Revise operators and hooks documentation layout URL: https://github.com/apache/airflow/pull/7132#issuecomment-573332749 I think we should wait until we finish [AIP-21](https://cwiki.apache.org/confluence/display/AIRFLOW/AIP-21%3A+Changes+in+import+paths) with changes on this page. It is possible that this will allow us to significantly simplify the page and we will be able to save the table, but only with two columns - `Name`, `Link to package`, `Guide`. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [airflow] ashb commented on issue #6090: [AIRFLOW-5470] Add Apache Livy REST operator
ashb commented on issue #6090: [AIRFLOW-5470] Add Apache Livy REST operator URL: https://github.com/apache/airflow/pull/6090#issuecomment-573332201 @lucacavazzana Please ping me when you've updated the filenames :) This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [airflow] mik-laj commented on a change in pull request #7138: [AIRFLOW-5912] Expose lineage API
mik-laj commented on a change in pull request #7138: [AIRFLOW-5912] Expose lineage API URL: https://github.com/apache/airflow/pull/7138#discussion_r365530276 ## File path: airflow/api/client/api_client.py ## @@ -70,3 +70,12 @@ def delete_pool(self, name): :param name: pool name """ raise NotImplementedError() + +def get_lineage(self, dag_id: str, execution_date: str): Review comment: the `airflow.api.cclient` package is intended for use by CLI. Can we call this method from the CLI? This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [airflow] mik-laj commented on issue #7138: [AIRFLOW-5912] Expose lineage API
mik-laj commented on issue #7138: [AIRFLOW-5912] Expose lineage API URL: https://github.com/apache/airflow/pull/7138#issuecomment-573331928 Can you also update the documentation? https://github.com/apache/airflow/blob/master/docs/rest-api-ref.rst This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [airflow] tooptoop4 commented on issue #6954: [AIRFLOW-4355] removed task should not lead to dagrun success
tooptoop4 commented on issue #6954: [AIRFLOW-4355] removed task should not lead to dagrun success URL: https://github.com/apache/airflow/pull/6954#issuecomment-573328666 updates made @ashb This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Commented] (AIRFLOW-2279) Clearing Tasks Across DAGs
[ https://issues.apache.org/jira/browse/AIRFLOW-2279?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17013523#comment-17013523 ] ASF GitHub Bot commented on AIRFLOW-2279: - yuqian90 commented on pull request #7139: [AIRFLOW-2279] Clear tasks across DAGs if marked by ExternalTaskMarker URL: https://github.com/apache/airflow/pull/7139 This change was already merged into master. I'm creating this PR to help committers merge into v1-10-test. --- Issue link: WILL BE INSERTED BY [boring-cyborg](https://github.com/kaxil/boring-cyborg) - [ ] Description above provides context of the change - [ ] Commit message/PR title starts with `[AIRFLOW-]`. AIRFLOW- = JIRA ID* - [ ] Unit tests coverage for changes (not needed for documentation changes) - [ ] Commits follow "[How to write a good git commit message](http://chris.beams.io/posts/git-commit/)" - [ ] Relevant documentation is updated including usage instructions. - [ ] I will engage committers as explained in [Contribution Workflow Example](https://github.com/apache/airflow/blob/master/CONTRIBUTING.rst#contribution-workflow-example). * For document-only changes commit message can start with `[AIRFLOW-]`. --- In case of fundamental code change, Airflow Improvement Proposal ([AIP](https://cwiki.apache.org/confluence/display/AIRFLOW/Airflow+Improvements+Proposals)) is needed. In case of a new dependency, check compliance with the [ASF 3rd Party License Policy](https://www.apache.org/legal/resolved.html#category-x). In case of backwards incompatible changes please leave a note in [UPDATING.md](https://github.com/apache/airflow/blob/master/UPDATING.md). Read the [Pull Request Guidelines](https://github.com/apache/airflow/blob/master/CONTRIBUTING.rst#pull-request-guidelines) for more information. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org > Clearing Tasks Across DAGs > -- > > Key: AIRFLOW-2279 > URL: https://issues.apache.org/jira/browse/AIRFLOW-2279 > Project: Apache Airflow > Issue Type: Improvement >Reporter: Achal Soni >Assignee: Qian Yu >Priority: Major > Fix For: 2.0.0 > > Attachments: cross_dag_ui_screenshot.png > > > At Stripe, we commonly have discrete dags that depend on each other by > leveraging ExternalTaskSensors. We also find ourselves routinely wanting to > not only clear tasks and their downstream tasks in a particular dag, but also > their downstream tasks in their dependent dags (linked by > ExternalTaskSensors). > We currently have extended Airflow to handle this by modifying the webapp and > cli tool to optionally clear dependent tasks across multiple dags (see > attached screenshot). > We want to open the floor for discussion with the larger Airflow community > about the usage of ExternalTaskSensors and specifically how to handle > clearing across dags. We are interested in learning more about the accepted > practices in this regard, and are very open/willing to contribute in this > area if there is interest! -- This message was sent by Atlassian Jira (v8.3.4#803005)
[GitHub] [airflow] yuqian90 opened a new pull request #7139: [AIRFLOW-2279] Clear tasks across DAGs if marked by ExternalTaskMarker
yuqian90 opened a new pull request #7139: [AIRFLOW-2279] Clear tasks across DAGs if marked by ExternalTaskMarker URL: https://github.com/apache/airflow/pull/7139 This change was already merged into master. I'm creating this PR to help committers merge into v1-10-test. --- Issue link: WILL BE INSERTED BY [boring-cyborg](https://github.com/kaxil/boring-cyborg) - [ ] Description above provides context of the change - [ ] Commit message/PR title starts with `[AIRFLOW-]`. AIRFLOW- = JIRA ID* - [ ] Unit tests coverage for changes (not needed for documentation changes) - [ ] Commits follow "[How to write a good git commit message](http://chris.beams.io/posts/git-commit/)" - [ ] Relevant documentation is updated including usage instructions. - [ ] I will engage committers as explained in [Contribution Workflow Example](https://github.com/apache/airflow/blob/master/CONTRIBUTING.rst#contribution-workflow-example). * For document-only changes commit message can start with `[AIRFLOW-]`. --- In case of fundamental code change, Airflow Improvement Proposal ([AIP](https://cwiki.apache.org/confluence/display/AIRFLOW/Airflow+Improvements+Proposals)) is needed. In case of a new dependency, check compliance with the [ASF 3rd Party License Policy](https://www.apache.org/legal/resolved.html#category-x). In case of backwards incompatible changes please leave a note in [UPDATING.md](https://github.com/apache/airflow/blob/master/UPDATING.md). Read the [Pull Request Guidelines](https://github.com/apache/airflow/blob/master/CONTRIBUTING.rst#pull-request-guidelines) for more information. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [airflow] yuqian90 commented on a change in pull request #7038: [AIRFLOW-4495] allow externally triggered dags to run for future exec dates
yuqian90 commented on a change in pull request #7038: [AIRFLOW-4495] allow
externally triggered dags to run for future exec dates
URL: https://github.com/apache/airflow/pull/7038#discussion_r365526869
##
File path: airflow/jobs/scheduler_job.py
##
@@ -629,8 +629,9 @@ def _process_task_instances(self, dag,
task_instances_list, session=None):
active_dag_runs = []
for run in dag_runs:
self.log.info("Examining DAG run %s", run)
-# don't consider runs that are executed in the future
-if run.execution_date > timezone.utcnow():
+# don't consider runs that are executed in the future unless
+# specified by config and schedule_interval is None
+if run.execution_date > timezone.utcnow() and not
run.dag.allow_future_exec_dates:
Review comment:
Just realized this line needs to be changed. Note that run.dag is not yet
set at this point. You need to use the `dag` variable. Probably worth
considering adding a test for `run.execution_date > timezone.utcnow()` in
test_scheduler_job.py to detect this.
```
if run.execution_date > timezone.utcnow() and not
dag.allow_future_exec_dates:
```
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [airflow] bolkedebruin commented on issue #7079: [AIRFLOW-6487] Romove unnecessary for-loop
bolkedebruin commented on issue #7079: [AIRFLOW-6487] Romove unnecessary for-loop URL: https://github.com/apache/airflow/pull/7079#issuecomment-573326441 Can you ensure tests are provided? This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Resolved] (AIRFLOW-5704) Docker scripts for kind kubernetes tests can be improved
[ https://issues.apache.org/jira/browse/AIRFLOW-5704?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jarek Potiuk resolved AIRFLOW-5704. --- Resolution: Fixed > Docker scripts for kind kubernetes tests can be improved > > > Key: AIRFLOW-5704 > URL: https://issues.apache.org/jira/browse/AIRFLOW-5704 > Project: Apache Airflow > Issue Type: Sub-task > Components: breeze, ci, dependencies >Affects Versions: 2.0.0 >Reporter: Jarek Potiuk >Assignee: Jarek Potiuk >Priority: Major > Fix For: 2.0.0 > > > The docker CI image for kind tests can be improved > > * Kubernetes Version and all the installation of docker + kubectl + kind can > be added back > * Running kubernetes scripts should be possible from within breeze without > special "kubernetes" environment > * --env breeze switch should be removed > * "bare" environment should be replaced by --no-deps switch > * ENV variable should disappear > > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[GitHub] [airflow] codecov-io edited a comment on issue #7136: [AIRFLOW-6536] Make job_id parameter optional
codecov-io edited a comment on issue #7136: [AIRFLOW-6536] Make job_id parameter optional URL: https://github.com/apache/airflow/pull/7136#issuecomment-573326191 # [Codecov](https://codecov.io/gh/apache/airflow/pull/7136?src=pr=h1) Report > Merging [#7136](https://codecov.io/gh/apache/airflow/pull/7136?src=pr=desc) into [master](https://codecov.io/gh/apache/airflow/commit/a676f159d4fb01f6e35baa04efa4914a695cd788?src=pr=desc) will **decrease** coverage by `0.28%`. > The diff coverage is `n/a`. [](https://codecov.io/gh/apache/airflow/pull/7136?src=pr=tree) ```diff @@Coverage Diff @@ ## master#7136 +/- ## == - Coverage 85.24% 84.95% -0.29% == Files 683 683 Lines 3915539155 == - Hits3337833266 -112 - Misses 5777 5889 +112 ``` | [Impacted Files](https://codecov.io/gh/apache/airflow/pull/7136?src=pr=tree) | Coverage Δ | | |---|---|---| | [airflow/contrib/operators/databricks\_operator.py](https://codecov.io/gh/apache/airflow/pull/7136/diff?src=pr=tree#diff-YWlyZmxvdy9jb250cmliL29wZXJhdG9ycy9kYXRhYnJpY2tzX29wZXJhdG9yLnB5) | `92.24% <ø> (ø)` | :arrow_up: | | [airflow/kubernetes/volume\_mount.py](https://codecov.io/gh/apache/airflow/pull/7136/diff?src=pr=tree#diff-YWlyZmxvdy9rdWJlcm5ldGVzL3ZvbHVtZV9tb3VudC5weQ==) | `44.44% <0%> (-55.56%)` | :arrow_down: | | [airflow/kubernetes/volume.py](https://codecov.io/gh/apache/airflow/pull/7136/diff?src=pr=tree#diff-YWlyZmxvdy9rdWJlcm5ldGVzL3ZvbHVtZS5weQ==) | `52.94% <0%> (-47.06%)` | :arrow_down: | | [airflow/kubernetes/pod\_launcher.py](https://codecov.io/gh/apache/airflow/pull/7136/diff?src=pr=tree#diff-YWlyZmxvdy9rdWJlcm5ldGVzL3BvZF9sYXVuY2hlci5weQ==) | `45.25% <0%> (-46.72%)` | :arrow_down: | | [airflow/kubernetes/refresh\_config.py](https://codecov.io/gh/apache/airflow/pull/7136/diff?src=pr=tree#diff-YWlyZmxvdy9rdWJlcm5ldGVzL3JlZnJlc2hfY29uZmlnLnB5) | `50.98% <0%> (-23.53%)` | :arrow_down: | | [...rflow/contrib/operators/kubernetes\_pod\_operator.py](https://codecov.io/gh/apache/airflow/pull/7136/diff?src=pr=tree#diff-YWlyZmxvdy9jb250cmliL29wZXJhdG9ycy9rdWJlcm5ldGVzX3BvZF9vcGVyYXRvci5weQ==) | `78.31% <0%> (-20.49%)` | :arrow_down: | | [airflow/jobs/backfill\_job.py](https://codecov.io/gh/apache/airflow/pull/7136/diff?src=pr=tree#diff-YWlyZmxvdy9qb2JzL2JhY2tmaWxsX2pvYi5weQ==) | `91.59% <0%> (-0.29%)` | :arrow_down: | -- [Continue to review full report at Codecov](https://codecov.io/gh/apache/airflow/pull/7136?src=pr=continue). > **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta) > `Δ = absolute (impact)`, `ø = not affected`, `? = missing data` > Powered by [Codecov](https://codecov.io/gh/apache/airflow/pull/7136?src=pr=footer). Last update [a676f15...f69ea52](https://codecov.io/gh/apache/airflow/pull/7136?src=pr=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments). This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Commented] (AIRFLOW-5704) Docker scripts for kind kubernetes tests can be improved
[ https://issues.apache.org/jira/browse/AIRFLOW-5704?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17013514#comment-17013514 ] ASF GitHub Bot commented on AIRFLOW-5704: - potiuk commented on pull request #6516: [AIRFLOW-5704] Improve Kind Kubernetes scripts for local testing URL: https://github.com/apache/airflow/pull/6516 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org > Docker scripts for kind kubernetes tests can be improved > > > Key: AIRFLOW-5704 > URL: https://issues.apache.org/jira/browse/AIRFLOW-5704 > Project: Apache Airflow > Issue Type: Sub-task > Components: breeze, ci, dependencies >Affects Versions: 2.0.0 >Reporter: Jarek Potiuk >Assignee: Jarek Potiuk >Priority: Major > Fix For: 2.0.0 > > > The docker CI image for kind tests can be improved > > * Kubernetes Version and all the installation of docker + kubectl + kind can > be added back > * Running kubernetes scripts should be possible from within breeze without > special "kubernetes" environment > * --env breeze switch should be removed > * "bare" environment should be replaced by --no-deps switch > * ENV variable should disappear > > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (AIRFLOW-5704) Docker scripts for kind kubernetes tests can be improved
[ https://issues.apache.org/jira/browse/AIRFLOW-5704?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17013515#comment-17013515 ] ASF subversion and git services commented on AIRFLOW-5704: -- Commit 73403cc8f430b540db492afc5ca3d0664c0ca98b in airflow's branch refs/heads/master from Jarek Potiuk [ https://gitbox.apache.org/repos/asf?p=airflow.git;h=73403cc ] [AIRFLOW-5704] Improve Kind Kubernetes scripts for local testing (#6516) * Fixed problem that Kubernetes tests were testing latest master rather than what came from the local sources. * Kind (Kubernetes in Dcocker) is run in the same Docker as Breeze env * Moved Kubernetes scripts to 'in_container' dir where they belong now * Kubernetes cluster is reused until it is stopped * Kubernetes image is build from image in docker already + mounted sources * Kubectl version name is corrected in the Dockerfile * KUBERNETES_VERSION can now be used to select Kubernetes version * Running kubernetes scripts is now easy in Breeze * We can start/recreate/stop cluster using ---kind-cluster * Instructions on how to run Kubernetes tests are updated * The old "bare" environment is replaced by --no-deps switch > Docker scripts for kind kubernetes tests can be improved > > > Key: AIRFLOW-5704 > URL: https://issues.apache.org/jira/browse/AIRFLOW-5704 > Project: Apache Airflow > Issue Type: Sub-task > Components: breeze, ci, dependencies >Affects Versions: 2.0.0 >Reporter: Jarek Potiuk >Assignee: Jarek Potiuk >Priority: Major > Fix For: 2.0.0 > > > The docker CI image for kind tests can be improved > > * Kubernetes Version and all the installation of docker + kubectl + kind can > be added back > * Running kubernetes scripts should be possible from within breeze without > special "kubernetes" environment > * --env breeze switch should be removed > * "bare" environment should be replaced by --no-deps switch > * ENV variable should disappear > > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[GitHub] [airflow] codecov-io commented on issue #7136: [AIRFLOW-6536] Make job_id parameter optional
codecov-io commented on issue #7136: [AIRFLOW-6536] Make job_id parameter optional URL: https://github.com/apache/airflow/pull/7136#issuecomment-573326191 # [Codecov](https://codecov.io/gh/apache/airflow/pull/7136?src=pr=h1) Report > Merging [#7136](https://codecov.io/gh/apache/airflow/pull/7136?src=pr=desc) into [master](https://codecov.io/gh/apache/airflow/commit/a676f159d4fb01f6e35baa04efa4914a695cd788?src=pr=desc) will **decrease** coverage by `0.28%`. > The diff coverage is `n/a`. [](https://codecov.io/gh/apache/airflow/pull/7136?src=pr=tree) ```diff @@Coverage Diff @@ ## master#7136 +/- ## == - Coverage 85.24% 84.95% -0.29% == Files 683 683 Lines 3915539155 == - Hits3337833266 -112 - Misses 5777 5889 +112 ``` | [Impacted Files](https://codecov.io/gh/apache/airflow/pull/7136?src=pr=tree) | Coverage Δ | | |---|---|---| | [airflow/contrib/operators/databricks\_operator.py](https://codecov.io/gh/apache/airflow/pull/7136/diff?src=pr=tree#diff-YWlyZmxvdy9jb250cmliL29wZXJhdG9ycy9kYXRhYnJpY2tzX29wZXJhdG9yLnB5) | `92.24% <ø> (ø)` | :arrow_up: | | [airflow/kubernetes/volume\_mount.py](https://codecov.io/gh/apache/airflow/pull/7136/diff?src=pr=tree#diff-YWlyZmxvdy9rdWJlcm5ldGVzL3ZvbHVtZV9tb3VudC5weQ==) | `44.44% <0%> (-55.56%)` | :arrow_down: | | [airflow/kubernetes/volume.py](https://codecov.io/gh/apache/airflow/pull/7136/diff?src=pr=tree#diff-YWlyZmxvdy9rdWJlcm5ldGVzL3ZvbHVtZS5weQ==) | `52.94% <0%> (-47.06%)` | :arrow_down: | | [airflow/kubernetes/pod\_launcher.py](https://codecov.io/gh/apache/airflow/pull/7136/diff?src=pr=tree#diff-YWlyZmxvdy9rdWJlcm5ldGVzL3BvZF9sYXVuY2hlci5weQ==) | `45.25% <0%> (-46.72%)` | :arrow_down: | | [airflow/kubernetes/refresh\_config.py](https://codecov.io/gh/apache/airflow/pull/7136/diff?src=pr=tree#diff-YWlyZmxvdy9rdWJlcm5ldGVzL3JlZnJlc2hfY29uZmlnLnB5) | `50.98% <0%> (-23.53%)` | :arrow_down: | | [...rflow/contrib/operators/kubernetes\_pod\_operator.py](https://codecov.io/gh/apache/airflow/pull/7136/diff?src=pr=tree#diff-YWlyZmxvdy9jb250cmliL29wZXJhdG9ycy9rdWJlcm5ldGVzX3BvZF9vcGVyYXRvci5weQ==) | `78.31% <0%> (-20.49%)` | :arrow_down: | | [airflow/jobs/backfill\_job.py](https://codecov.io/gh/apache/airflow/pull/7136/diff?src=pr=tree#diff-YWlyZmxvdy9qb2JzL2JhY2tmaWxsX2pvYi5weQ==) | `91.59% <0%> (-0.29%)` | :arrow_down: | -- [Continue to review full report at Codecov](https://codecov.io/gh/apache/airflow/pull/7136?src=pr=continue). > **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta) > `Δ = absolute (impact)`, `ø = not affected`, `? = missing data` > Powered by [Codecov](https://codecov.io/gh/apache/airflow/pull/7136?src=pr=footer). Last update [a676f15...f69ea52](https://codecov.io/gh/apache/airflow/pull/7136?src=pr=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments). This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [airflow] potiuk merged pull request #6516: [AIRFLOW-5704] Improve Kind Kubernetes scripts for local testing
potiuk merged pull request #6516: [AIRFLOW-5704] Improve Kind Kubernetes scripts for local testing URL: https://github.com/apache/airflow/pull/6516 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Commented] (AIRFLOW-5912) Expose lineage through REST
[ https://issues.apache.org/jira/browse/AIRFLOW-5912?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17013513#comment-17013513 ] ASF GitHub Bot commented on AIRFLOW-5912: - bolkedebruin commented on pull request #7138: [AIRFLOW-5912] Expose lineage API URL: https://github.com/apache/airflow/pull/7138 Only lineage items obtained from XCom were rendered rather than all. Additionally source tasks are recorded. --- Issue link: WILL BE INSERTED BY [boring-cyborg](https://github.com/kaxil/boring-cyborg) - [X] Description above provides context of the change - [X] Commit message/PR title starts with `[AIRFLOW-]`. AIRFLOW- = JIRA ID* - [X] Unit tests coverage for changes (not needed for documentation changes) - [X] Commits follow "[How to write a good git commit message](http://chris.beams.io/posts/git-commit/)" - [X] Relevant documentation is updated including usage instructions. - [X] I will engage committers as explained in [Contribution Workflow Example](https://github.com/apache/airflow/blob/master/CONTRIBUTING.rst#contribution-workflow-example). * For document-only changes commit message can start with `[AIRFLOW-]`. --- In case of fundamental code change, Airflow Improvement Proposal ([AIP](https://cwiki.apache.org/confluence/display/AIRFLOW/Airflow+Improvements+Proposals)) is needed. In case of a new dependency, check compliance with the [ASF 3rd Party License Policy](https://www.apache.org/legal/resolved.html#category-x). In case of backwards incompatible changes please leave a note in [UPDATING.md](https://github.com/apache/airflow/blob/master/UPDATING.md). Read the [Pull Request Guidelines](https://github.com/apache/airflow/blob/master/CONTRIBUTING.rst#pull-request-guidelines) for more information. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org > Expose lineage through REST > --- > > Key: AIRFLOW-5912 > URL: https://issues.apache.org/jira/browse/AIRFLOW-5912 > Project: Apache Airflow > Issue Type: Sub-task > Components: lineage >Affects Versions: 1.10.6 >Reporter: Bolke de Bruin >Priority: Major > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[GitHub] [airflow] bolkedebruin opened a new pull request #7138: [AIRFLOW-5912] Expose lineage API
bolkedebruin opened a new pull request #7138: [AIRFLOW-5912] Expose lineage API URL: https://github.com/apache/airflow/pull/7138 Only lineage items obtained from XCom were rendered rather than all. Additionally source tasks are recorded. --- Issue link: WILL BE INSERTED BY [boring-cyborg](https://github.com/kaxil/boring-cyborg) - [X] Description above provides context of the change - [X] Commit message/PR title starts with `[AIRFLOW-]`. AIRFLOW- = JIRA ID* - [X] Unit tests coverage for changes (not needed for documentation changes) - [X] Commits follow "[How to write a good git commit message](http://chris.beams.io/posts/git-commit/)" - [X] Relevant documentation is updated including usage instructions. - [X] I will engage committers as explained in [Contribution Workflow Example](https://github.com/apache/airflow/blob/master/CONTRIBUTING.rst#contribution-workflow-example). * For document-only changes commit message can start with `[AIRFLOW-]`. --- In case of fundamental code change, Airflow Improvement Proposal ([AIP](https://cwiki.apache.org/confluence/display/AIRFLOW/Airflow+Improvements+Proposals)) is needed. In case of a new dependency, check compliance with the [ASF 3rd Party License Policy](https://www.apache.org/legal/resolved.html#category-x). In case of backwards incompatible changes please leave a note in [UPDATING.md](https://github.com/apache/airflow/blob/master/UPDATING.md). Read the [Pull Request Guidelines](https://github.com/apache/airflow/blob/master/CONTRIBUTING.rst#pull-request-guidelines) for more information. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Commented] (AIRFLOW-6393) Bug fixes to lineage
[ https://issues.apache.org/jira/browse/AIRFLOW-6393?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17013512#comment-17013512 ] ASF GitHub Bot commented on AIRFLOW-6393: - bolkedebruin commented on pull request #7137: [AIRFLOW-6393] Ensure rendering of all lineage items and record source URL: https://github.com/apache/airflow/pull/7137 Only lineage items obtained from XCom were rendered rather than all. Additionally source tasks are recorded. --- Issue link: WILL BE INSERTED BY [boring-cyborg](https://github.com/kaxil/boring-cyborg) - [X] Description above provides context of the change - [X] Commit message/PR title starts with `[AIRFLOW-]`. AIRFLOW- = JIRA ID* - [X] Unit tests coverage for changes (not needed for documentation changes) - [X] Commits follow "[How to write a good git commit message](http://chris.beams.io/posts/git-commit/)" - [X] Relevant documentation is updated including usage instructions. - [X] I will engage committers as explained in [Contribution Workflow Example](https://github.com/apache/airflow/blob/master/CONTRIBUTING.rst#contribution-workflow-example). * For document-only changes commit message can start with `[AIRFLOW-]`. --- In case of fundamental code change, Airflow Improvement Proposal ([AIP](https://cwiki.apache.org/confluence/display/AIRFLOW/Airflow+Improvements+Proposals)) is needed. In case of a new dependency, check compliance with the [ASF 3rd Party License Policy](https://www.apache.org/legal/resolved.html#category-x). In case of backwards incompatible changes please leave a note in [UPDATING.md](https://github.com/apache/airflow/blob/master/UPDATING.md). Read the [Pull Request Guidelines](https://github.com/apache/airflow/blob/master/CONTRIBUTING.rst#pull-request-guidelines) for more information. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org > Bug fixes to lineage > > > Key: AIRFLOW-6393 > URL: https://issues.apache.org/jira/browse/AIRFLOW-6393 > Project: Apache Airflow > Issue Type: Sub-task > Components: lineage >Affects Versions: master >Reporter: Bolke de Bruin >Priority: Major > Fix For: 2.0.0 > > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[GitHub] [airflow] bolkedebruin opened a new pull request #7137: [AIRFLOW-6393] Ensure rendering of all lineage items and record source
bolkedebruin opened a new pull request #7137: [AIRFLOW-6393] Ensure rendering of all lineage items and record source URL: https://github.com/apache/airflow/pull/7137 Only lineage items obtained from XCom were rendered rather than all. Additionally source tasks are recorded. --- Issue link: WILL BE INSERTED BY [boring-cyborg](https://github.com/kaxil/boring-cyborg) - [X] Description above provides context of the change - [X] Commit message/PR title starts with `[AIRFLOW-]`. AIRFLOW- = JIRA ID* - [X] Unit tests coverage for changes (not needed for documentation changes) - [X] Commits follow "[How to write a good git commit message](http://chris.beams.io/posts/git-commit/)" - [X] Relevant documentation is updated including usage instructions. - [X] I will engage committers as explained in [Contribution Workflow Example](https://github.com/apache/airflow/blob/master/CONTRIBUTING.rst#contribution-workflow-example). * For document-only changes commit message can start with `[AIRFLOW-]`. --- In case of fundamental code change, Airflow Improvement Proposal ([AIP](https://cwiki.apache.org/confluence/display/AIRFLOW/Airflow+Improvements+Proposals)) is needed. In case of a new dependency, check compliance with the [ASF 3rd Party License Policy](https://www.apache.org/legal/resolved.html#category-x). In case of backwards incompatible changes please leave a note in [UPDATING.md](https://github.com/apache/airflow/blob/master/UPDATING.md). Read the [Pull Request Guidelines](https://github.com/apache/airflow/blob/master/CONTRIBUTING.rst#pull-request-guidelines) for more information. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [airflow] yuqian90 commented on a change in pull request #7038: [AIRFLOW-4495] allow externally triggered dags to run for future exec dates
yuqian90 commented on a change in pull request #7038: [AIRFLOW-4495] allow
externally triggered dags to run for future exec dates
URL: https://github.com/apache/airflow/pull/7038#discussion_r365525918
##
File path: tests/ti_deps/deps/test_runnable_exec_date_dep.py
##
@@ -37,14 +37,14 @@ def _get_task_instance(self, execution_date,
dag_end_date=None, task_end_date=No
@freeze_time('2016-01-01')
def test_exec_date_after_end_date(self):
"""
-If the dag's execution date is in the future this dep should fail
+If the dag's execution date is in the future this dep should succeed
"""
ti = self._get_task_instance(
dag_end_date=datetime(2016, 1, 3),
task_end_date=datetime(2016, 1, 3),
execution_date=datetime(2016, 1, 2),
)
-self.assertFalse(RunnableExecDateDep().is_met(ti=ti))
+self.assertTrue(RunnableExecDateDep().is_met(ti=ti))
Review comment:
@tooptoop4 , in some sense what you discovered is a flaw in the test itself.
Since the dag in the test is a Mock object, `bool(dag.allow_future_exec_dates)`
is always True (that's how Mock objects behave).
So you can easily fix the test, and also like @kaxil suggested, you should
add a test for the scenarios where allow_future_exec_dates is True and False.
Here's what I suggest. In the following snippet,
`test_exec_date_after_end_date` is what the original test tries to test. You
should also add `test_exec_date_after_end_date_allow_future_exec_dates` which
sets `allow_future_exec_dates` to True to test the new scenario your PR
creates. Note that difference in the comments and in the assertTrue vs
assertFalse.
```python
class TestRunnableExecDateDep(unittest.TestCase):
def _get_task_instance(self, execution_date, dag_end_date=None,
task_end_date=None,
allow_future_exec_dates=False):
dag = Mock(end_date=dag_end_date)
dag.allow_future_exec_dates = allow_future_exec_dates
task = Mock(dag=dag, end_date=task_end_date)
return TaskInstance(task=task, execution_date=execution_date)
@freeze_time('2016-01-01')
def test_exec_date_after_end_date(self):
"""
If the dag's execution date is in the future this dep should fail
"""
ti = self._get_task_instance(
dag_end_date=datetime(2016, 1, 3),
task_end_date=datetime(2016, 1, 3),
execution_date=datetime(2016, 1, 2),
)
self.assertFalse(RunnableExecDateDep().is_met(ti=ti))
@freeze_time('2016-01-01')
def test_exec_date_after_end_date_allow_future_exec_dates(self):
"""
If the dag's execution date is in the future and
allow_future_exec_dates is True,
this dep should pass
"""
ti = self._get_task_instance(
dag_end_date=datetime(2016, 1, 3),
task_end_date=datetime(2016, 1, 3),
execution_date=datetime(2016, 1, 2),
allow_future_exec_dates=True
)
self.assertTrue(RunnableExecDateDep().is_met(ti=ti))
```
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[jira] [Commented] (AIRFLOW-4385) Add namespace into pod's yaml
[ https://issues.apache.org/jira/browse/AIRFLOW-4385?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17013507#comment-17013507 ] ASF GitHub Bot commented on AIRFLOW-4385: - stale[bot] commented on pull request #5152: [AIRFLOW-4385] Add namespace into pod's yaml URL: https://github.com/apache/airflow/pull/5152 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org > Add namespace into pod's yaml > - > > Key: AIRFLOW-4385 > URL: https://issues.apache.org/jira/browse/AIRFLOW-4385 > Project: Apache Airflow > Issue Type: Improvement > Components: executors, worker >Reporter: Nikolay >Assignee: Nikolay >Priority: Major > Labels: kubernetes > Attachments: pods_logs.txt > > > During integration phase between Airflow and VerticalPodAutoscaler (link: > [https://github.com/kubernetes/autoscaler/tree/master/vertical-pod-autoscaler]) > i stumbled into an issue when VerticalPodAutoscaler library doesn't work for > Airflow worker pods. > How VerticalPodAutoscaler produces resource recommendations: > - monitor newly created pods > - compare newly created pods with existing configuration (compare by label > and namespace) > - provide new recommendations if configuration exists for pod > VerticalPodAutoscaler finds all created pods by label, but namespace > comparison fails every time. I did some investigation and realise that when > airflow calls a kubernetes api, it passes yaml file without > +metadata.namespace+ property. Go libraries can deserialise this yaml, but > there is no information about namespace. > Attaching logs from VerticalPodAutoscaler, there you can see how airflow pod > was deserialised (namespace is missing). Works for custom pod. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[GitHub] [airflow] stale[bot] closed pull request #5152: [AIRFLOW-4385] Add namespace into pod's yaml
stale[bot] closed pull request #5152: [AIRFLOW-4385] Add namespace into pod's yaml URL: https://github.com/apache/airflow/pull/5152 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [airflow] mustafagok opened a new pull request #7136: [AIRFLOW-6467] Make job_id parameter optional
mustafagok opened a new pull request #7136: [AIRFLOW-6467] Make job_id parameter optional URL: https://github.com/apache/airflow/pull/7136 job_id parameter should be optional because it can be passed in json(dict) parameter. --- Issue link: WILL BE INSERTED BY [boring-cyborg](https://github.com/kaxil/boring-cyborg) - [x] Description above provides context of the change - [x] Commit message/PR title starts with `[AIRFLOW-]`. AIRFLOW- = JIRA ID* - [x] Unit tests coverage for changes (not needed for documentation changes) - [x] Commits follow "[How to write a good git commit message](http://chris.beams.io/posts/git-commit/)" - [x] Relevant documentation is updated including usage instructions. - [x] I will engage committers as explained in [Contribution Workflow Example](https://github.com/apache/airflow/blob/master/CONTRIBUTING.rst#contribution-workflow-example). * For document-only changes commit message can start with `[AIRFLOW-]`. --- In case of fundamental code change, Airflow Improvement Proposal ([AIP](https://cwiki.apache.org/confluence/display/AIRFLOW/Airflow+Improvements+Proposals)) is needed. In case of a new dependency, check compliance with the [ASF 3rd Party License Policy](https://www.apache.org/legal/resolved.html#category-x). In case of backwards incompatible changes please leave a note in [UPDATING.md](https://github.com/apache/airflow/blob/master/UPDATING.md). Read the [Pull Request Guidelines](https://github.com/apache/airflow/blob/master/CONTRIBUTING.rst#pull-request-guidelines) for more information. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Commented] (AIRFLOW-6467) Remove unnecessary local variable
[ https://issues.apache.org/jira/browse/AIRFLOW-6467?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17013506#comment-17013506 ] ASF GitHub Bot commented on AIRFLOW-6467: - mustafagok commented on pull request #7136: [AIRFLOW-6467] Make job_id parameter optional URL: https://github.com/apache/airflow/pull/7136 job_id parameter should be optional because it can be passed in json(dict) parameter. --- Issue link: WILL BE INSERTED BY [boring-cyborg](https://github.com/kaxil/boring-cyborg) - [x] Description above provides context of the change - [x] Commit message/PR title starts with `[AIRFLOW-]`. AIRFLOW- = JIRA ID* - [x] Unit tests coverage for changes (not needed for documentation changes) - [x] Commits follow "[How to write a good git commit message](http://chris.beams.io/posts/git-commit/)" - [x] Relevant documentation is updated including usage instructions. - [x] I will engage committers as explained in [Contribution Workflow Example](https://github.com/apache/airflow/blob/master/CONTRIBUTING.rst#contribution-workflow-example). * For document-only changes commit message can start with `[AIRFLOW-]`. --- In case of fundamental code change, Airflow Improvement Proposal ([AIP](https://cwiki.apache.org/confluence/display/AIRFLOW/Airflow+Improvements+Proposals)) is needed. In case of a new dependency, check compliance with the [ASF 3rd Party License Policy](https://www.apache.org/legal/resolved.html#category-x). In case of backwards incompatible changes please leave a note in [UPDATING.md](https://github.com/apache/airflow/blob/master/UPDATING.md). Read the [Pull Request Guidelines](https://github.com/apache/airflow/blob/master/CONTRIBUTING.rst#pull-request-guidelines) for more information. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org > Remove unnecessary local variable > - > > Key: AIRFLOW-6467 > URL: https://issues.apache.org/jira/browse/AIRFLOW-6467 > Project: Apache Airflow > Issue Type: Test > Components: tests >Affects Versions: 1.10.7 >Reporter: Mustafa Gök >Assignee: Mustafa Gök >Priority: Trivial > Fix For: 1.10.8 > > > In test_templated_sensor (tests/sensors/test_external_task_sensor.py) unit > test method, self.dag can be used instead of creating a local variable named > "dag" (line 98). -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work started] (AIRFLOW-6536) Make "job_id" parameter of the DatabricksRunNowOperator optional
[
https://issues.apache.org/jira/browse/AIRFLOW-6536?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Work on AIRFLOW-6536 started by Mustafa Gök.
> Make "job_id" parameter of the DatabricksRunNowOperator optional
>
>
> Key: AIRFLOW-6536
> URL: https://issues.apache.org/jira/browse/AIRFLOW-6536
> Project: Apache Airflow
> Issue Type: Improvement
> Components: contrib
>Affects Versions: 1.10.7
>Reporter: Mustafa Gök
>Assignee: Mustafa Gök
>Priority: Minor
>
> "job_id" parameter should be optional because it can be passed in json(dict)
> parameter.
> line 317 (example in docstring, but it gives error):
> {code:python}
> notebook_run = DatabricksRunNowOperator(task_id='notebook_run', json=json)
> {code}
> line 458:
> {code:python}
> if job_id is not None:
> self.json['job_id'] = job_id
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Created] (AIRFLOW-6536) Make "job_id" parameter of the DatabricksRunNowOperator optional
Mustafa Gök created AIRFLOW-6536:
Summary: Make "job_id" parameter of the DatabricksRunNowOperator
optional
Key: AIRFLOW-6536
URL: https://issues.apache.org/jira/browse/AIRFLOW-6536
Project: Apache Airflow
Issue Type: Improvement
Components: contrib
Affects Versions: 1.10.7
Reporter: Mustafa Gök
Assignee: Mustafa Gök
"job_id" parameter should be optional because it can be passed in json(dict)
parameter.
line 317 (example in docstring, but it gives error):
{code:python}
notebook_run = DatabricksRunNowOperator(task_id='notebook_run', json=json)
{code}
line 458:
{code:python}
if job_id is not None:
self.json['job_id'] = job_id
{code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[GitHub] [airflow] codecov-io edited a comment on issue #7133: [AIRFLOW-6535] add exception to fail without retry
codecov-io edited a comment on issue #7133: [AIRFLOW-6535] add exception to fail without retry URL: https://github.com/apache/airflow/pull/7133#issuecomment-573236471 # [Codecov](https://codecov.io/gh/apache/airflow/pull/7133?src=pr=h1) Report > Merging [#7133](https://codecov.io/gh/apache/airflow/pull/7133?src=pr=desc) into [master](https://codecov.io/gh/apache/airflow/commit/86f525f6527e45c9ff1b8f844569c4d9d22bccac?src=pr=desc) will **decrease** coverage by `0.29%`. > The diff coverage is `58.82%`. [](https://codecov.io/gh/apache/airflow/pull/7133?src=pr=tree) ```diff @@Coverage Diff@@ ## master#7133 +/- ## = - Coverage 85.24% 84.95% -0.3% = Files 683 683 Lines 3915639167 +11 = - Hits3337933273-106 - Misses 5777 5894+117 ``` | [Impacted Files](https://codecov.io/gh/apache/airflow/pull/7133?src=pr=tree) | Coverage Δ | | |---|---|---| | [airflow/exceptions.py](https://codecov.io/gh/apache/airflow/pull/7133/diff?src=pr=tree#diff-YWlyZmxvdy9leGNlcHRpb25zLnB5) | `100% <100%> (ø)` | :arrow_up: | | [airflow/models/taskinstance.py](https://codecov.io/gh/apache/airflow/pull/7133/diff?src=pr=tree#diff-YWlyZmxvdy9tb2RlbHMvdGFza2luc3RhbmNlLnB5) | `94.15% <56.25%> (-0.78%)` | :arrow_down: | | [airflow/kubernetes/volume\_mount.py](https://codecov.io/gh/apache/airflow/pull/7133/diff?src=pr=tree#diff-YWlyZmxvdy9rdWJlcm5ldGVzL3ZvbHVtZV9tb3VudC5weQ==) | `44.44% <0%> (-55.56%)` | :arrow_down: | | [airflow/kubernetes/volume.py](https://codecov.io/gh/apache/airflow/pull/7133/diff?src=pr=tree#diff-YWlyZmxvdy9rdWJlcm5ldGVzL3ZvbHVtZS5weQ==) | `52.94% <0%> (-47.06%)` | :arrow_down: | | [airflow/kubernetes/pod\_launcher.py](https://codecov.io/gh/apache/airflow/pull/7133/diff?src=pr=tree#diff-YWlyZmxvdy9rdWJlcm5ldGVzL3BvZF9sYXVuY2hlci5weQ==) | `45.25% <0%> (-46.72%)` | :arrow_down: | | [airflow/kubernetes/refresh\_config.py](https://codecov.io/gh/apache/airflow/pull/7133/diff?src=pr=tree#diff-YWlyZmxvdy9rdWJlcm5ldGVzL3JlZnJlc2hfY29uZmlnLnB5) | `50.98% <0%> (-23.53%)` | :arrow_down: | | [...rflow/contrib/operators/kubernetes\_pod\_operator.py](https://codecov.io/gh/apache/airflow/pull/7133/diff?src=pr=tree#diff-YWlyZmxvdy9jb250cmliL29wZXJhdG9ycy9rdWJlcm5ldGVzX3BvZF9vcGVyYXRvci5weQ==) | `78.31% <0%> (-20.49%)` | :arrow_down: | | [airflow/kubernetes/worker\_configuration.py](https://codecov.io/gh/apache/airflow/pull/7133/diff?src=pr=tree#diff-YWlyZmxvdy9rdWJlcm5ldGVzL3dvcmtlcl9jb25maWd1cmF0aW9uLnB5) | `99.28% <0%> (-0.01%)` | :arrow_down: | -- [Continue to review full report at Codecov](https://codecov.io/gh/apache/airflow/pull/7133?src=pr=continue). > **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta) > `Δ = absolute (impact)`, `ø = not affected`, `? = missing data` > Powered by [Codecov](https://codecov.io/gh/apache/airflow/pull/7133?src=pr=footer). Last update [86f525f...8f2e836](https://codecov.io/gh/apache/airflow/pull/7133?src=pr=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments). This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [airflow] codecov-io edited a comment on issue #7133: [AIRFLOW-6535] add exception to fail without retry
codecov-io edited a comment on issue #7133: [AIRFLOW-6535] add exception to fail without retry URL: https://github.com/apache/airflow/pull/7133#issuecomment-573236471 # [Codecov](https://codecov.io/gh/apache/airflow/pull/7133?src=pr=h1) Report > Merging [#7133](https://codecov.io/gh/apache/airflow/pull/7133?src=pr=desc) into [master](https://codecov.io/gh/apache/airflow/commit/28d8ef1eff901cce36e6c3fa53b944a1cf7fa00a?src=pr=desc) will **decrease** coverage by `0.22%`. > The diff coverage is `58.82%`. [](https://codecov.io/gh/apache/airflow/pull/7133?src=pr=tree) ```diff @@Coverage Diff @@ ## master#7133 +/- ## == - Coverage 85.17% 84.95% -0.23% == Files 683 683 Lines 3915539167 +12 == - Hits3335233273 -79 - Misses 5803 5894 +91 ``` | [Impacted Files](https://codecov.io/gh/apache/airflow/pull/7133?src=pr=tree) | Coverage Δ | | |---|---|---| | [airflow/exceptions.py](https://codecov.io/gh/apache/airflow/pull/7133/diff?src=pr=tree#diff-YWlyZmxvdy9leGNlcHRpb25zLnB5) | `100% <100%> (ø)` | :arrow_up: | | [airflow/models/taskinstance.py](https://codecov.io/gh/apache/airflow/pull/7133/diff?src=pr=tree#diff-YWlyZmxvdy9tb2RlbHMvdGFza2luc3RhbmNlLnB5) | `94.15% <56.25%> (-0.78%)` | :arrow_down: | | [airflow/kubernetes/volume\_mount.py](https://codecov.io/gh/apache/airflow/pull/7133/diff?src=pr=tree#diff-YWlyZmxvdy9rdWJlcm5ldGVzL3ZvbHVtZV9tb3VudC5weQ==) | `44.44% <0%> (-55.56%)` | :arrow_down: | | [airflow/kubernetes/volume.py](https://codecov.io/gh/apache/airflow/pull/7133/diff?src=pr=tree#diff-YWlyZmxvdy9rdWJlcm5ldGVzL3ZvbHVtZS5weQ==) | `52.94% <0%> (-47.06%)` | :arrow_down: | | [airflow/kubernetes/pod\_launcher.py](https://codecov.io/gh/apache/airflow/pull/7133/diff?src=pr=tree#diff-YWlyZmxvdy9rdWJlcm5ldGVzL3BvZF9sYXVuY2hlci5weQ==) | `45.25% <0%> (-46.72%)` | :arrow_down: | | [airflow/kubernetes/refresh\_config.py](https://codecov.io/gh/apache/airflow/pull/7133/diff?src=pr=tree#diff-YWlyZmxvdy9rdWJlcm5ldGVzL3JlZnJlc2hfY29uZmlnLnB5) | `50.98% <0%> (-23.53%)` | :arrow_down: | | [...rflow/contrib/operators/kubernetes\_pod\_operator.py](https://codecov.io/gh/apache/airflow/pull/7133/diff?src=pr=tree#diff-YWlyZmxvdy9jb250cmliL29wZXJhdG9ycy9rdWJlcm5ldGVzX3BvZF9vcGVyYXRvci5weQ==) | `78.31% <0%> (-20.49%)` | :arrow_down: | | [airflow/hooks/postgres\_hook.py](https://codecov.io/gh/apache/airflow/pull/7133/diff?src=pr=tree#diff-YWlyZmxvdy9ob29rcy9wb3N0Z3Jlc19ob29rLnB5) | `94.36% <0%> (+1.4%)` | :arrow_up: | | [airflow/hooks/dbapi\_hook.py](https://codecov.io/gh/apache/airflow/pull/7133/diff?src=pr=tree#diff-YWlyZmxvdy9ob29rcy9kYmFwaV9ob29rLnB5) | `91.73% <0%> (+1.65%)` | :arrow_up: | | [airflow/utils/sqlalchemy.py](https://codecov.io/gh/apache/airflow/pull/7133/diff?src=pr=tree#diff-YWlyZmxvdy91dGlscy9zcWxhbGNoZW15LnB5) | `96.66% <0%> (+1.66%)` | :arrow_up: | | ... and [1 more](https://codecov.io/gh/apache/airflow/pull/7133/diff?src=pr=tree-more) | | -- [Continue to review full report at Codecov](https://codecov.io/gh/apache/airflow/pull/7133?src=pr=continue). > **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta) > `Δ = absolute (impact)`, `ø = not affected`, `? = missing data` > Powered by [Codecov](https://codecov.io/gh/apache/airflow/pull/7133?src=pr=footer). Last update [28d8ef1...8f2e836](https://codecov.io/gh/apache/airflow/pull/7133?src=pr=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments). This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [airflow] potiuk commented on issue #6516: [AIRFLOW-5704] Improve Kind Kubernetes scripts for local testing
potiuk commented on issue #6516: [AIRFLOW-5704] Improve Kind Kubernetes scripts for local testing URL: https://github.com/apache/airflow/pull/6516#issuecomment-573320784 @nuclearpinguin @kaxil @mik-laj - one last final review and I'd love to merge it This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [airflow] codecov-io edited a comment on issue #6516: [AIRFLOW-5704] Improve Kind Kubernetes scripts for local testing
codecov-io edited a comment on issue #6516: [AIRFLOW-5704] Improve Kind Kubernetes scripts for local testing URL: https://github.com/apache/airflow/pull/6516#issuecomment-572851732 # [Codecov](https://codecov.io/gh/apache/airflow/pull/6516?src=pr=h1) Report > Merging [#6516](https://codecov.io/gh/apache/airflow/pull/6516?src=pr=desc) into [master](https://codecov.io/gh/apache/airflow/commit/a676f159d4fb01f6e35baa04efa4914a695cd788?src=pr=desc) will **not change** coverage. > The diff coverage is `n/a`. [](https://codecov.io/gh/apache/airflow/pull/6516?src=pr=tree) ```diff @@ Coverage Diff @@ ## master#6516 +/- ## === Coverage 85.24% 85.24% === Files 683 683 Lines 3915539155 === Hits3337833378 Misses 5777 5777 ``` -- [Continue to review full report at Codecov](https://codecov.io/gh/apache/airflow/pull/6516?src=pr=continue). > **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta) > `Δ = absolute (impact)`, `ø = not affected`, `? = missing data` > Powered by [Codecov](https://codecov.io/gh/apache/airflow/pull/6516?src=pr=footer). Last update [a676f15...bed4dcb](https://codecov.io/gh/apache/airflow/pull/6516?src=pr=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments). This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [airflow] codecov-io edited a comment on issue #6516: [AIRFLOW-5704] Improve Kind Kubernetes scripts for local testing
codecov-io edited a comment on issue #6516: [AIRFLOW-5704] Improve Kind Kubernetes scripts for local testing URL: https://github.com/apache/airflow/pull/6516#issuecomment-572851732 # [Codecov](https://codecov.io/gh/apache/airflow/pull/6516?src=pr=h1) Report > Merging [#6516](https://codecov.io/gh/apache/airflow/pull/6516?src=pr=desc) into [master](https://codecov.io/gh/apache/airflow/commit/a676f159d4fb01f6e35baa04efa4914a695cd788?src=pr=desc) will **not change** coverage. > The diff coverage is `n/a`. [](https://codecov.io/gh/apache/airflow/pull/6516?src=pr=tree) ```diff @@ Coverage Diff @@ ## master#6516 +/- ## === Coverage 85.24% 85.24% === Files 683 683 Lines 3915539155 === Hits3337833378 Misses 5777 5777 ``` -- [Continue to review full report at Codecov](https://codecov.io/gh/apache/airflow/pull/6516?src=pr=continue). > **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta) > `Δ = absolute (impact)`, `ø = not affected`, `? = missing data` > Powered by [Codecov](https://codecov.io/gh/apache/airflow/pull/6516?src=pr=footer). Last update [a676f15...bed4dcb](https://codecov.io/gh/apache/airflow/pull/6516?src=pr=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments). This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [airflow] codecov-io edited a comment on issue #6516: [AIRFLOW-5704] Improve Kind Kubernetes scripts for local testing
codecov-io edited a comment on issue #6516: [AIRFLOW-5704] Improve Kind Kubernetes scripts for local testing URL: https://github.com/apache/airflow/pull/6516#issuecomment-572851732 # [Codecov](https://codecov.io/gh/apache/airflow/pull/6516?src=pr=h1) Report > Merging [#6516](https://codecov.io/gh/apache/airflow/pull/6516?src=pr=desc) into [master](https://codecov.io/gh/apache/airflow/commit/a676f159d4fb01f6e35baa04efa4914a695cd788?src=pr=desc) will **not change** coverage. > The diff coverage is `n/a`. [](https://codecov.io/gh/apache/airflow/pull/6516?src=pr=tree) ```diff @@ Coverage Diff @@ ## master#6516 +/- ## === Coverage 85.24% 85.24% === Files 683 683 Lines 3915539155 === Hits3337833378 Misses 5777 5777 ``` -- [Continue to review full report at Codecov](https://codecov.io/gh/apache/airflow/pull/6516?src=pr=continue). > **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta) > `Δ = absolute (impact)`, `ø = not affected`, `? = missing data` > Powered by [Codecov](https://codecov.io/gh/apache/airflow/pull/6516?src=pr=footer). Last update [a676f15...bed4dcb](https://codecov.io/gh/apache/airflow/pull/6516?src=pr=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments). This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
