[airflow] branch constraints-main updated: Updating constraints. Build id:1689622060

2022-01-12 Thread github-bot
This is an automated email from the ASF dual-hosted git repository. github-bot pushed a commit to branch constraints-main in repository https://gitbox.apache.org/repos/asf/airflow.git The following commit(s) were added to refs/heads/constraints-main by this push: new f3dd74f Updating

[GitHub] [airflow] SamWheating commented on issue #20832: Unable to specify Python version for AwsGlueJobOperator

2022-01-12 Thread GitBox
SamWheating commented on issue #20832: URL: https://github.com/apache/airflow/issues/20832#issuecomment-1011511506 Also for what its worth, I think that you `Command` block is invalid, as the `Command.Name` you're using (`abalone-preprocess`) field must be one of `glueetl`, `pythonshell`

[GitHub] [airflow] SamWheating edited a comment on issue #20832: Unable to specify Python version for AwsGlueJobOperator

2022-01-12 Thread GitBox
SamWheating edited a comment on issue #20832: URL: https://github.com/apache/airflow/issues/20832#issuecomment-1011502821 I think that this is because the GlueHook is pretty opinionated and hardcodes the value of `Command` when running the `glue_client.create_job` command:

[GitHub] [airflow] ferruzzi edited a comment on pull request #20819: Adds support for optional kwargs in the EKS Operators

2022-01-12 Thread GitBox
ferruzzi edited a comment on pull request #20819: URL: https://github.com/apache/airflow/pull/20819#issuecomment-1011461226 https://github.com/apache/airflow/pull/20819/commits/6d059d4af4fcaeae0d3980b49d86c73cf42751d7 should address the tag override issue, and

[GitHub] [airflow] SamWheating commented on issue #20832: Unable to specify Python version for AwsGlueJobOperator

2022-01-12 Thread GitBox
SamWheating commented on issue #20832: URL: https://github.com/apache/airflow/issues/20832#issuecomment-1011502821 I think that this is because the GlueHook is pretty opinionated and hardcodes the value of `Command` when running the `create_glue_job` command:

[GitHub] [airflow] josh-fell edited a comment on pull request #20795: Fix remaining mypy issues in "core" Airflow

2022-01-12 Thread GitBox
josh-fell edited a comment on pull request #20795: URL: https://github.com/apache/airflow/pull/20795#issuecomment-1011482747 > > @josh-fell Oh I like https://github.com/apache/airflow/pull/20470/files#diff-af25dd96233a97ace811c614fa7d5bd059cbbc1571e421f6675e16f6290814c5 more than adding

[GitHub] [airflow] josh-fell commented on pull request #20795: Fix remaining mypy issues in "core" Airflow

2022-01-12 Thread GitBox
josh-fell commented on pull request #20795: URL: https://github.com/apache/airflow/pull/20795#issuecomment-1011482747 > > @josh-fell Oh I like https://github.com/apache/airflow/pull/20470/files#diff-af25dd96233a97ace811c614fa7d5bd059cbbc1571e421f6675e16f6290814c5 more than adding all

[GitHub] [airflow] edithturn edited a comment on pull request #20763: [WIP] Verify enough resources for breeze

2022-01-12 Thread GitBox
edithturn edited a comment on pull request #20763: URL: https://github.com/apache/airflow/pull/20763#issuecomment-1011476143 I rebased it :fearful: I am not sure if this is good, because I still have BREEZE.md and the rebase --continue didn't ask me to squash. I need to master

[GitHub] [airflow] edithturn edited a comment on pull request #20763: [WIP] Verify enough resources for breeze

2022-01-12 Thread GitBox
edithturn edited a comment on pull request #20763: URL: https://github.com/apache/airflow/pull/20763#issuecomment-1011476143 I rebased it :fearful: I am not sure if this is good, because I still have BREEZE.md and the rebase --continue didn't ask me to squash. I need to master rebase,

[GitHub] [airflow] edithturn edited a comment on pull request #20763: [WIP] Verify enough resources for breeze

2022-01-12 Thread GitBox
edithturn edited a comment on pull request #20763: URL: https://github.com/apache/airflow/pull/20763#issuecomment-1011476143 I rebased it :fearful: ![image](https://user-images.githubusercontent.com/58795858/149225004-013497d6-2918-4f7c-9e28-0acb194f776a.png), I am not sure if this is

[GitHub] [airflow] edithturn commented on pull request #20763: [WIP] Verify enough resources for breeze

2022-01-12 Thread GitBox
edithturn commented on pull request #20763: URL: https://github.com/apache/airflow/pull/20763#issuecomment-1011476143 I rebased it ![image](https://user-images.githubusercontent.com/58795858/149225004-013497d6-2918-4f7c-9e28-0acb194f776a.png), I am not sure if this is good, because I

[GitHub] [airflow] ferruzzi commented on a change in pull request #20819: Adds support for optional kwargs in the EKS Operators

2022-01-12 Thread GitBox
ferruzzi commented on a change in pull request #20819: URL: https://github.com/apache/airflow/pull/20819#discussion_r783459893 ## File path: airflow/providers/amazon/aws/operators/eks.py ## @@ -89,36 +91,45 @@ class EksCreateClusterOperator(BaseOperator): If this is

[GitHub] [airflow] ferruzzi commented on a change in pull request #20819: Adds support for optional kwargs in the EKS Operators

2022-01-12 Thread GitBox
ferruzzi commented on a change in pull request #20819: URL: https://github.com/apache/airflow/pull/20819#discussion_r783453020 ## File path: airflow/providers/amazon/aws/operators/eks.py ## @@ -89,36 +91,45 @@ class EksCreateClusterOperator(BaseOperator): If this is

[GitHub] [airflow] ferruzzi commented on pull request #20819: Adds support for optional kwargs in the EKS Operators

2022-01-12 Thread GitBox
ferruzzi commented on pull request #20819: URL: https://github.com/apache/airflow/pull/20819#issuecomment-1011461226 [42751d7](https://github.com/apache/airflow/pull/20819/commits/6d059d4af4fcaeae0d3980b49d86c73cf42751d7) should address the tag override issue, and

[GitHub] [airflow] ferruzzi commented on a change in pull request #20819: Adds support for optional kwargs in the EKS Operators

2022-01-12 Thread GitBox
ferruzzi commented on a change in pull request #20819: URL: https://github.com/apache/airflow/pull/20819#discussion_r783436088 ## File path: airflow/providers/amazon/aws/hooks/eks.py ## @@ -135,13 +146,12 @@ def create_nodegroup( :rtype: Dict """

[airflow] branch main updated (aebb7ca -> fe5aba2)

2022-01-12 Thread potiuk
This is an automated email from the ASF dual-hosted git repository. potiuk pushed a change to branch main in repository https://gitbox.apache.org/repos/asf/airflow.git. from aebb7ca Add Helm Chart 1.4.0 in airflow_helmchart_bug_report.yml (#20828) add fe5aba2 Fix grammatical error

[GitHub] [airflow] potiuk merged pull request #20837: Fix grammatical error in Variable.get docstring

2022-01-12 Thread GitBox
potiuk merged pull request #20837: URL: https://github.com/apache/airflow/pull/20837 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

[GitHub] [airflow] boring-cyborg[bot] commented on pull request #20837: Fix grammatical error in Variable.get docstring

2022-01-12 Thread GitBox
boring-cyborg[bot] commented on pull request #20837: URL: https://github.com/apache/airflow/pull/20837#issuecomment-1011433860 Awesome work, congrats on your first merged pull request! -- This is an automated message from the Apache Git Service. To respond to the message, please log

[GitHub] [airflow] github-actions[bot] commented on pull request #20837: Fix grammatical error in Variable.get docstring

2022-01-12 Thread GitBox
github-actions[bot] commented on pull request #20837: URL: https://github.com/apache/airflow/pull/20837#issuecomment-1011429345 The PR most likely needs to run full matrix of tests because it modifies parts of the core of Airflow. However, committers might decide to merge it quickly and

[GitHub] [airflow] boring-cyborg[bot] commented on pull request #20837: Fix grammatical error in Variable.get docstring

2022-01-12 Thread GitBox
boring-cyborg[bot] commented on pull request #20837: URL: https://github.com/apache/airflow/pull/20837#issuecomment-1011423150 Congratulations on your first Pull Request and welcome to the Apache Airflow community! If you have any issues or are unsure about any anything please check our

[GitHub] [airflow] anna-hope opened a new pull request #20837: Fix grammatical error in Variable.get docstring

2022-01-12 Thread GitBox
anna-hope opened a new pull request #20837: URL: https://github.com/apache/airflow/pull/20837 --- **^ Add meaningful description above** Read the **[Pull Request Guidelines](https://github.com/apache/airflow/blob/main/CONTRIBUTING.rst#pull-request-guidelines)** for more

[GitHub] [airflow] potiuk commented on pull request #20763: [WIP] Verify enough resources for breeze

2022-01-12 Thread GitBox
potiuk commented on pull request #20763: URL: https://github.com/apache/airflow/pull/20763#issuecomment-1011417552 Just rebase and I am ready to merge it :) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL

[GitHub] [airflow] potiuk commented on pull request #20763: [WIP] Verify enough resources for breeze

2022-01-12 Thread GitBox
potiuk commented on pull request #20763: URL: https://github.com/apache/airflow/pull/20763#issuecomment-1011417214 > The BREEZE variable is not set. Defaulting to a blank string. > ERROR: for airflow "Container" 00543b681b2e "is unhealthy. No no , that's fine. The tests failed

[GitHub] [airflow] edithturn edited a comment on pull request #20763: [WIP] Verify enough resources for breeze

2022-01-12 Thread GitBox
edithturn edited a comment on pull request #20763: URL: https://github.com/apache/airflow/pull/20763#issuecomment-1011406936 @potiuk this is how this script **run_resource_check.py** is showing the information in the job: ![Screenshot from 2022-01-12

[GitHub] [airflow] edithturn commented on pull request #20763: [WIP] Verify enough resources for breeze

2022-01-12 Thread GitBox
edithturn commented on pull request #20763: URL: https://github.com/apache/airflow/pull/20763#issuecomment-1011406936 @potiuk this si how this script is showing the information with the job: ![Screenshot from 2022-01-12

[GitHub] [airflow] ashb commented on a change in pull request #20743: Serialize mapped tasks and task groups

2022-01-12 Thread GitBox
ashb commented on a change in pull request #20743: URL: https://github.com/apache/airflow/pull/20743#discussion_r783404688 ## File path: airflow/models/baseoperator.py ## @@ -207,6 +207,7 @@ def apply_defaults(self: "BaseOperator", *args: Any, **kwargs: Any) -> Any:

[airflow] branch main updated (7e185ef -> aebb7ca)

2022-01-12 Thread potiuk
This is an automated email from the ASF dual-hosted git repository. potiuk pushed a change to branch main in repository https://gitbox.apache.org/repos/asf/airflow.git. from 7e185ef Integrate Breeze2 documentation together (#20836) add aebb7ca Add Helm Chart 1.4.0 in

[airflow] branch main updated (8f6c921 -> 7e185ef)

2022-01-12 Thread potiuk
This is an automated email from the ASF dual-hosted git repository. potiuk pushed a change to branch main in repository https://gitbox.apache.org/repos/asf/airflow.git. from 8f6c921 Move tests out of test core that are duplicated/should live elsewhere (#20826) add 7e185ef Integrate

[GitHub] [airflow] potiuk merged pull request #20828: Add Helm Chart 1.4.0 in airflow_helmchart_bug_report.yml

2022-01-12 Thread GitBox
potiuk merged pull request #20828: URL: https://github.com/apache/airflow/pull/20828 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

[GitHub] [airflow] potiuk merged pull request #20836: Docs - Integrate Breeze2 documentation

2022-01-12 Thread GitBox
potiuk merged pull request #20836: URL: https://github.com/apache/airflow/pull/20836 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

[GitHub] [airflow] potiuk commented on pull request #20664: Switch to buildx to build airflow images

2022-01-12 Thread GitBox
potiuk commented on pull request #20664: URL: https://github.com/apache/airflow/pull/20664#issuecomment-1011391486 FYI. I will run a few more tests in my fork - just to make sure everything is fine with caching of build-image workflow results and will merge it when I come back from

[GitHub] [airflow] github-actions[bot] commented on pull request #20836: Docs - Integrate Breeze2 documentation

2022-01-12 Thread GitBox
github-actions[bot] commented on pull request #20836: URL: https://github.com/apache/airflow/pull/20836#issuecomment-1011390251 The PR is likely ready to be merged. No tests are needed as no important environment files, nor python files were modified by it. However, committers might

[GitHub] [airflow] edithturn opened a new pull request #20836: pre commits test

2022-01-12 Thread GitBox
edithturn opened a new pull request #20836: URL: https://github.com/apache/airflow/pull/20836 Integrating Breeze2 for Windows with Breeze2 autocomplete on a single Readme. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and

[GitHub] [airflow] potiuk commented on a change in pull request #20722: Speed up creation of DagRun for large DAGs (5k+ tasks) by 25-140%

2022-01-12 Thread GitBox
potiuk commented on a change in pull request #20722: URL: https://github.com/apache/airflow/pull/20722#discussion_r783382420 ## File path: airflow/configuration.py ## @@ -493,6 +493,32 @@ def getimport(self, section, key, **kwargs): f'Current value:

[GitHub] [airflow] potiuk commented on pull request #20795: Fix remaining mypy issues in "core" Airflow

2022-01-12 Thread GitBox
potiuk commented on pull request #20795: URL: https://github.com/apache/airflow/pull/20795#issuecomment-1011376175 > @josh-fell Oh I like https://github.com/apache/airflow/pull/20470/files#diff-af25dd96233a97ace811c614fa7d5bd059cbbc1571e421f6675e16f6290814c5 more than adding all those

[GitHub] [airflow] andriisoldatenko commented on a change in pull request #20722: Speed up creation of DagRun for large DAGs (5k+ tasks) by 25-140%

2022-01-12 Thread GitBox
andriisoldatenko commented on a change in pull request #20722: URL: https://github.com/apache/airflow/pull/20722#discussion_r783372254 ## File path: airflow/configuration.py ## @@ -493,6 +493,32 @@ def getimport(self, section, key, **kwargs): f'Current value:

[GitHub] [airflow-site] potiuk merged pull request #537: Add documentation for packages - 2022-01-08

2022-01-12 Thread GitBox
potiuk merged pull request #537: URL: https://github.com/apache/airflow-site/pull/537 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

[airflow] tag providers-sftp/2.4.1 created (now da9210e)

2022-01-12 Thread potiuk
This is an automated email from the ASF dual-hosted git repository. potiuk pushed a change to tag providers-sftp/2.4.1 in repository https://gitbox.apache.org/repos/asf/airflow.git. at da9210e (commit) No new revisions were added by this update.

[airflow] tag providers-cncf-kubernetes/3.0.1 created (now da9210e)

2022-01-12 Thread potiuk
This is an automated email from the ASF dual-hosted git repository. potiuk pushed a change to tag providers-cncf-kubernetes/3.0.1 in repository https://gitbox.apache.org/repos/asf/airflow.git. at da9210e (commit) No new revisions were added by this update.

[GitHub] [airflow] potiuk commented on pull request #20701: [WIP] Check free space python

2022-01-12 Thread GitBox
potiuk commented on pull request #20701: URL: https://github.com/apache/airflow/pull/20701#issuecomment-1011359233 > We have two jobs ci and build-image: > > * ci job is running `run: airflow-freespace`: https://github.com/apache/airflow/runs/4791100593?check_suite_focus=true, but

[GitHub] [airflow] potiuk commented on pull request #20701: [WIP] Check free space python

2022-01-12 Thread GitBox
potiuk commented on pull request #20701: URL: https://github.com/apache/airflow/pull/20701#issuecomment-1011350316 > From what I understand: "sudo apt-get clean" will finish with a state zero (0), if this finish with a state <> than zero, and if the check is true it will generate an

svn commit: r52044 - /release/airflow/providers/

2022-01-12 Thread potiuk
Author: potiuk Date: Wed Jan 12 18:47:07 2022 New Revision: 52044 Log: Release Airflow Providers on Wed Jan 12 19:47:03 CET 2022 Added: release/airflow/providers/apache-airflow-providers-cncf-kubernetes-3.0.1.tar.gz - copied unchanged from r52043,

[GitHub] [airflow] github-actions[bot] commented on pull request #20798: Add "Greater/Smaller than or Equal" to filters in the browse views

2022-01-12 Thread GitBox
github-actions[bot] commented on pull request #20798: URL: https://github.com/apache/airflow/pull/20798#issuecomment-1011337066 The PR is likely OK to be merged with just subset of tests for default Python and Database versions without running the full matrix of tests, because it does not

[airflow] branch main updated (82e466d -> 8f6c921)

2022-01-12 Thread ash
This is an automated email from the ASF dual-hosted git repository. ash pushed a change to branch main in repository https://gitbox.apache.org/repos/asf/airflow.git. from 82e466d Fix airflow trigger cli (#20781) add 8f6c921 Move tests out of test core that are duplicated/should live

[GitHub] [airflow] ashb merged pull request #20826: Move tests out of test core that are duplicated/should live elsewhere

2022-01-12 Thread GitBox
ashb merged pull request #20826: URL: https://github.com/apache/airflow/pull/20826 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

[GitHub] [airflow] ashb commented on pull request #20826: Move tests out of test core that are duplicated/should live elsewhere

2022-01-12 Thread GitBox
ashb commented on pull request #20826: URL: https://github.com/apache/airflow/pull/20826#issuecomment-1011331183 One flaky DASK test on a single matrix: ``` tests/executors/test_dask_executor.py::TestDaskExecutorQueue::test_dask_queues: ValueError: The futures should have

[GitHub] [airflow] turbaszek commented on a change in pull request #20530: Add sensor decorator

2022-01-12 Thread GitBox
turbaszek commented on a change in pull request #20530: URL: https://github.com/apache/airflow/pull/20530#discussion_r783331224 ## File path: airflow/decorators/__init__.py ## @@ -19,6 +19,7 @@ from airflow.decorators.python import PythonDecoratorMixin, python_task # noqa

[GitHub] [airflow] edithturn commented on a change in pull request #20763: [WIP] Verify enough resources for breeze

2022-01-12 Thread GitBox
edithturn commented on a change in pull request #20763: URL: https://github.com/apache/airflow/pull/20763#discussion_r783312075 ## File path: scripts/in_container/run_resource_check.py ## @@ -0,0 +1,102 @@ +#!/usr/bin/env python + + +# Licensed to the Apache Software

[GitHub] [airflow] github-actions[bot] commented on pull request #20833: Speedup liveness probe for scheduler and triggerer

2022-01-12 Thread GitBox
github-actions[bot] commented on pull request #20833: URL: https://github.com/apache/airflow/pull/20833#issuecomment-1011321651 The PR is likely OK to be merged with just subset of tests for default Python and Database versions without running the full matrix of tests, because it does not

[GitHub] [airflow] SamWheating commented on a change in pull request #20733: Add Audit Log View to Dag View

2022-01-12 Thread GitBox
SamWheating commented on a change in pull request #20733: URL: https://github.com/apache/airflow/pull/20733#discussion_r783322325 ## File path: airflow/config_templates/default_airflow.cfg ## @@ -658,6 +658,18 @@ auto_refresh_interval = 3 # Boolean for displaying warning for

[GitHub] [airflow] dstandish opened a new pull request #20833: Sched trig liveness speedup

2022-01-12 Thread GitBox
dstandish opened a new pull request #20833: URL: https://github.com/apache/airflow/pull/20833 Speedup liveness probe for scheduler and triggerer Liveness probe is submitted through `/entrypoint` which by default runs `airflow db check`. This can be slow. We can disable by setting

[GitHub] [airflow] edithturn commented on pull request #20701: [WIP] Check free space python

2022-01-12 Thread GitBox
edithturn commented on pull request #20701: URL: https://github.com/apache/airflow/pull/20701#issuecomment-1011312698 We have two jobs ci and build-image: * ci job is running `run: airflow-freespace`: https://github.com/apache/airflow/runs/4791100593?check_suite_focus=true, but *

[GitHub] [airflow] SamWheating commented on a change in pull request #20349: Fix Scheduler crash when executing task instances of missing DAG

2022-01-12 Thread GitBox
SamWheating commented on a change in pull request #20349: URL: https://github.com/apache/airflow/pull/20349#discussion_r783317103 ## File path: airflow/jobs/scheduler_job.py ## @@ -403,6 +403,15 @@ def _executable_task_instances_to_queued(self, max_tis: int, session: Session

[GitHub] [airflow] SamWheating commented on a change in pull request #20349: Fix Scheduler crash when executing task instances of missing DAG

2022-01-12 Thread GitBox
SamWheating commented on a change in pull request #20349: URL: https://github.com/apache/airflow/pull/20349#discussion_r783317103 ## File path: airflow/jobs/scheduler_job.py ## @@ -403,6 +403,15 @@ def _executable_task_instances_to_queued(self, max_tis: int, session: Session

[airflow] branch constraints-main updated: Updating constraints. Build id:1687991922

2022-01-12 Thread github-bot
This is an automated email from the ASF dual-hosted git repository. github-bot pushed a commit to branch constraints-main in repository https://gitbox.apache.org/repos/asf/airflow.git The following commit(s) were added to refs/heads/constraints-main by this push: new 42571b9 Updating

[GitHub] [airflow] SamWheating commented on a change in pull request #19769: Handle stuck queued tasks in Celery

2022-01-12 Thread GitBox
SamWheating commented on a change in pull request #19769: URL: https://github.com/apache/airflow/pull/19769#discussion_r783313436 ## File path: airflow/executors/celery_executor.py ## @@ -377,6 +385,49 @@ def _check_for_stalled_adopted_tasks(self): for key in

[GitHub] [airflow] SamWheating commented on a change in pull request #19769: Handle stuck queued tasks in Celery

2022-01-12 Thread GitBox
SamWheating commented on a change in pull request #19769: URL: https://github.com/apache/airflow/pull/19769#discussion_r783312574 ## File path: airflow/executors/celery_executor.py ## @@ -377,6 +385,49 @@ def _check_for_stalled_adopted_tasks(self): for key in

[GitHub] [airflow] edithturn commented on a change in pull request #20763: [WIP] Verify enough resources for breeze

2022-01-12 Thread GitBox
edithturn commented on a change in pull request #20763: URL: https://github.com/apache/airflow/pull/20763#discussion_r783312075 ## File path: scripts/in_container/run_resource_check.py ## @@ -0,0 +1,102 @@ +#!/usr/bin/env python + + +# Licensed to the Apache Software

[GitHub] [airflow] github-actions[bot] commented on pull request #20664: Switch to buildx to build airflow images

2022-01-12 Thread GitBox
github-actions[bot] commented on pull request #20664: URL: https://github.com/apache/airflow/pull/20664#issuecomment-1011290969 The PR most likely needs to run full matrix of tests because it modifies parts of the core of Airflow. However, committers might decide to merge it quickly and

[GitHub] [airflow] boring-cyborg[bot] commented on issue #20832: Unable to specify Python version for AwsGlueJobOperator

2022-01-12 Thread GitBox
boring-cyborg[bot] commented on issue #20832: URL: https://github.com/apache/airflow/issues/20832#issuecomment-1011275920 Thanks for opening your first issue here! Be sure to follow the issue template! -- This is an automated message from the Apache Git Service. To respond to the

[GitHub] [airflow] jfamestad opened a new issue #20832: Unable to specify Python version for AwsGlueJobOperator

2022-01-12 Thread GitBox
jfamestad opened a new issue #20832: URL: https://github.com/apache/airflow/issues/20832 ### Apache Airflow Provider(s) amazon ### Versions of Apache Airflow Providers _No response_ ### Apache Airflow version 2.0.2 ### Operating System Amazon

[GitHub] [airflow] kaxil commented on a change in pull request #20349: Fix Scheduler crash when executing task instances of missing DAG

2022-01-12 Thread GitBox
kaxil commented on a change in pull request #20349: URL: https://github.com/apache/airflow/pull/20349#discussion_r783272552 ## File path: tests/jobs/test_scheduler_job.py ## @@ -645,6 +645,36 @@ def test_find_executable_task_instances_in_default_pool(self, dag_maker):

[GitHub] [airflow] kaxil commented on a change in pull request #20349: Fix Scheduler crash when executing task instances of missing DAG

2022-01-12 Thread GitBox
kaxil commented on a change in pull request #20349: URL: https://github.com/apache/airflow/pull/20349#discussion_r783270374 ## File path: tests/jobs/test_scheduler_job.py ## @@ -645,6 +645,36 @@ def test_find_executable_task_instances_in_default_pool(self, dag_maker):

[GitHub] [airflow] alexcolpitts96 commented on issue #13675: TaskInstances do not succeed when using enable_logging=True option in DockerSwarmOperator

2022-01-12 Thread GitBox
alexcolpitts96 commented on issue #13675: URL: https://github.com/apache/airflow/issues/13675#issuecomment-1011244694 @eladkal Sorry to bump an old issue, but it seems to persist with version release:2.2.3+06c82e17e9d7ff1bf261357e84c6013ccdb3c241 Containers are spawned, complete

[GitHub] [airflow] askmeegs opened a new issue #20829: Update google-cloud-container package to v2.10.1

2022-01-12 Thread GitBox
askmeegs opened a new issue #20829: URL: https://github.com/apache/airflow/issues/20829 ### Description The google-cloud-container [version constraint in setup.py](https://github.com/apache/airflow/blob/main/setup.py#L305) restricts the version to < 2.0.0, but the latest version of

[GitHub] [airflow] boring-cyborg[bot] commented on issue #20829: Update google-cloud-container package to v2.10.1

2022-01-12 Thread GitBox
boring-cyborg[bot] commented on issue #20829: URL: https://github.com/apache/airflow/issues/20829#issuecomment-1011207396 Thanks for opening your first issue here! Be sure to follow the issue template! -- This is an automated message from the Apache Git Service. To respond to the

[GitHub] [airflow] kaxil commented on a change in pull request #19769: Handle stuck queued tasks in Celery

2022-01-12 Thread GitBox
kaxil commented on a change in pull request #19769: URL: https://github.com/apache/airflow/pull/19769#discussion_r783205515 ## File path: airflow/config_templates/config.yml ## @@ -1678,6 +1678,13 @@ type: string example: ~ default: "False" +- name:

[GitHub] [airflow] github-actions[bot] commented on pull request #20828: Add Helm Chart 1.4.0 in airflow_helmchart_bug_report.yml

2022-01-12 Thread GitBox
github-actions[bot] commented on pull request #20828: URL: https://github.com/apache/airflow/pull/20828#issuecomment-1011181987 The PR is likely ready to be merged. No tests are needed as no important environment files, nor python files were modified by it. However, committers might

[GitHub] [airflow] uranusjr commented on a change in pull request #20286: Add TaskMap and TaskInstance.map_id

2022-01-12 Thread GitBox
uranusjr commented on a change in pull request #20286: URL: https://github.com/apache/airflow/pull/20286#discussion_r783194609 ## File path: airflow/models/taskinstance.py ## @@ -2128,6 +2138,14 @@ def set_duration(self) -> None: self.duration = None

[GitHub] [airflow] PApostol opened a new pull request #20828: Add Helm Chart 1.4.0 in airflow_helmchart_bug_report.yml

2022-01-12 Thread GitBox
PApostol opened a new pull request #20828: URL: https://github.com/apache/airflow/pull/20828 Update `.github/ISSUE_TEMPLATE/airflow_helmchart_bug_report.yml` with the new version of Helm Chart. -- This is an automated message from the Apache Git Service. To respond to the message,

[GitHub] [airflow] edithturn commented on a change in pull request #20763: [WIP] Verify enough resources for breeze

2022-01-12 Thread GitBox
edithturn commented on a change in pull request #20763: URL: https://github.com/apache/airflow/pull/20763#discussion_r783189748 ## File path: scripts/in_container/run_resource_check.py ## @@ -0,0 +1,102 @@ +#!/usr/bin/env python + + +# Licensed to the Apache Software

[GitHub] [airflow] ashb commented on a change in pull request #20286: Add TaskMap and TaskInstance.map_id

2022-01-12 Thread GitBox
ashb commented on a change in pull request #20286: URL: https://github.com/apache/airflow/pull/20286#discussion_r783184829 ## File path: airflow/models/taskinstance.py ## @@ -2128,6 +2138,14 @@ def set_duration(self) -> None: self.duration = None

[GitHub] [airflow] ashb commented on pull request #19808: Refactor dangling row check to use SQLA queries

2022-01-12 Thread GitBox
ashb commented on pull request #19808: URL: https://github.com/apache/airflow/pull/19808#issuecomment-1011160510 I've rebased this, and I'll need to re-run these checks still work as it's been a while. -- This is an automated message from the Apache Git Service. To respond to the

[GitHub] [airflow] ashb commented on pull request #20286: Add TaskMap and TaskInstance.map_id

2022-01-12 Thread GitBox
ashb commented on pull request #20286: URL: https://github.com/apache/airflow/pull/20286#issuecomment-1011149780 > so a bit of mocking is needed. But otherwise things seem to work as expected. Yeah, I guess those'll be fixed by https://github.com/apache/airflow/pull/20743 --

[GitHub] [airflow] ashb commented on a change in pull request #20286: Add TaskMap and TaskInstance.map_id

2022-01-12 Thread GitBox
ashb commented on a change in pull request #20286: URL: https://github.com/apache/airflow/pull/20286#discussion_r783171485 ## File path: airflow/models/baseoperator.py ## @@ -1632,6 +1632,33 @@ def defer( def map(self, **kwargs) -> "MappedOperator": return

[GitHub] [airflow] edithturn commented on a change in pull request #20763: [WIP] Verify enough resources for breeze

2022-01-12 Thread GitBox
edithturn commented on a change in pull request #20763: URL: https://github.com/apache/airflow/pull/20763#discussion_r783163958 ## File path: scripts/in_container/run_resource_check.py ## @@ -0,0 +1,102 @@ +#!/usr/bin/env python + + +# Licensed to the Apache Software

[GitHub] [airflow] edithturn commented on a change in pull request #20763: [WIP] Verify enough resources for breeze

2022-01-12 Thread GitBox
edithturn commented on a change in pull request #20763: URL: https://github.com/apache/airflow/pull/20763#discussion_r783163958 ## File path: scripts/in_container/run_resource_check.py ## @@ -0,0 +1,102 @@ +#!/usr/bin/env python + + +# Licensed to the Apache Software

[GitHub] [airflow] josh-fell commented on a change in pull request #19852: created SFTPBatchOperator which add batch function

2022-01-12 Thread GitBox
josh-fell commented on a change in pull request #19852: URL: https://github.com/apache/airflow/pull/19852#discussion_r783132963 ## File path: airflow/providers/sftp/operators/sftp_batch.py ## @@ -0,0 +1,312 @@ +# +# Licensed to the Apache Software Foundation (ASF) under one +#

[GitHub] [airflow] bbovenzi commented on a change in pull request #20733: Add Audit Log View to Dag View

2022-01-12 Thread GitBox
bbovenzi commented on a change in pull request #20733: URL: https://github.com/apache/airflow/pull/20733#discussion_r783159243 ## File path: airflow/www/templates/airflow/dag_audit_log.html ## @@ -0,0 +1,92 @@ +{# + Licensed to the Apache Software Foundation (ASF) under one +

[GitHub] [airflow] bbovenzi commented on a change in pull request #20733: Add Audit Log View to Dag View

2022-01-12 Thread GitBox
bbovenzi commented on a change in pull request #20733: URL: https://github.com/apache/airflow/pull/20733#discussion_r783155908 ## File path: airflow/www/templates/airflow/dag_audit_log.html ## @@ -0,0 +1,92 @@ +{# + Licensed to the Apache Software Foundation (ASF) under one +

[GitHub] [airflow] ashb commented on a change in pull request #20826: Move tests out of test core that are duplicated/should live elsewhere

2022-01-12 Thread GitBox
ashb commented on a change in pull request #20826: URL: https://github.com/apache/airflow/pull/20826#discussion_r783154034 ## File path: tests/core/test_core.py ## @@ -16,32 +16,21 @@ # specific language governing permissions and limitations # under the License. -import os

[GitHub] [airflow] bbovenzi commented on a change in pull request #20733: Add Audit Log View to Dag View

2022-01-12 Thread GitBox
bbovenzi commented on a change in pull request #20733: URL: https://github.com/apache/airflow/pull/20733#discussion_r783154481 ## File path: airflow/www/templates/airflow/dag_audit_log.html ## @@ -0,0 +1,92 @@ +{# + Licensed to the Apache Software Foundation (ASF) under one +

[GitHub] [airflow] bbovenzi commented on a change in pull request #20733: Add Audit Log View to Dag View

2022-01-12 Thread GitBox
bbovenzi commented on a change in pull request #20733: URL: https://github.com/apache/airflow/pull/20733#discussion_r783153020 ## File path: airflow/config_templates/default_airflow.cfg ## @@ -658,6 +658,18 @@ auto_refresh_interval = 3 # Boolean for displaying warning for

[GitHub] [airflow] uranusjr commented on pull request #20286: Add TaskMap and TaskInstance.map_id

2022-01-12 Thread GitBox
uranusjr commented on pull request #20286: URL: https://github.com/apache/airflow/pull/20286#issuecomment-1011122326 Alright, modified the code a bit and added a few tests. We can’t put mapped task in a DAG yet (`bag_dag` kept throwing AttributeError at me about a ton of things expected

[GitHub] [airflow] mikaeld edited a comment on issue #12136: KubernetesPodOperator breaks with active log-collection for long running tasks

2022-01-12 Thread GitBox
mikaeld edited a comment on issue #12136: URL: https://github.com/apache/airflow/issues/12136#issuecomment-107406 @flrn77 what provider package version are you using for `cncf.kubernetes` when facing the issue? The possible fix mentioned by @nikie for `cncf.kubernetes` version

[GitHub] [airflow] mikaeld edited a comment on issue #12136: KubernetesPodOperator breaks with active log-collection for long running tasks

2022-01-12 Thread GitBox
mikaeld edited a comment on issue #12136: URL: https://github.com/apache/airflow/issues/12136#issuecomment-107406 @flrn77 what provider package version are you using for `cncf.kubernetes` when facing the issue? The possible fix mentioned by @nikie for `cncf.kubernetes` version

[GitHub] [airflow] ashb commented on pull request #20795: Fix remaining mypy issues in "core" Airflow

2022-01-12 Thread GitBox
ashb commented on pull request #20795: URL: https://github.com/apache/airflow/pull/20795#issuecomment-109023 @josh-fell Oh I like https://github.com/apache/airflow/pull/20470/files#diff-af25dd96233a97ace811c614fa7d5bd059cbbc1571e421f6675e16f6290814c5 more than adding all those "not

[GitHub] [airflow] mikaeld edited a comment on issue #12136: KubernetesPodOperator breaks with active log-collection for long running tasks

2022-01-12 Thread GitBox
mikaeld edited a comment on issue #12136: URL: https://github.com/apache/airflow/issues/12136#issuecomment-107406 @flrn77 what provider package version are you using for `cncf.kubernetes` when facing the issue? The possible fix mentioned by @nikie for `cncf.kubernetes` version

[GitHub] [airflow] mikaeld commented on issue #12136: KubernetesPodOperator breaks with active log-collection for long running tasks

2022-01-12 Thread GitBox
mikaeld commented on issue #12136: URL: https://github.com/apache/airflow/issues/12136#issuecomment-107406 @flrn77 what provider package version are you using for `cncf.kubernetes` when facing the issue? The possible fix mentioned by @nikie for `cncf.kubernetes` version `2.0.3`,

[GitHub] [airflow] edithturn edited a comment on pull request #20701: [WIP] Check free space python

2022-01-12 Thread GitBox
edithturn edited a comment on pull request #20701: URL: https://github.com/apache/airflow/pull/20701#issuecomment-1011103682 Just to add more information about "check": "If check is true, and the process exits with a non-zero exit code, a CalledProcessError exception will be raised.

[GitHub] [airflow] edithturn commented on pull request #20701: [WIP] Check free space python

2022-01-12 Thread GitBox
edithturn commented on pull request #20701: URL: https://github.com/apache/airflow/pull/20701#issuecomment-1011103682 Just to add more information about "check": "If check is true, and the process exits with a non-zero exit code, a CalledProcessError exception will be raised.

[airflow] branch main updated (19fb7fd -> 82e466d)

2022-01-12 Thread potiuk
This is an automated email from the ASF dual-hosted git repository. potiuk pushed a change to branch main in repository https://gitbox.apache.org/repos/asf/airflow.git. from 19fb7fd Better signing instructions for helm chart releases (#20796) add 82e466d Fix airflow trigger cli

[GitHub] [airflow] potiuk closed issue #20579: Airflow 2.2.3 : "airflow dags trigger" command gets "Calling `DAG.create_dagrun()` without an explicit data interval is deprecated"

2022-01-12 Thread GitBox
potiuk closed issue #20579: URL: https://github.com/apache/airflow/issues/20579 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

[GitHub] [airflow] potiuk merged pull request #20781: Fix airflow dag trigger cli

2022-01-12 Thread GitBox
potiuk merged pull request #20781: URL: https://github.com/apache/airflow/pull/20781 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

[GitHub] [airflow] edithturn edited a comment on pull request #20701: [WIP] Check free space python

2022-01-12 Thread GitBox
edithturn edited a comment on pull request #20701: URL: https://github.com/apache/airflow/pull/20701#issuecomment-1011082989 I got it @potiuk "apt is for the terminal and gives beautiful output while apt-get and apt-cache are for scripts and give stable, parsable output. ", making the

[GitHub] [airflow] edithturn commented on pull request #20701: [WIP] Check free space python

2022-01-12 Thread GitBox
edithturn commented on pull request #20701: URL: https://github.com/apache/airflow/pull/20701#issuecomment-1011082989 I got it @potiuk "apt is for the terminal and gives beautiful output while apt-get and apt-cache are for scripts and give stable, parsable output. ", making the

[GitHub] [airflow] potiuk commented on pull request #19852: created SFTPBatchOperator which add batch function

2022-01-12 Thread GitBox
potiuk commented on pull request #19852: URL: https://github.com/apache/airflow/pull/19852#issuecomment-1011082315 Fixing doc/static checks and rebasing ? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above

[GitHub] [airflow] josh-fell commented on pull request #20795: Fix remaining mypy issues in "core" Airflow

2022-01-12 Thread GitBox
josh-fell commented on pull request #20795: URL: https://github.com/apache/airflow/pull/20795#issuecomment-1011076700 Do we want to use this as the fix for `airflow/dag_processing` rather than #20470? -- This is an automated message from the Apache Git Service. To respond to the

[GitHub] [airflow] potiuk commented on pull request #20795: Fix remaining mypy issues in "core" Airflow

2022-01-12 Thread GitBox
potiuk commented on pull request #20795: URL: https://github.com/apache/airflow/pull/20795#issuecomment-1011064541 Still few nits, but it's good enough :) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above

[GitHub] [airflow] github-actions[bot] commented on pull request #20827: Add explanation to stub files introduced to handle errors in examples

2022-01-12 Thread GitBox
github-actions[bot] commented on pull request #20827: URL: https://github.com/apache/airflow/pull/20827#issuecomment-1011064346 The PR is likely OK to be merged with just subset of tests for default Python and Database versions without running the full matrix of tests, because it does not

<    1   2   3   >