[GitHub] [airflow] potiuk commented on pull request #20847: Added other Opentelemery Instrumentations

2022-01-12 Thread GitBox
potiuk commented on pull request #20847: URL: https://github.com/apache/airflow/pull/20847#issuecomment-1011887017 Nice. @Melodie97 - while you are at it - maybe - while preparing to the demo next week you could also take a look at the beginning of this:

[GitHub] [airflow] Melodie97 commented on issue #20774: Expand POC with all possible standard metrics available in Open Telemetry

2022-01-12 Thread GitBox
Melodie97 commented on issue #20774: URL: https://github.com/apache/airflow/issues/20774#issuecomment-1011885124 Ok, I'm done with them all -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the

[GitHub] [airflow] Melodie97 opened a new pull request #20847: Added other Opentelemery Instrumentations

2022-01-12 Thread GitBox
Melodie97 opened a new pull request #20847: URL: https://github.com/apache/airflow/pull/20847 --- **^ Add meaningful description above** Read the **[Pull Request Guidelines](https://github.com/apache/airflow/blob/main/CONTRIBUTING.rst#pull-request-guidelines)** for more

[GitHub] [airflow] potiuk commented on a change in pull request #20843: Fix flaky templatized call

2022-01-12 Thread GitBox
potiuk commented on a change in pull request #20843: URL: https://github.com/apache/airflow/pull/20843#discussion_r783700863 ## File path: tests/sensors/test_python.py ## @@ -85,7 +85,7 @@ def test_python_callable_arguments_are_templatized(self): ds_templated =

[GitHub] [airflow] potiuk commented on a change in pull request #20843: Fix flaky templatized call

2022-01-12 Thread GitBox
potiuk commented on a change in pull request #20843: URL: https://github.com/apache/airflow/pull/20843#discussion_r783700863 ## File path: tests/sensors/test_python.py ## @@ -85,7 +85,7 @@ def test_python_callable_arguments_are_templatized(self): ds_templated =

[GitHub] [airflow] potiuk commented on a change in pull request #20843: Fix flaky templatized call

2022-01-12 Thread GitBox
potiuk commented on a change in pull request #20843: URL: https://github.com/apache/airflow/pull/20843#discussion_r783698559 ## File path: tests/sensors/test_python.py ## @@ -85,7 +85,7 @@ def test_python_callable_arguments_are_templatized(self): ds_templated =

[GitHub] [airflow] potiuk commented on a change in pull request #20843: Fix flaky templatized call

2022-01-12 Thread GitBox
potiuk commented on a change in pull request #20843: URL: https://github.com/apache/airflow/pull/20843#discussion_r783698559 ## File path: tests/sensors/test_python.py ## @@ -85,7 +85,7 @@ def test_python_callable_arguments_are_templatized(self): ds_templated =

[GitHub] [airflow] potiuk commented on a change in pull request #20843: Fix flaky templatized call

2022-01-12 Thread GitBox
potiuk commented on a change in pull request #20843: URL: https://github.com/apache/airflow/pull/20843#discussion_r783698559 ## File path: tests/sensors/test_python.py ## @@ -85,7 +85,7 @@ def test_python_callable_arguments_are_templatized(self): ds_templated =

[GitHub] [airflow] potiuk commented on issue #17279: Logout airflow on Web UI does not work using OAuth2

2022-01-12 Thread GitBox
potiuk commented on issue #17279: URL: https://github.com/apache/airflow/issues/17279#issuecomment-1011872225 cc: @kaxil -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment.

[GitHub] [airflow] potiuk edited a comment on issue #17279: Logout airflow on Web UI does not work using OAuth2

2022-01-12 Thread GitBox
potiuk edited a comment on issue #17279: URL: https://github.com/apache/airflow/issues/17279#issuecomment-1011871954 Flask App builder constraints are updated already - both main and 2.2 branch. So we could fix it in main and cherry-pick to 2.2.4 -- This is an automated message from

[GitHub] [airflow] potiuk commented on issue #17279: Logout airflow on Web UI does not work using OAuth2

2022-01-12 Thread GitBox
potiuk commented on issue #17279: URL: https://github.com/apache/airflow/issues/17279#issuecomment-1011871954 Flask App builder constraints are updated already - both main and 2.2 branch. So we could fix it in main and cherry-pick to 2.3.4 -- This is an automated message from the

[GitHub] [airflow] blag edited a comment on issue #17279: Logout airflow on Web UI does not work using OAuth2

2022-01-12 Thread GitBox
blag edited a comment on issue #17279: URL: https://github.com/apache/airflow/issues/17279#issuecomment-991370837 Now that dpgaspar/Flask-AppBuilder#1749 is merged, this is one step closer to being fixed. TODO as of now: * [x] Wait for a new release of FAB that includes that

[airflow] branch main updated (8dc68d4 -> c49d6ec)

2022-01-12 Thread potiuk
This is an automated email from the ASF dual-hosted git repository. potiuk pushed a change to branch main in repository https://gitbox.apache.org/repos/asf/airflow.git. from 8dc68d4 Doc: Added an enum param example (#20841) add c49d6ec static code check doc fix (#20844) No new

[GitHub] [airflow] potiuk closed issue #20823: Static check docs mistakes in example

2022-01-12 Thread GitBox
potiuk closed issue #20823: URL: https://github.com/apache/airflow/issues/20823 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

[GitHub] [airflow] potiuk merged pull request #20844: static code check doc fix

2022-01-12 Thread GitBox
potiuk merged pull request #20844: URL: https://github.com/apache/airflow/pull/20844 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

[GitHub] [airflow] Bowrna opened a new pull request #20844: static code check doc fix

2022-01-12 Thread GitBox
Bowrna opened a new pull request #20844: URL: https://github.com/apache/airflow/pull/20844 Static code check docs issue in current breeze environment. closes: https://github.com/apache/airflow/issues/20823 --- **^ Add meaningful description above** Read the **[Pull

[GitHub] [airflow-client-python] feluelle commented on issue #20: get_tasks api is broken

2022-01-12 Thread GitBox
feluelle commented on issue #20: URL: https://github.com/apache/airflow-client-python/issues/20#issuecomment-1011859928 @msumit do you have any idea why this is? I am running into this problem still on the latest version (2.2.0). -- This is an automated message from the Apache Git

[GitHub] [airflow] potiuk edited a comment on issue #19957: Airflow crashes with a psycopg2.errors.DeadlockDetected exception

2022-01-12 Thread GitBox
potiuk edited a comment on issue #19957: URL: https://github.com/apache/airflow/issues/19957#issuecomment-1011857056 I actually spent some time few days ago looking at the mini-scheduler code but I could not really find a flaw there. The fact that it did not help you indicates that my

[GitHub] [airflow] potiuk commented on issue #19957: Airflow crashes with a psycopg2.errors.DeadlockDetected exception

2022-01-12 Thread GitBox
potiuk commented on issue #19957: URL: https://github.com/apache/airflow/issues/19957#issuecomment-1011857056 I actually spent some time few days ago looking at the mini-scheduler code but I could not really find a flaw there. The fact that it did not help you indicates that my hypothesis

[GitHub] [airflow] github-actions[bot] commented on pull request #20349: Fix Scheduler crash when executing task instances of missing DAG

2022-01-12 Thread GitBox
github-actions[bot] commented on pull request #20349: URL: https://github.com/apache/airflow/pull/20349#issuecomment-1011850115 The PR most likely needs to run full matrix of tests because it modifies parts of the core of Airflow. However, committers might decide to merge it quickly and

[GitHub] [airflow] uranusjr commented on a change in pull request #20795: Fix remaining mypy issues in "core" Airflow

2022-01-12 Thread GitBox
uranusjr commented on a change in pull request #20795: URL: https://github.com/apache/airflow/pull/20795#discussion_r783675997 ## File path: airflow/models/taskmixin.py ## @@ -114,6 +115,14 @@ class DAGNode(DependencyMixin, metaclass=ABCMeta): def node_id(self) -> str:

[GitHub] [airflow] potiuk commented on pull request #20245: Exclude snowflake-sqlalchemy v1.2.5

2022-01-12 Thread GitBox
potiuk commented on pull request #20245: URL: https://github.com/apache/airflow/pull/20245#issuecomment-1011849274 https://github.com/apache/airflow/blob/constraints-2.2.3/constraints-3.7.txt#L485 -- This is an automated message from the Apache Git Service. To respond to the message,

[GitHub] [airflow] potiuk commented on pull request #20245: Exclude snowflake-sqlalchemy v1.2.5

2022-01-12 Thread GitBox
potiuk commented on pull request #20245: URL: https://github.com/apache/airflow/pull/20245#issuecomment-1011848901 @NadimYounes -Indeed 2.2.3 constraints were prepared before snowflake was yanked. It should be fixed now. -- This is an automated message from the Apache Git Service. To

[airflow] tag constraints-2.2.3 updated (62d490d -> 8a96274)

2022-01-12 Thread potiuk
This is an automated email from the ASF dual-hosted git repository. potiuk pushed a change to tag constraints-2.2.3 in repository https://gitbox.apache.org/repos/asf/airflow.git. *** WARNING: tag constraints-2.2.3 was modified! *** from 62d490d (commit) to 8a96274 (commit) from

[GitHub] [airflow] uranusjr commented on a change in pull request #20843: Fix flaky templatized call

2022-01-12 Thread GitBox
uranusjr commented on a change in pull request #20843: URL: https://github.com/apache/airflow/pull/20843#discussion_r783674157 ## File path: tests/sensors/test_python.py ## @@ -85,7 +85,7 @@ def test_python_callable_arguments_are_templatized(self): ds_templated =

[airflow] 01/01: Fix constraints to yanked snowflake release

2022-01-12 Thread potiuk
This is an automated email from the ASF dual-hosted git repository. potiuk pushed a commit to branch constraints-2-2-3-fixed in repository https://gitbox.apache.org/repos/asf/airflow.git commit 8a962740270a9ecf6b4f910173321b183e7adf4a Author: Jarek Potiuk AuthorDate: Thu Jan 13 07:53:36 2022

[airflow] branch constraints-2-2-3-fixed created (now 8a96274)

2022-01-12 Thread potiuk
This is an automated email from the ASF dual-hosted git repository. potiuk pushed a change to branch constraints-2-2-3-fixed in repository https://gitbox.apache.org/repos/asf/airflow.git. at 8a96274 Fix constraints to yanked snowflake release This branch includes the following new

[GitHub] [airflow] uranusjr commented on a change in pull request #20843: Fix flaky templatized call

2022-01-12 Thread GitBox
uranusjr commented on a change in pull request #20843: URL: https://github.com/apache/airflow/pull/20843#discussion_r783673639 ## File path: tests/sensors/test_python.py ## @@ -85,7 +85,7 @@ def test_python_callable_arguments_are_templatized(self): ds_templated =

[GitHub] [airflow] uranusjr commented on a change in pull request #20838: Add `maintenance cleanup` CLI command for purging old data

2022-01-12 Thread GitBox
uranusjr commented on a change in pull request #20838: URL: https://github.com/apache/airflow/pull/20838#discussion_r783672138 ## File path: airflow/utils/metastore_cleanup.py ## @@ -0,0 +1,289 @@ +# Licensed to the Apache Software Foundation (ASF) under one +# or more

[GitHub] [airflow] uranusjr commented on a change in pull request #20838: Add `maintenance cleanup` CLI command for purging old data

2022-01-12 Thread GitBox
uranusjr commented on a change in pull request #20838: URL: https://github.com/apache/airflow/pull/20838#discussion_r783671395 ## File path: airflow/utils/metastore_cleanup.py ## @@ -0,0 +1,289 @@ +# Licensed to the Apache Software Foundation (ASF) under one +# or more

[GitHub] [airflow] ephraimbuddy commented on a change in pull request #20349: Fix Scheduler crash when executing task instances of missing DAG

2022-01-12 Thread GitBox
ephraimbuddy commented on a change in pull request #20349: URL: https://github.com/apache/airflow/pull/20349#discussion_r783670984 ## File path: airflow/jobs/scheduler_job.py ## @@ -403,6 +403,15 @@ def _executable_task_instances_to_queued(self, max_tis: int, session: Session

[airflow] branch main updated (83b1e36 -> 8dc68d4)

2022-01-12 Thread kaxilnaik
This is an automated email from the ASF dual-hosted git repository. kaxilnaik pushed a change to branch main in repository https://gitbox.apache.org/repos/asf/airflow.git. from 83b1e36 Speedup liveness probe for scheduler and triggerer (#20833) add 8dc68d4 Doc: Added an enum param

[GitHub] [airflow] kaxil merged pull request #20841: added an enum param example

2022-01-12 Thread GitBox
kaxil merged pull request #20841: URL: https://github.com/apache/airflow/pull/20841 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

[GitHub] [airflow] potiuk commented on pull request #20795: Fix remaining mypy issues in "core" Airflow

2022-01-12 Thread GitBox
potiuk commented on pull request #20795: URL: https://github.com/apache/airflow/pull/20795#issuecomment-1011843784 Flay test fixed in https://github.com/apache/airflow/pull/20843 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [airflow] potiuk commented on pull request #20843: Fix flaky templatized call

2022-01-12 Thread GitBox
potiuk commented on pull request #20843: URL: https://github.com/apache/airflow/pull/20843#issuecomment-1011843612 Fixes flaky test (example in #20795) https://github.com/apache/airflow/runs/4793994282?check_suite_focus=true -- This is an automated message from the Apache Git Service.

[GitHub] [airflow] uranusjr commented on a change in pull request #20838: Add `maintenance cleanup` CLI command for purging old data

2022-01-12 Thread GitBox
uranusjr commented on a change in pull request #20838: URL: https://github.com/apache/airflow/pull/20838#discussion_r783669005 ## File path: airflow/cli/commands/maintenance_command.py ## @@ -0,0 +1,31 @@ +# Licensed to the Apache Software Foundation (ASF) under one +# or more

[GitHub] [airflow] potiuk opened a new pull request #20843: Fix flaky templatized call

2022-01-12 Thread GitBox
potiuk opened a new pull request #20843: URL: https://github.com/apache/airflow/pull/20843 --- **^ Add meaningful description above** Read the **[Pull Request Guidelines](https://github.com/apache/airflow/blob/main/CONTRIBUTING.rst#pull-request-guidelines)** for more

[GitHub] [airflow] kaxil commented on a change in pull request #19769: Handle stuck queued tasks in Celery

2022-01-12 Thread GitBox
kaxil commented on a change in pull request #19769: URL: https://github.com/apache/airflow/pull/19769#discussion_r783667980 ## File path: airflow/executors/celery_executor.py ## @@ -377,6 +385,49 @@ def _check_for_stalled_adopted_tasks(self): for key in

[GitHub] [airflow] ephraimbuddy commented on a change in pull request #19769: Handle stuck queued tasks in Celery

2022-01-12 Thread GitBox
ephraimbuddy commented on a change in pull request #19769: URL: https://github.com/apache/airflow/pull/19769#discussion_r783662283 ## File path: airflow/executors/celery_executor.py ## @@ -377,6 +385,49 @@ def _check_for_stalled_adopted_tasks(self): for key in

[GitHub] [airflow] uranusjr commented on a change in pull request #18575: Resync DAG during next parse if error in ``sync_to_db``

2022-01-12 Thread GitBox
uranusjr commented on a change in pull request #18575: URL: https://github.com/apache/airflow/pull/18575#discussion_r783661606 ## File path: airflow/models/dagbag.py ## @@ -584,22 +584,35 @@ def _serialize_dag_capturing_errors(dag, session): We can't place them

[GitHub] [airflow] github-actions[bot] commented on pull request #20841: added an enum param example

2022-01-12 Thread GitBox
github-actions[bot] commented on pull request #20841: URL: https://github.com/apache/airflow/pull/20841#issuecomment-1011831582 The PR is likely ready to be merged. No tests are needed as no important environment files, nor python files were modified by it. However, committers might

[GitHub] [airflow] uranusjr commented on a change in pull request #20286: Add TaskMap and TaskInstance.map_id

2022-01-12 Thread GitBox
uranusjr commented on a change in pull request #20286: URL: https://github.com/apache/airflow/pull/20286#discussion_r783658006 ## File path: airflow/models/taskinstance.py ## @@ -2128,6 +2138,14 @@ def set_duration(self) -> None: self.duration = None

[GitHub] [airflow] uranusjr commented on a change in pull request #20286: Add TaskMap and TaskInstance.map_id

2022-01-12 Thread GitBox
uranusjr commented on a change in pull request #20286: URL: https://github.com/apache/airflow/pull/20286#discussion_r783657603 ## File path: airflow/models/baseoperator.py ## @@ -1632,6 +1632,33 @@ def defer( def map(self, **kwargs) -> "MappedOperator": return

[jira] [Commented] (AIRFLOW-5071) Thousand os Executor reports task instance X finished (success) although the task says its queued. Was the task killed externally?

2022-01-12 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/AIRFLOW-5071?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17475121#comment-17475121 ] ASF GitHub Bot commented on AIRFLOW-5071: - ghostbody edited a comment on issue #10790: URL:

[GitHub] [airflow] ghostbody edited a comment on issue #10790: Copy of [AIRFLOW-5071] JIRA: Thousands of Executor reports task instance X finished (success) although the task says its queued. Was the

2022-01-12 Thread GitBox
ghostbody edited a comment on issue #10790: URL: https://github.com/apache/airflow/issues/10790#issuecomment-1011812804 we reviewed the code and found that in `local_task_job.py`, the parent process has a `heatbeat_callback`, and will check the state and child process return code of the

[jira] [Commented] (AIRFLOW-5071) Thousand os Executor reports task instance X finished (success) although the task says its queued. Was the task killed externally?

2022-01-12 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/AIRFLOW-5071?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17475117#comment-17475117 ] ASF GitHub Bot commented on AIRFLOW-5071: - ghostbody edited a comment on issue #10790: URL:

[GitHub] [airflow] ghostbody edited a comment on issue #10790: Copy of [AIRFLOW-5071] JIRA: Thousands of Executor reports task instance X finished (success) although the task says its queued. Was the

2022-01-12 Thread GitBox
ghostbody edited a comment on issue #10790: URL: https://github.com/apache/airflow/issues/10790#issuecomment-1011812804 we reviewed the code and found that in `local_task_job.py`, the parent process has a `heatbeat_callback`, and will check the state and child process return code of the

[jira] [Commented] (AIRFLOW-5071) Thousand os Executor reports task instance X finished (success) although the task says its queued. Was the task killed externally?

2022-01-12 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/AIRFLOW-5071?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17475116#comment-17475116 ] ASF GitHub Bot commented on AIRFLOW-5071: - ghostbody edited a comment on issue #10790: URL:

[GitHub] [airflow] ghostbody edited a comment on issue #10790: Copy of [AIRFLOW-5071] JIRA: Thousands of Executor reports task instance X finished (success) although the task says its queued. Was the

2022-01-12 Thread GitBox
ghostbody edited a comment on issue #10790: URL: https://github.com/apache/airflow/issues/10790#issuecomment-1011812804 we reviewed the code and found that in `local_task_job.py`, the parent process has a `heatbeat_callback`, and will check the state and child process return code of the

[jira] [Commented] (AIRFLOW-5071) Thousand os Executor reports task instance X finished (success) although the task says its queued. Was the task killed externally?

2022-01-12 Thread ASF GitHub Bot (Jira)
[ https://issues.apache.org/jira/browse/AIRFLOW-5071?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17475115#comment-17475115 ] ASF GitHub Bot commented on AIRFLOW-5071: - ghostbody commented on issue #10790: URL:

[GitHub] [airflow] ghostbody commented on issue #10790: Copy of [AIRFLOW-5071] JIRA: Thousands of Executor reports task instance X finished (success) although the task says its queued. Was the task ki

2022-01-12 Thread GitBox
ghostbody commented on issue #10790: URL: https://github.com/apache/airflow/issues/10790#issuecomment-1011812804 we reviewed the code and found that in `local_task_job.py`, the parent process has a `heatbeat_callback`, and will check the state and child process return code of the

[GitHub] [airflow] MatrixManAtYrService opened a new pull request #20841: added an enum param example

2022-01-12 Thread GitBox
MatrixManAtYrService opened a new pull request #20841: URL: https://github.com/apache/airflow/pull/20841 More examples makes it easier to compare our docs with the json-schema docs and figure out how they work together (I tested the code shown here). -- This is an automated message from

[GitHub] [airflow] uranusjr commented on a change in pull request #20743: Serialize mapped tasks and task groups

2022-01-12 Thread GitBox
uranusjr commented on a change in pull request #20743: URL: https://github.com/apache/airflow/pull/20743#discussion_r783623735 ## File path: airflow/models/baseoperator.py ## @@ -207,6 +207,7 @@ def apply_defaults(self: "BaseOperator", *args: Any, **kwargs: Any) -> Any:

[GitHub] [airflow] nirutgupta edited a comment on issue #20779: Logs of tasks when running is not available with kubernetesexecutor on webserver UI

2022-01-12 Thread GitBox
nirutgupta edited a comment on issue #20779: URL: https://github.com/apache/airflow/issues/20779#issuecomment-1011786738 k8s task handler.py which we can use https://gist.github.com/szeevs/938ad3cf96e732d4b1b55a74015aed5b log_config.py will look like ``` from copy import deepcopy

[GitHub] [airflow] nirutgupta edited a comment on issue #20779: Logs of tasks when running is not available with kubernetesexecutor on webserver UI

2022-01-12 Thread GitBox
nirutgupta edited a comment on issue #20779: URL: https://github.com/apache/airflow/issues/20779#issuecomment-1011786738 k8s task handler.py which we can use https://gist.github.com/szeevs/938ad3cf96e732d4b1b55a74015aed5b log_config.py will look like ``` from copy import deepcopy

[GitHub] [airflow] nirutgupta commented on issue #20779: Logs of tasks when running is not available with kubernetesexecutor on webserver UI

2022-01-12 Thread GitBox
nirutgupta commented on issue #20779: URL: https://github.com/apache/airflow/issues/20779#issuecomment-1011786738 k8s task handler.py which we can use https://gist.github.com/szeevs/938ad3cf96e732d4b1b55a74015aed5b log_config.py will look like `from copy import deepcopy from

[GitHub] [airflow] nirutgupta commented on issue #20779: Logs of tasks when running is not available with kubernetesexecutor on webserver UI

2022-01-12 Thread GitBox
nirutgupta commented on issue #20779: URL: https://github.com/apache/airflow/issues/20779#issuecomment-1011785432 I see a good working solution of this with https://szeevs.medium.com/handling-airflow-logs-with-kubernetes-executor-25c11ea831e4 All we need to do is make this

[GitHub] [airflow] uranusjr commented on a change in pull request #20349: Fix Scheduler crash when executing task instances of missing DAG

2022-01-12 Thread GitBox
uranusjr commented on a change in pull request #20349: URL: https://github.com/apache/airflow/pull/20349#discussion_r783617068 ## File path: tests/jobs/test_scheduler_job.py ## @@ -645,6 +645,36 @@ def test_find_executable_task_instances_in_default_pool(self, dag_maker):

[GitHub] [airflow] uranusjr commented on a change in pull request #20349: Fix Scheduler crash when executing task instances of missing DAG

2022-01-12 Thread GitBox
uranusjr commented on a change in pull request #20349: URL: https://github.com/apache/airflow/pull/20349#discussion_r783616795 ## File path: airflow/jobs/scheduler_job.py ## @@ -403,6 +403,15 @@ def _executable_task_instances_to_queued(self, max_tis: int, session: Session =

[airflow] branch constraints-main updated: Updating constraints. Build id:1690466917

2022-01-12 Thread github-bot
This is an automated email from the ASF dual-hosted git repository. github-bot pushed a commit to branch constraints-main in repository https://gitbox.apache.org/repos/asf/airflow.git The following commit(s) were added to refs/heads/constraints-main by this push: new 32bc5e2 Updating

[airflow] branch main updated (fe5aba2 -> 83b1e36)

2022-01-12 Thread dstandish
This is an automated email from the ASF dual-hosted git repository. dstandish pushed a change to branch main in repository https://gitbox.apache.org/repos/asf/airflow.git. from fe5aba2 Fix grammatical error in Variable.get docstring (#20837) add 83b1e36 Speedup liveness probe for

[GitHub] [airflow] dstandish commented on a change in pull request #20838: Add `maintenance cleanup` CLI command for purging old data

2022-01-12 Thread GitBox
dstandish commented on a change in pull request #20838: URL: https://github.com/apache/airflow/pull/20838#discussion_r783602645 ## File path: airflow/utils/metastore_cleanup.py ## @@ -0,0 +1,289 @@ +# Licensed to the Apache Software Foundation (ASF) under one +# or more

[GitHub] [airflow] dstandish commented on a change in pull request #20838: Add `maintenance cleanup` CLI command for purging old data

2022-01-12 Thread GitBox
dstandish commented on a change in pull request #20838: URL: https://github.com/apache/airflow/pull/20838#discussion_r783602610 ## File path: airflow/cli/cli_parser.py ## @@ -378,6 +383,27 @@ def _check(value): ARG_CONF = Arg(('-c', '--conf'), help="JSON string that gets

[GitHub] [airflow] dstandish commented on a change in pull request #20838: Add `maintenance cleanup` CLI command for purging old data

2022-01-12 Thread GitBox
dstandish commented on a change in pull request #20838: URL: https://github.com/apache/airflow/pull/20838#discussion_r783602548 ## File path: airflow/utils/metastore_cleanup.py ## @@ -0,0 +1,289 @@ +# Licensed to the Apache Software Foundation (ASF) under one +# or more

[airflow] branch constraints-main updated: Updating constraints. Build id:1690466917

2022-01-12 Thread github-bot
This is an automated email from the ASF dual-hosted git repository. github-bot pushed a commit to branch constraints-main in repository https://gitbox.apache.org/repos/asf/airflow.git The following commit(s) were added to refs/heads/constraints-main by this push: new 32bc5e2 Updating

[airflow] branch main updated (fe5aba2 -> 83b1e36)

2022-01-12 Thread dstandish
This is an automated email from the ASF dual-hosted git repository. dstandish pushed a change to branch main in repository https://gitbox.apache.org/repos/asf/airflow.git. from fe5aba2 Fix grammatical error in Variable.get docstring (#20837) add 83b1e36 Speedup liveness probe for

[airflow] branch constraints-main updated: Updating constraints. Build id:1690466917

2022-01-12 Thread github-bot
This is an automated email from the ASF dual-hosted git repository. github-bot pushed a commit to branch constraints-main in repository https://gitbox.apache.org/repos/asf/airflow.git The following commit(s) were added to refs/heads/constraints-main by this push: new 32bc5e2 Updating

[airflow] branch main updated (fe5aba2 -> 83b1e36)

2022-01-12 Thread dstandish
This is an automated email from the ASF dual-hosted git repository. dstandish pushed a change to branch main in repository https://gitbox.apache.org/repos/asf/airflow.git. from fe5aba2 Fix grammatical error in Variable.get docstring (#20837) add 83b1e36 Speedup liveness probe for

[GitHub] [airflow] dwiajik edited a comment on issue #19957: Airflow crashes with a psycopg2.errors.DeadlockDetected exception

2022-01-12 Thread GitBox
dwiajik edited a comment on issue #19957: URL: https://github.com/apache/airflow/issues/19957#issuecomment-1011683020 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To

[GitHub] [airflow] dwiajik commented on issue #19957: Airflow crashes with a psycopg2.errors.DeadlockDetected exception

2022-01-12 Thread GitBox
dwiajik commented on issue #19957: URL: https://github.com/apache/airflow/issues/19957#issuecomment-1011683020 I am experiencing the same problem and have set `schedule_after_task_execution` to `False`. The issue still persist. Do you have any suggestion? Thanks -- This is an automated

[GitHub] [airflow] jedcunningham opened a new pull request #18575: Resync DAG during next parse if error in ``sync_to_db``

2022-01-12 Thread GitBox
jedcunningham opened a new pull request #18575: URL: https://github.com/apache/airflow/pull/18575 If there is an issue syncing DAG specific permissions, force syncing to happen again the next time the DAG is parsed so errors will be added to ``import_errors`` and shown in the UI.

[airflow] branch main updated (fe5aba2 -> 83b1e36)

2022-01-12 Thread dstandish
This is an automated email from the ASF dual-hosted git repository. dstandish pushed a change to branch main in repository https://gitbox.apache.org/repos/asf/airflow.git. from fe5aba2 Fix grammatical error in Variable.get docstring (#20837) add 83b1e36 Speedup liveness probe for

[GitHub] [airflow] NadimYounes edited a comment on pull request #20245: Exclude snowflake-sqlalchemy v1.2.5

2022-01-12 Thread GitBox
NadimYounes edited a comment on pull request #20245: URL: https://github.com/apache/airflow/pull/20245#issuecomment-1011631448 @mik-laj Not sure I am following. If I try installing airflow with the snowflake provider using the command below: ``` pip install

[GitHub] [airflow] NadimYounes edited a comment on pull request #20245: Exclude snowflake-sqlalchemy v1.2.5

2022-01-12 Thread GitBox
NadimYounes edited a comment on pull request #20245: URL: https://github.com/apache/airflow/pull/20245#issuecomment-1011631448 @mik-laj Not sure I am following. If I try installing airflow with the snowflake provider using the command below: ``` pip install

[GitHub] [airflow] NadimYounes edited a comment on pull request #20245: Exclude snowflake-sqlalchemy v1.2.5

2022-01-12 Thread GitBox
NadimYounes edited a comment on pull request #20245: URL: https://github.com/apache/airflow/pull/20245#issuecomment-1011631448 @mik-laj Not sure I am following. If I try installing airflow with the snowflake provider using the command below: pip install

[GitHub] [airflow] NadimYounes edited a comment on pull request #20245: Exclude snowflake-sqlalchemy v1.2.5

2022-01-12 Thread GitBox
NadimYounes edited a comment on pull request #20245: URL: https://github.com/apache/airflow/pull/20245#issuecomment-1011631448 @mik-laj Not sure I am following. If I try installing airflow with the snowflake provider using the command below: pip install `apache-airflow[snowflake]`

[GitHub] [airflow] NadimYounes edited a comment on pull request #20245: Exclude snowflake-sqlalchemy v1.2.5

2022-01-12 Thread GitBox
NadimYounes edited a comment on pull request #20245: URL: https://github.com/apache/airflow/pull/20245#issuecomment-1011631448 @mik-laj Not sure I am following. If I try installing airflow with the snowflake provider using the command below: pip install `apache-airflow[snowflake]`

[GitHub] [airflow] NadimYounes commented on pull request #20245: Exclude snowflake-sqlalchemy v1.2.5

2022-01-12 Thread GitBox
NadimYounes commented on pull request #20245: URL: https://github.com/apache/airflow/pull/20245#issuecomment-1011631448 @mik-laj Not sure I am following. If I try installing pip install `apache-airflow[snowflake]` -c

[GitHub] [airflow] sergxm2 edited a comment on issue #19222: none_failed_min_one_success trigger rule not working with BranchPythonOperator in certain cases.

2022-01-12 Thread GitBox
sergxm2 edited a comment on issue #19222: URL: https://github.com/apache/airflow/issues/19222#issuecomment-1011610462 Seeing the same issue with BranchPythonOperator / branching and the final task (i.e. task6) being incorrectly skipped instead of being called. This is observed in 2.2.x

[GitHub] [airflow] boring-cyborg[bot] commented on issue #20839: Cannot edit custom fields on provider connections

2022-01-12 Thread GitBox
boring-cyborg[bot] commented on issue #20839: URL: https://github.com/apache/airflow/issues/20839#issuecomment-1011613591 Thanks for opening your first issue here! Be sure to follow the issue template! -- This is an automated message from the Apache Git Service. To respond to the

[GitHub] [airflow] mike-mcdonald opened a new issue #20839: Cannot edit custom fields on provider connections

2022-01-12 Thread GitBox
mike-mcdonald opened a new issue #20839: URL: https://github.com/apache/airflow/issues/20839 ### Apache Airflow version 2.2.3 (latest released) ### What happened Connections from providers are not saving edited values in any custom connection forms. You can work around

[GitHub] [airflow] sergxm2 commented on issue #19222: none_failed_min_one_success trigger rule not working with BranchPythonOperator in certain cases.

2022-01-12 Thread GitBox
sergxm2 commented on issue #19222: URL: https://github.com/apache/airflow/issues/19222#issuecomment-1011610462 Seeing the same issue with BranchPythonOperator / branching and the final task (i.e. task6) being incorrectly skipped instead of being called. This is observed in 2.2.x but not

[GitHub] [airflow] dstandish merged pull request #20833: Speedup liveness probe for scheduler and triggerer

2022-01-12 Thread GitBox
dstandish merged pull request #20833: URL: https://github.com/apache/airflow/pull/20833 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

[GitHub] [airflow] mik-laj commented on pull request #20245: Exclude snowflake-sqlalchemy v1.2.5

2022-01-12 Thread GitBox
mik-laj commented on pull request #20245: URL: https://github.com/apache/airflow/pull/20245#issuecomment-1011599004 @NadimYounes This release has been yanked, so it shouldn't be installed automatically anymore. -- This is an automated message from the Apache Git Service. To respond to

[GitHub] [airflow] danmactough commented on issue #13542: Task stuck in "scheduled" or "queued" state, pool has all slots queued, nothing is executing

2022-01-12 Thread GitBox
danmactough commented on issue #13542: URL: https://github.com/apache/airflow/issues/13542#issuecomment-1011598836 Airflow 2.0.2+e494306fb01f3a026e7e2832ca94902e96b526fa (MWAA on AWS) This happens to us a LOT: a DAG will be running, task instances will be marked as "queued", but

[GitHub] [airflow] NadimYounes edited a comment on pull request #20245: Exclude snowflake-sqlalchemy v1.2.5

2022-01-12 Thread GitBox
NadimYounes edited a comment on pull request #20245: URL: https://github.com/apache/airflow/pull/20245#issuecomment-1011580548 @potiuk @mik-laj Can we also fix the constraints for airflow `2.2.3`? It looks like `snowflake-sqlalchemy` is set to `1.2.5`

[GitHub] [airflow] NadimYounes commented on pull request #20245: Exclude snowflake-sqlalchemy v1.2.5

2022-01-12 Thread GitBox
NadimYounes commented on pull request #20245: URL: https://github.com/apache/airflow/pull/20245#issuecomment-1011580548 @potiuk @mik-laj Can we also fix the constraints for airflow `2.2.3`? It looks like `snowflake-sqlalchemy` is still set to `1.2.5`

[GitHub] [airflow] SamWheating commented on a change in pull request #20838: Add `maintenance cleanup` CLI command for purging old data

2022-01-12 Thread GitBox
SamWheating commented on a change in pull request #20838: URL: https://github.com/apache/airflow/pull/20838#discussion_r783539524 ## File path: airflow/utils/metastore_cleanup.py ## @@ -0,0 +1,289 @@ +# Licensed to the Apache Software Foundation (ASF) under one +# or more

[GitHub] [airflow] github-actions[bot] closed pull request #18575: Resync DAG during next parse if error in ``sync_to_db``

2022-01-12 Thread GitBox
github-actions[bot] closed pull request #18575: URL: https://github.com/apache/airflow/pull/18575 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

[GitHub] [airflow] NadimYounes removed a comment on pull request #20245: Exclude snowflake-sqlalchemy v1.2.5

2022-01-12 Thread GitBox
NadimYounes removed a comment on pull request #20245: URL: https://github.com/apache/airflow/pull/20245#issuecomment-1011561961 https://github.com/apache/airflow/pull/20245#issuecomment-992387108 Can we please fix the constraints for 2.2.3? -- This is an automated message

[GitHub] [airflow] NadimYounes commented on pull request #20245: Exclude snowflake-sqlalchemy v1.2.5

2022-01-12 Thread GitBox
NadimYounes commented on pull request #20245: URL: https://github.com/apache/airflow/pull/20245#issuecomment-1011561961 https://github.com/apache/airflow/pull/20245#issuecomment-992387108 Can we please fix the constraints for 2.2.3? -- This is an automated message from the

[GitHub] [airflow] mik-laj commented on a change in pull request #20838: Add `maintenance cleanup` CLI command for purging old data

2022-01-12 Thread GitBox
mik-laj commented on a change in pull request #20838: URL: https://github.com/apache/airflow/pull/20838#discussion_r783528233 ## File path: airflow/utils/metastore_cleanup.py ## @@ -0,0 +1,289 @@ +# Licensed to the Apache Software Foundation (ASF) under one +# or more

[GitHub] [airflow] mik-laj commented on a change in pull request #20838: Add `maintenance cleanup` CLI command for purging old data

2022-01-12 Thread GitBox
mik-laj commented on a change in pull request #20838: URL: https://github.com/apache/airflow/pull/20838#discussion_r783528077 ## File path: airflow/utils/metastore_cleanup.py ## @@ -0,0 +1,289 @@ +# Licensed to the Apache Software Foundation (ASF) under one +# or more

[GitHub] [airflow] mik-laj commented on a change in pull request #20838: Add `maintenance cleanup` CLI command for purging old data

2022-01-12 Thread GitBox
mik-laj commented on a change in pull request #20838: URL: https://github.com/apache/airflow/pull/20838#discussion_r783523781 ## File path: airflow/cli/cli_parser.py ## @@ -1054,6 +1080,14 @@ class GroupCommand(NamedTuple): args=(ARG_CLEAR_ONLY,), ), )

[GitHub] [airflow] mik-laj commented on a change in pull request #20838: Add `maintenance cleanup` CLI command for purging old data

2022-01-12 Thread GitBox
mik-laj commented on a change in pull request #20838: URL: https://github.com/apache/airflow/pull/20838#discussion_r783523446 ## File path: airflow/cli/cli_parser.py ## @@ -1054,6 +1080,14 @@ class GroupCommand(NamedTuple): args=(ARG_CLEAR_ONLY,), ), )

[GitHub] [airflow] mik-laj commented on a change in pull request #20838: Add `maintenance cleanup` CLI command for purging old data

2022-01-12 Thread GitBox
mik-laj commented on a change in pull request #20838: URL: https://github.com/apache/airflow/pull/20838#discussion_r783520216 ## File path: airflow/cli/cli_parser.py ## @@ -378,6 +383,27 @@ def _check(value): ARG_CONF = Arg(('-c', '--conf'), help="JSON string that gets

[GitHub] [airflow] mik-laj commented on a change in pull request #20838: Add `maintenance cleanup` CLI command for purging old data

2022-01-12 Thread GitBox
mik-laj commented on a change in pull request #20838: URL: https://github.com/apache/airflow/pull/20838#discussion_r783519744 ## File path: airflow/utils/metastore_cleanup.py ## @@ -0,0 +1,289 @@ +# Licensed to the Apache Software Foundation (ASF) under one +# or more

[GitHub] [airflow] dstandish opened a new pull request #20838: Add `maintenance cleanup` CLI command for purging old data

2022-01-12 Thread GitBox
dstandish opened a new pull request #20838: URL: https://github.com/apache/airflow/pull/20838 Must supply "purge before date". Can optionally provide table list. Dry run will only print the number of rows meeting criteria. If not dry run, will require the user to confirm before

[GitHub] [airflow] SamWheating edited a comment on issue #20832: Unable to specify Python version for AwsGlueJobOperator

2022-01-12 Thread GitBox
SamWheating edited a comment on issue #20832: URL: https://github.com/apache/airflow/issues/20832#issuecomment-1011511506 Also for what its worth, I think that you `Command` block is invalid, as the `Command.Name` you're using (`abalone-preprocess`) must be one of `glueetl`, `pythonshell`

[GitHub] [airflow] SamWheating edited a comment on issue #20832: Unable to specify Python version for AwsGlueJobOperator

2022-01-12 Thread GitBox
SamWheating edited a comment on issue #20832: URL: https://github.com/apache/airflow/issues/20832#issuecomment-1011502821 I think that this is because the GlueHook is pretty opinionated and hardcodes the value of `Command` when running the `glue_client.create_job` command:

  1   2   3   >