[GitHub] [airflow] potiuk commented on pull request #33422: Make sure impersonation tests use a temporary `AIRFLOW_HOME`

2023-08-15 Thread via GitHub
potiuk commented on PR #33422: URL: https://github.com/apache/airflow/pull/33422#issuecomment-1680007971 > Well, sqlite test failure it is a bit expected, because impersonate user doesn't have access to `/root/airflow/sqlite/airflow.db` 樂 Should not it be using the database in the

[airflow] branch constraints-2-7 updated: Updating constraints. Github run id:5874754672

2023-08-15 Thread github-bot
This is an automated email from the ASF dual-hosted git repository. github-bot pushed a commit to branch constraints-2-7 in repository https://gitbox.apache.org/repos/asf/airflow.git The following commit(s) were added to refs/heads/constraints-2-7 by this push: new 6f6cca375c Updating

[GitHub] [airflow] Bowrna opened a new issue, #33426: Statsd metrics dagrun.schedule_delay sends time in different format against the seconds specified in docs

2023-08-15 Thread via GitHub
Bowrna opened a new issue, #33426: URL: https://github.com/apache/airflow/issues/33426 ### What do you see as an issue? For the airflow statsd metric received at the statsd client, the metric `dagrun.schedule_delay` described in airflow docs says it sends the seconds of delay

[airflow] 01/01: fixup! Update RELEASE_NOTES.rst

2023-08-15 Thread ephraimanierobi
This is an automated email from the ASF dual-hosted git repository. ephraimanierobi pushed a commit to branch v2-7-test in repository https://gitbox.apache.org/repos/asf/airflow.git commit c08c82e9dd0e4aabaa5121519819a636df635210 Author: Ephraim Anierobi AuthorDate: Mon Aug 14 12:16:57 2023

[airflow] branch v2-7-test updated (997c749856 -> c08c82e9dd)

2023-08-15 Thread ephraimanierobi
This is an automated email from the ASF dual-hosted git repository. ephraimanierobi pushed a change to branch v2-7-test in repository https://gitbox.apache.org/repos/asf/airflow.git omit 997c749856 Apply suggestions from code review omit b741d389ba fixup! Update RELEASE_NOTES.rst

[airflow] branch v2-7-test updated (b741d389ba -> 997c749856)

2023-08-15 Thread ephraimanierobi
This is an automated email from the ASF dual-hosted git repository. ephraimanierobi pushed a change to branch v2-7-test in repository https://gitbox.apache.org/repos/asf/airflow.git from b741d389ba fixup! Update RELEASE_NOTES.rst add 997c749856 Apply suggestions from code review No

[airflow] 01/02: Improve graph nesting logic (#33421)

2023-08-15 Thread ephraimanierobi
This is an automated email from the ASF dual-hosted git repository. ephraimanierobi pushed a commit to branch v2-7-test in repository https://gitbox.apache.org/repos/asf/airflow.git commit 57114c653891ac2cb5015a8f9e5011ebb9723220 Author: Brent Bovenzi AuthorDate: Wed Aug 16 00:38:46 2023 -0400

[airflow] 02/02: fixup! Update RELEASE_NOTES.rst

2023-08-15 Thread ephraimanierobi
This is an automated email from the ASF dual-hosted git repository. ephraimanierobi pushed a commit to branch v2-7-test in repository https://gitbox.apache.org/repos/asf/airflow.git commit b741d389ba0830d3d83913386101dc09484e37ae Author: Ephraim Anierobi AuthorDate: Mon Aug 14 12:16:57 2023

[airflow] branch v2-7-test updated (206699d059 -> b741d389ba)

2023-08-15 Thread ephraimanierobi
This is an automated email from the ASF dual-hosted git repository. ephraimanierobi pushed a change to branch v2-7-test in repository https://gitbox.apache.org/repos/asf/airflow.git omit 206699d059 fixup! Update RELEASE_NOTES.rst new 57114c6538 Improve graph nesting logic (#33421)

[airflow] branch main updated: Improve graph nesting logic (#33421)

2023-08-15 Thread ephraimanierobi
This is an automated email from the ASF dual-hosted git repository. ephraimanierobi pushed a commit to branch main in repository https://gitbox.apache.org/repos/asf/airflow.git The following commit(s) were added to refs/heads/main by this push: new 6547b8182f Improve graph nesting logic

[GitHub] [airflow] vandonr-amz commented on a diff in pull request #33423: Add a mechanism to warn if executors override existing CLI commands

2023-08-15 Thread via GitHub
vandonr-amz commented on code in PR #33423: URL: https://github.com/apache/airflow/pull/33423#discussion_r1295362936 ## airflow/cli/cli_parser.py: ## @@ -65,6 +65,22 @@ ALL_COMMANDS_DICT: dict[str, CLICommand] = {sp.name: sp for sp in airflow_commands} +# Check if

[GitHub] [airflow] uranusjr commented on a diff in pull request #33417: Add typing to concrete taskflow examples

2023-08-15 Thread via GitHub
uranusjr commented on code in PR #33417: URL: https://github.com/apache/airflow/pull/33417#discussion_r1295360220 ## docs/shared/template-examples/taskflow.rst: ## @@ -17,8 +17,12 @@ .. code-block:: python +from airflow.models.taskinstance import TaskInstance +

[GitHub] [airflow] uranusjr commented on a diff in pull request #33413: Add read only validation to read only fields

2023-08-15 Thread via GitHub
uranusjr commented on code in PR #33413: URL: https://github.com/apache/airflow/pull/33413#discussion_r1295358300 ## airflow/www/forms.py: ## @@ -124,21 +124,28 @@ class DagRunEditForm(DynamicForm): We don't actually want to allow editing, so everything is read-only here.

[GitHub] [airflow] Lee-W commented on a diff in pull request #33405: respect "soft_fail" argument when running BatchSensor in deferrable mode

2023-08-15 Thread via GitHub
Lee-W commented on code in PR #33405: URL: https://github.com/apache/airflow/pull/33405#discussion_r1295349323 ## airflow/providers/amazon/aws/sensors/batch.py: ## @@ -115,7 +115,15 @@ def execute_complete(self, context: Context, event: dict[str, Any]) -> None: Relies

[GitHub] [airflow] uranusjr commented on a diff in pull request #33405: respect "soft_fail" argument when running BatchSensor in deferrable mode

2023-08-15 Thread via GitHub
uranusjr commented on code in PR #33405: URL: https://github.com/apache/airflow/pull/33405#discussion_r1295347917 ## airflow/providers/amazon/aws/sensors/batch.py: ## @@ -115,7 +115,15 @@ def execute_complete(self, context: Context, event: dict[str, Any]) -> None:

[GitHub] [airflow] Lee-W commented on a diff in pull request #33403: Respect "soft_fail" for core async sensors

2023-08-15 Thread via GitHub
Lee-W commented on code in PR #33403: URL: https://github.com/apache/airflow/pull/33403#discussion_r1295335522 ## airflow/triggers/temporal.py: ## @@ -34,18 +35,29 @@ class DateTimeTrigger(BaseTrigger): The provided datetime MUST be in UTC. """ -def

[GitHub] [airflow] Lee-W commented on a diff in pull request #33405: respect "soft_fail" argument when running BatchSensor in deferrable mode

2023-08-15 Thread via GitHub
Lee-W commented on code in PR #33405: URL: https://github.com/apache/airflow/pull/33405#discussion_r1295333987 ## airflow/providers/amazon/aws/sensors/batch.py: ## @@ -115,7 +115,15 @@ def execute_complete(self, context: Context, event: dict[str, Any]) -> None: Relies

[GitHub] [airflow] uranusjr commented on pull request #33425: Add sql_hook_params to S3ToSql and SqlToS3 operators

2023-08-15 Thread via GitHub
uranusjr commented on PR #33425: URL: https://github.com/apache/airflow/pull/33425#issuecomment-1679891886 Because you changed how the BaseHook is accessed and it breaks the mock. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [airflow] uranusjr commented on a diff in pull request #33405: respect "soft_fail" argument when running BatchSensor in deferrable mode

2023-08-15 Thread via GitHub
uranusjr commented on code in PR #33405: URL: https://github.com/apache/airflow/pull/33405#discussion_r1295331003 ## airflow/providers/amazon/aws/sensors/batch.py: ## @@ -115,7 +115,15 @@ def execute_complete(self, context: Context, event: dict[str, Any]) -> None:

[GitHub] [airflow] uranusjr commented on a diff in pull request #33403: Respect "soft_fail" for core async sensors

2023-08-15 Thread via GitHub
uranusjr commented on code in PR #33403: URL: https://github.com/apache/airflow/pull/33403#discussion_r1295330532 ## airflow/triggers/temporal.py: ## @@ -34,18 +35,29 @@ class DateTimeTrigger(BaseTrigger): The provided datetime MUST be in UTC. """ -def

[GitHub] [airflow] Lee-W commented on a diff in pull request #33405: respect "soft_fail" argument when running BatchSensor in deferrable mode

2023-08-15 Thread via GitHub
Lee-W commented on code in PR #33405: URL: https://github.com/apache/airflow/pull/33405#discussion_r1295327281 ## airflow/providers/amazon/aws/sensors/__init__.py: ## Review Comment: Sounds good. I just move it to `airflow/providers/amazon/aws/sensors/utils.py` --

[GitHub] [airflow] Lee-W commented on a diff in pull request #33403: Respect "soft_fail" for core async sensors

2023-08-15 Thread via GitHub
Lee-W commented on code in PR #33403: URL: https://github.com/apache/airflow/pull/33403#discussion_r1295321179 ## airflow/triggers/temporal.py: ## @@ -34,18 +35,29 @@ class DateTimeTrigger(BaseTrigger): The provided datetime MUST be in UTC. """ -def

[GitHub] [airflow] alexbegg commented on pull request #33425: Add sql_hook_params to S3ToSql and SqlToS3 operators

2023-08-15 Thread via GitHub
alexbegg commented on PR #33425: URL: https://github.com/apache/airflow/pull/33425#issuecomment-1679857091 FYI, I added a test to tests/providers/amazon/aws/transfers/test_s3_to_sql.py that is passing, but the other 2 tests in that file are failing. The code for those tests has not

[GitHub] [airflow] uranusjr commented on a diff in pull request #33403: Respect "soft_fail" for core async sensors

2023-08-15 Thread via GitHub
uranusjr commented on code in PR #33403: URL: https://github.com/apache/airflow/pull/33403#discussion_r1295291332 ## airflow/triggers/temporal.py: ## @@ -34,18 +35,29 @@ class DateTimeTrigger(BaseTrigger): The provided datetime MUST be in UTC. """ -def

[GitHub] [airflow] uranusjr commented on a diff in pull request #33403: Respect "soft_fail" for core async sensors

2023-08-15 Thread via GitHub
uranusjr commented on code in PR #33403: URL: https://github.com/apache/airflow/pull/33403#discussion_r1295290027 ## airflow/triggers/temporal.py: ## @@ -34,18 +35,29 @@ class DateTimeTrigger(BaseTrigger): The provided datetime MUST be in UTC. """ -def

[GitHub] [airflow] uranusjr commented on a diff in pull request #33405: respect "soft_fail" argument when running BatchSensor in deferrable mode

2023-08-15 Thread via GitHub
uranusjr commented on code in PR #33405: URL: https://github.com/apache/airflow/pull/33405#discussion_r1295289373 ## airflow/providers/amazon/aws/sensors/__init__.py: ## Review Comment: Better put this in somewhere like `utils.py`; importing `__init__` in child modules

[GitHub] [airflow] thesuperzapper commented on pull request #31829: Disable celery `task_acks_late`

2023-08-15 Thread via GitHub
thesuperzapper commented on PR #31829: URL: https://github.com/apache/airflow/pull/31829#issuecomment-1679827878 @hussein-awala have you had any chance to look into it? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use

[GitHub] [airflow] uranusjr commented on a diff in pull request #33423: Add a mechanism to warn if executors override existing CLI commands

2023-08-15 Thread via GitHub
uranusjr commented on code in PR #33423: URL: https://github.com/apache/airflow/pull/33423#discussion_r1295279424 ## airflow/cli/cli_parser.py: ## @@ -65,6 +65,22 @@ ALL_COMMANDS_DICT: dict[str, CLICommand] = {sp.name: sp for sp in airflow_commands} +# Check if

[GitHub] [airflow] Lee-W commented on pull request #33403: Respect "soft_fail" for core async sensors

2023-08-15 Thread via GitHub
Lee-W commented on PR #33403: URL: https://github.com/apache/airflow/pull/33403#issuecomment-1679821158 > Let's wait #33424 before deciding if we need to merge this one. Hi, I just checked #33424. We might still need this one, as the exceptions are raised in the `execute` method

[GitHub] [airflow] alexbegg opened a new pull request, #33425: Adding sql_hook_params to S3ToSql and SqlToS3 operators

2023-08-15 Thread via GitHub
alexbegg opened a new pull request, #33425: URL: https://github.com/apache/airflow/pull/33425 Adding `sql_hook_params` parameter to `S3ToSqlOperator` and `SqlToS3Operator`. This will allow you to pass extra config params to the underlying SQL hook. This uses the same

[GitHub] [airflow] github-actions[bot] closed pull request #31206: Add default value to data columns in SQLtoS3Operator

2023-08-15 Thread via GitHub
github-actions[bot] closed pull request #31206: Add default value to data columns in SQLtoS3Operator URL: https://github.com/apache/airflow/pull/31206 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go

[GitHub] [airflow] github-actions[bot] closed pull request #31637: Respect `AUTOCOMMIT` in `session_parameters` of `SnowflakeHook`

2023-08-15 Thread via GitHub
github-actions[bot] closed pull request #31637: Respect `AUTOCOMMIT` in `session_parameters` of `SnowflakeHook` URL: https://github.com/apache/airflow/pull/31637 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL

[GitHub] [airflow] jedcunningham commented on pull request #31066: Support naming customization on helm chart resources

2023-08-15 Thread via GitHub
jedcunningham commented on PR #31066: URL: https://github.com/apache/airflow/pull/31066#issuecomment-1679782581 Thanks @fgalind1! Congrats on your first commit  -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL

[GitHub] [airflow] boring-cyborg[bot] commented on pull request #31066: Support naming customization on helm chart resources

2023-08-15 Thread via GitHub
boring-cyborg[bot] commented on PR #31066: URL: https://github.com/apache/airflow/pull/31066#issuecomment-1679782203 Awesome work, congrats on your first merged pull request! You are invited to check our [Issue Tracker](https://github.com/apache/airflow/issues) for additional

[airflow] branch main updated: Support naming customization on helm chart resources (#31066)

2023-08-15 Thread jedcunningham
This is an automated email from the ASF dual-hosted git repository. jedcunningham pushed a commit to branch main in repository https://gitbox.apache.org/repos/asf/airflow.git The following commit(s) were added to refs/heads/main by this push: new 1a828f393b Support naming customization on

[GitHub] [airflow] jedcunningham merged pull request #31066: Support naming customization on helm chart resources

2023-08-15 Thread via GitHub
jedcunningham merged PR #31066: URL: https://github.com/apache/airflow/pull/31066 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

[GitHub] [airflow] vandonr-amz commented on a diff in pull request #33423: Add a mechanism to warn if executors override existing CLI commands

2023-08-15 Thread via GitHub
vandonr-amz commented on code in PR #33423: URL: https://github.com/apache/airflow/pull/33423#discussion_r1295252828 ## airflow/cli/cli_parser.py: ## Review Comment: > Yeah, that's what I meant, raise an exception. ok :p So, yeah, that'd be good if an executor

[GitHub] [airflow] vandonr-amz commented on a diff in pull request #33423: Add a mechanism to warn if executors override existing CLI commands

2023-08-15 Thread via GitHub
vandonr-amz commented on code in PR #33423: URL: https://github.com/apache/airflow/pull/33423#discussion_r1295251837 ## airflow/cli/cli_parser.py: ## Review Comment: there is `log.fatal` which is marked deprecated in favor of `log.critical`, the highest level of logging.

[GitHub] [airflow] jedcunningham commented on a diff in pull request #33423: Add a mechanism to warn if executors override existing CLI commands

2023-08-15 Thread via GitHub
jedcunningham commented on code in PR #33423: URL: https://github.com/apache/airflow/pull/33423#discussion_r1295251461 ## airflow/cli/cli_parser.py: ## Review Comment: Yeah, that's what I meant, raise an exception. -- This is an automated message from the Apache Git

[GitHub] [airflow] o-nikolas commented on a diff in pull request #33423: Add a mechanism to warn if executors override existing CLI commands

2023-08-15 Thread via GitHub
o-nikolas commented on code in PR #33423: URL: https://github.com/apache/airflow/pull/33423#discussion_r1295241845 ## airflow/cli/cli_parser.py: ## Review Comment: To me fatal means abort the whole thing and exit, there's no fatal log level. _But_ if that's indeed not

[GitHub] [airflow] vandonr-amz commented on a diff in pull request #33423: Add a mechanism to warn if executors override existing CLI commands

2023-08-15 Thread via GitHub
vandonr-amz commented on code in PR #33423: URL: https://github.com/apache/airflow/pull/33423#discussion_r1295240828 ## airflow/cli/cli_parser.py: ## Review Comment: I think Jed is not talking about an exception but more about the level of logging to use (i.e. use

[GitHub] [airflow] o-nikolas commented on a diff in pull request #33423: Add a mechanism to warn if executors override existing CLI commands

2023-08-15 Thread via GitHub
o-nikolas commented on code in PR #33423: URL: https://github.com/apache/airflow/pull/33423#discussion_r1295239347 ## airflow/cli/cli_parser.py: ## Review Comment: Yeah, that's a tough one. I see both sides. If a user really wants their cli command to have a name already

[GitHub] [airflow] vandonr-amz commented on a diff in pull request #33423: Add a mechanism to warn if executors override existing CLI commands

2023-08-15 Thread via GitHub
vandonr-amz commented on code in PR #33423: URL: https://github.com/apache/airflow/pull/33423#discussion_r1295222030 ## airflow/cli/cli_parser.py: ## Review Comment: Yeah, I wasn't sure... It's not preventing airflow from working, but it could be something needing

[airflow] branch main updated: Remove uploading to test pypi for releases (#33412)

2023-08-15 Thread jedcunningham
This is an automated email from the ASF dual-hosted git repository. jedcunningham pushed a commit to branch main in repository https://gitbox.apache.org/repos/asf/airflow.git The following commit(s) were added to refs/heads/main by this push: new 23dfa7ffed Remove uploading to test pypi

[GitHub] [airflow] jedcunningham merged pull request #33412: Remove uploading to test pypi for releases

2023-08-15 Thread via GitHub
jedcunningham merged PR #33412: URL: https://github.com/apache/airflow/pull/33412 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

[GitHub] [airflow] jedcunningham commented on a diff in pull request #33423: Add a mechanism to warn if executors override existing CLI commands

2023-08-15 Thread via GitHub
jedcunningham commented on code in PR #33423: URL: https://github.com/apache/airflow/pull/33423#discussion_r1295217210 ## airflow/cli/cli_parser.py: ## Review Comment: I wonder if instead of just warning, we should make it fatal? -- This is an automated message from

[GitHub] [airflow] vandonr-amz commented on a diff in pull request #33213: Implement `is_authorized()` in auth manager

2023-08-15 Thread via GitHub
vandonr-amz commented on code in PR #33213: URL: https://github.com/apache/airflow/pull/33213#discussion_r1295210703 ## airflow/auth/managers/fab/fab_auth_manager.py: ## @@ -17,13 +17,37 @@ # under the License. from __future__ import annotations +import itertools + from

[GitHub] [airflow] hussein-awala opened a new pull request, #33424: Create a new method used to resume the task in order to implement specific logic for operators

2023-08-15 Thread via GitHub
hussein-awala opened a new pull request, #33424: URL: https://github.com/apache/airflow/pull/33424 related: #33405 related: #33196 This PR creates a new method used by the TI to resume the execution, and moves handling the triggers errors to this method. In this way we

[GitHub] [airflow] vandonr-amz opened a new pull request, #33423: Add a mechanism to warn if executors override existing CLI commands

2023-08-15 Thread via GitHub
vandonr-amz opened a new pull request, #33423: URL: https://github.com/apache/airflow/pull/33423 Currently, if an executor redefines and existing command, I think it silently overrides the existing one, which can be an issue. I'm in the process of adding one more way to add commands, so

[GitHub] [airflow] jedcunningham commented on a diff in pull request #32724: feat(chart): Add existing secret support for PGBouncer metrics exporter.

2023-08-15 Thread via GitHub
jedcunningham commented on code in PR #32724: URL: https://github.com/apache/airflow/pull/32724#discussion_r1295190150 ## chart/templates/secrets/pgbouncer-stats-secret.yaml: ## Review Comment: I agree there isn't much point, but if the env var uses the config... the

[GitHub] [airflow] hussein-awala commented on a diff in pull request #33406: fix(TimeSensorAsync): use DAG timezone

2023-08-15 Thread via GitHub
hussein-awala commented on code in PR #33406: URL: https://github.com/apache/airflow/pull/33406#discussion_r1295148713 ## airflow/sensors/time_sensor.py: ## @@ -64,7 +64,7 @@ def __init__(self, *, target_time, **kwargs): self.target_time = target_time

[GitHub] [airflow] hussein-awala commented on pull request #33413: Add read only validation to read only fields

2023-08-15 Thread via GitHub
hussein-awala commented on PR #33413: URL: https://github.com/apache/airflow/pull/33413#issuecomment-1679659825 Actually it is not related to this change, and it was fixed by #33416, I merge main -- This is an automated message from the Apache Git Service. To respond to the message,

[airflow] branch main updated (60df70526a -> 88b8ba9861)

2023-08-15 Thread husseinawala
This is an automated email from the ASF dual-hosted git repository. husseinawala pushed a change to branch main in repository https://gitbox.apache.org/repos/asf/airflow.git from 60df70526a Add parameter to pass role ARN to `GlueJobOperator ` (#33408) add 88b8ba9861 Update gcp.rst

[GitHub] [airflow] hussein-awala merged pull request #33420: Update gcp.rst

2023-08-15 Thread via GitHub
hussein-awala merged PR #33420: URL: https://github.com/apache/airflow/pull/33420 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

[GitHub] [airflow] hussein-awala commented on pull request #33412: Remove uploading to test pypi for releases

2023-08-15 Thread via GitHub
hussein-awala commented on PR #33412: URL: https://github.com/apache/airflow/pull/33412#issuecomment-1679658187 The failed test is not related to this change, and it was fixed by #33416 -- This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [airflow] hussein-awala commented on pull request #33413: Add read only validation to read only fields

2023-08-15 Thread via GitHub
hussein-awala commented on PR #33413: URL: https://github.com/apache/airflow/pull/33413#issuecomment-1679648730 Could you check the failed static checks? Here is the doc: https://github.com/apache/airflow/blob/main/STATIC_CODE_CHECKS.rst -- This is an automated message from the Apache

[GitHub] [airflow] vincbeck commented on a diff in pull request #33213: Implement `is_authorized()` in auth manager

2023-08-15 Thread via GitHub
vincbeck commented on code in PR #33213: URL: https://github.com/apache/airflow/pull/33213#discussion_r1295119967 ## airflow/auth/managers/fab/fab_auth_manager.py: ## @@ -17,13 +17,37 @@ # under the License. from __future__ import annotations +import itertools + from flask

[airflow] branch main updated (cc360b73c9 -> 60df70526a)

2023-08-15 Thread vincbeck
This is an automated email from the ASF dual-hosted git repository. vincbeck pushed a change to branch main in repository https://gitbox.apache.org/repos/asf/airflow.git from cc360b73c9 Fixing static checks for FabAirflowSecurityManagerOverride (#33416) add 60df70526a Add parameter to

[GitHub] [airflow] vincbeck merged pull request #33408: Add parameter to pass role ARN to `GlueJobOperator `

2023-08-15 Thread via GitHub
vincbeck merged PR #33408: URL: https://github.com/apache/airflow/pull/33408 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

[GitHub] [airflow] vandonr-amz commented on a diff in pull request #33213: Implement `is_authorized()` in auth manager

2023-08-15 Thread via GitHub
vandonr-amz commented on code in PR #33213: URL: https://github.com/apache/airflow/pull/33213#discussion_r1295115914 ## airflow/auth/managers/fab/fab_auth_manager.py: ## @@ -17,13 +17,37 @@ # under the License. from __future__ import annotations +import itertools + from

[GitHub] [airflow] hussein-awala commented on a diff in pull request #33420: Update gcp.rst

2023-08-15 Thread via GitHub
hussein-awala commented on code in PR #33420: URL: https://github.com/apache/airflow/pull/33420#discussion_r1295123482 ## docs/apache-airflow-providers-google/connections/gcp.rst: ## @@ -27,7 +27,7 @@ The Google Cloud connection type enables the Google Cloud Integrations.

[GitHub] [airflow] potiuk commented on pull request #33355: Add Python Virtualenv Operator Caching

2023-08-15 Thread via GitHub
potiuk commented on PR #33355: URL: https://github.com/apache/airflow/pull/33355#issuecomment-1679615177 > I propose I could add a note to the docs. Yes. Note in the docs should be sufficient (avoid modifying the venv after it is created or similar ) . checking venv context is not a

[GitHub] [airflow] fdemiane commented on pull request #32606: Add `CloudBatchHook` and operators

2023-08-15 Thread via GitHub
fdemiane commented on PR #32606: URL: https://github.com/apache/airflow/pull/32606#issuecomment-1679624577 Hello @potiuk, I need a reviewer to review and accept the pull request. @ahidalgob already reviewed it. I would appreciate your assistance. Thanks! -- This is an automated

[GitHub] [airflow] hussein-awala commented on issue #33402: Scheduler fails to remove pods in Completed state

2023-08-15 Thread via GitHub
hussein-awala commented on issue #33402: URL: https://github.com/apache/airflow/issues/33402#issuecomment-1679598748 Actually we set the resource version to 0 on failure: ```python except Exception: self.log.exception("Unknown error in

[GitHub] [airflow] potiuk commented on a diff in pull request #33355: Add Python Virtualenv Operator Caching

2023-08-15 Thread via GitHub
potiuk commented on code in PR #33355: URL: https://github.com/apache/airflow/pull/33355#discussion_r1295118289 ## airflow/operators/python.py: ## @@ -606,7 +613,39 @@ def _prepare_venv(self, venv_path: Path) -> None: index_urls=self.index_urls, ) +

[GitHub] [airflow] vincbeck commented on a diff in pull request #33213: Implement `is_authorized()` in auth manager

2023-08-15 Thread via GitHub
vincbeck commented on code in PR #33213: URL: https://github.com/apache/airflow/pull/33213#discussion_r1295078542 ## airflow/auth/managers/fab/fab_auth_manager.py: ## @@ -17,13 +17,37 @@ # under the License. from __future__ import annotations +import itertools + from flask

[GitHub] [airflow] vincbeck commented on a diff in pull request #33213: Implement `is_authorized()` in auth manager

2023-08-15 Thread via GitHub
vincbeck commented on code in PR #33213: URL: https://github.com/apache/airflow/pull/33213#discussion_r1295115248 ## airflow/auth/managers/fab/fab_auth_manager.py: ## @@ -17,13 +17,37 @@ # under the License. from __future__ import annotations +import itertools + from flask

[GitHub] [airflow] vincbeck commented on a diff in pull request #33213: Implement `is_authorized()` in auth manager

2023-08-15 Thread via GitHub
vincbeck commented on code in PR #33213: URL: https://github.com/apache/airflow/pull/33213#discussion_r1295081179 ## airflow/auth/managers/fab/fab_auth_manager.py: ## @@ -81,3 +148,35 @@ def get_url_user_profile(self) -> str | None: if not

[GitHub] [airflow] vandonr-amz commented on a diff in pull request #33213: Implement `is_authorized()` in auth manager

2023-08-15 Thread via GitHub
vandonr-amz commented on code in PR #33213: URL: https://github.com/apache/airflow/pull/33213#discussion_r1295115914 ## airflow/auth/managers/fab/fab_auth_manager.py: ## @@ -17,13 +17,37 @@ # under the License. from __future__ import annotations +import itertools + from

[GitHub] [airflow] yvesmarinho commented on issue #32727: airflow.providers.google.cloud.operators.gcs.GCSListObjectsOperator not filter file

2023-08-15 Thread via GitHub
yvesmarinho commented on issue #32727: URL: https://github.com/apache/airflow/issues/32727#issuecomment-1679588811 please give me more days and i will come back with the test result -- This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [airflow] hussein-awala commented on issue #33402: Scheduler fails to remove pods in Completed state

2023-08-15 Thread via GitHub
hussein-awala commented on issue #33402: URL: https://github.com/apache/airflow/issues/33402#issuecomment-1679565485 I have two questions: 1. I see in the log: ``` Deleted pod: TaskInstanceKey(dag_id='SAP_WMD_AGWI_RSRVD_TIMEOUT_H', task_id='SAP_WMD_AGWI_RSRVD_TIMEOUT_H',

[GitHub] [airflow] bbovenzi opened a new pull request, #33421: Improve graph nesting logic

2023-08-15 Thread via GitHub
bbovenzi opened a new pull request, #33421: URL: https://github.com/apache/airflow/pull/33421 Fix how we handle nested groups to make sure edge labels inside of groups are correctly positions and that nested nodes with the same label don't mess with rendering. | Before |

[GitHub] [airflow] Dan-Git-Rep opened a new pull request, #33420: Update gcp.rst

2023-08-15 Thread via GitHub
Dan-Git-Rep opened a new pull request, #33420: URL: https://github.com/apache/airflow/pull/33420 three options to connect to GCP not the initial specified two --- **^ Add meaningful description above** Read the **[Pull Request

[GitHub] [airflow] potiuk commented on pull request #33419: Revert "Remove airflow user from `Dockerfile.ci` (#31970)"

2023-08-15 Thread via GitHub
potiuk commented on PR #33419: URL: https://github.com/apache/airflow/pull/33419#issuecomment-1679539083 It turned out that airflow user is useful for SSH debugging. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the

[GitHub] [airflow] ferruzzi merged pull request #33416: Fixing static checks for FabAirflowSecurityManagerOverride

2023-08-15 Thread via GitHub
ferruzzi merged PR #33416: URL: https://github.com/apache/airflow/pull/33416 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

[GitHub] [airflow] Taragolis opened a new pull request, #33422: Make sure impersonation tests use a temporary `AIRFLOW_HOME`

2023-08-15 Thread via GitHub
Taragolis opened a new pull request, #33422: URL: https://github.com/apache/airflow/pull/33422 Continuation of #30313 ``` The impersonation tests of ours modify permissions for the AIRFLOW_HOME recursively. This is quite a bad idea when the tests are executed

[airflow] branch main updated (3766ab07d7 -> cc360b73c9)

2023-08-15 Thread ferruzzi
This is an automated email from the ASF dual-hosted git repository. ferruzzi pushed a change to branch main in repository https://gitbox.apache.org/repos/asf/airflow.git from 3766ab07d7 More complete fix for dev release scripts to filter commit for PR (#33418) add cc360b73c9 Fixing

[GitHub] [airflow] vincbeck commented on a diff in pull request #33213: Implement `is_authorized()` in auth manager

2023-08-15 Thread via GitHub
vincbeck commented on code in PR #33213: URL: https://github.com/apache/airflow/pull/33213#discussion_r1295078542 ## airflow/auth/managers/fab/fab_auth_manager.py: ## @@ -17,13 +17,37 @@ # under the License. from __future__ import annotations +import itertools + from flask

[GitHub] [airflow] boring-cyborg[bot] commented on pull request #33420: Update gcp.rst

2023-08-15 Thread via GitHub
boring-cyborg[bot] commented on PR #33420: URL: https://github.com/apache/airflow/pull/33420#issuecomment-1679559348 Congratulations on your first Pull Request and welcome to the Apache Airflow community! If you have any issues or are unsure about any anything please check our Contribution

[GitHub] [airflow] potiuk merged pull request #33418: More complete fix for dev release scripts to filter commit for PR

2023-08-15 Thread via GitHub
potiuk merged PR #33418: URL: https://github.com/apache/airflow/pull/33418 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

[GitHub] [airflow] shahar1 commented on pull request #33419: Revert "Remove airflow user from `Dockerfile.ci` (#31970)"

2023-08-15 Thread via GitHub
shahar1 commented on PR #33419: URL: https://github.com/apache/airflow/pull/33419#issuecomment-1679545805 It might be worth adding documentation of where/how it's being used to avoid (non-)accidental deletion for the next time :) -- This is an automated message from the Apache Git

[GitHub] [airflow] potiuk commented on pull request #31970: Remove the creation of `airflow` user from `Dockerfile.ci`

2023-08-15 Thread via GitHub
potiuk commented on PR #31970: URL: https://github.com/apache/airflow/pull/31970#issuecomment-1679534537 Ah... WE need to revert this one, It was actually used to enable ssh debugging. -- This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [airflow] potiuk commented on pull request #33419: Revert "Remove airflow user from `Dockerfile.ci` (#31970)"

2023-08-15 Thread via GitHub
potiuk commented on PR #33419: URL: https://github.com/apache/airflow/pull/33419#issuecomment-1679539859 Following the slack conversation: https://apache-airflow.slack.com/archives/CQ9QHSFQX/p1692085262768459 -- This is an automated message from the Apache Git Service. To respond to the

[GitHub] [airflow] potiuk opened a new pull request, #33418: More complete fix for dev release scripts to filter commit for PR

2023-08-15 Thread via GitHub
potiuk opened a new pull request, #33418: URL: https://github.com/apache/airflow/pull/33418 This is a more complete fix to #33411. This is also a follow up on earlier implementation of #33261 that addressed checking if PRs are merged. This one applies the same pattern to finding commit but

[GitHub] [airflow] jens-scheffler-bosch commented on pull request #33355: Add Python Virtualenv Operator Caching

2023-08-15 Thread via GitHub
jens-scheffler-bosch commented on PR #33355: URL: https://github.com/apache/airflow/pull/33355#issuecomment-1679517355 > Interesting approach. Glad that you hought about lock and hash @jens-scheffler-bosch . I could think about the case where the venv will be partially created if it fails

[GitHub] [airflow] potiuk opened a new pull request, #33419: Revert "Remove airflow user from `Dockerfile.ci` (#31970)"

2023-08-15 Thread via GitHub
potiuk opened a new pull request, #33419: URL: https://github.com/apache/airflow/pull/33419 This reverts commit a29887451fa4807e56fb60e9ef72af4bcdf4924f. --- **^ Add meaningful description above** Read the **[Pull Request

[GitHub] [airflow] ferruzzi commented on pull request #33416: Fixing static checks for FabAirflowSecurityManagerOverride

2023-08-15 Thread via GitHub
ferruzzi commented on PR #33416: URL: https://github.com/apache/airflow/pull/33416#issuecomment-1679501589 LGTM, I'll approve and merge it as soon as the CI goes green. Thanks for that. I made sure everything that was already committed passed, but there are bound to be a couple that

[airflow] branch main updated (5b104a9020 -> 3766ab07d7)

2023-08-15 Thread potiuk
This is an automated email from the ASF dual-hosted git repository. potiuk pushed a change to branch main in repository https://gitbox.apache.org/repos/asf/airflow.git from 5b104a9020 Fix getting correct commit from multiple referenced PR (#33411) add 3766ab07d7 More complete fix for

[GitHub] [airflow] vincbeck commented on a diff in pull request #33219: Added Amazon SageMaker Notebook hook and operators

2023-08-15 Thread via GitHub
vincbeck commented on code in PR #33219: URL: https://github.com/apache/airflow/pull/33219#discussion_r1294992440 ## airflow/providers/amazon/aws/operators/sagemaker.py: ## @@ -1523,3 +1524,246 @@ def execute(self, context: Context) -> str: arn = ans["ExperimentArn"]

[GitHub] [airflow] eumiro commented on a diff in pull request #33234: Refactor: Simplify code in smaller providers

2023-08-15 Thread via GitHub
eumiro commented on code in PR #33234: URL: https://github.com/apache/airflow/pull/33234#discussion_r1295021450 ## airflow/providers/http/hooks/http.py: ## @@ -363,8 +360,10 @@ async def run( else: raise AirflowException(f"Unexpected HTTP Method:

[GitHub] [airflow] eumiro commented on a diff in pull request #33227: Refactor: Simplify code in providers/apache

2023-08-15 Thread via GitHub
eumiro commented on code in PR #33227: URL: https://github.com/apache/airflow/pull/33227#discussion_r1295013074 ## airflow/providers/apache/livy/hooks/livy.py: ## @@ -448,7 +448,7 @@ def _validate_extra_conf(conf: dict[Any, Any]) -> bool: if conf: if not

[GitHub] [airflow] alexbegg commented on pull request #33416: Fixing static checks for FabAirflowSecurityManagerOverride

2023-08-15 Thread via GitHub
alexbegg commented on PR #33416: URL: https://github.com/apache/airflow/pull/33416#issuecomment-1679479589 I fixed a typo in the commit message -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to

[GitHub] [airflow] vincbeck commented on a diff in pull request #33219: Added Amazon SageMaker Notebook hook and operators

2023-08-15 Thread via GitHub
vincbeck commented on code in PR #33219: URL: https://github.com/apache/airflow/pull/33219#discussion_r1294990193 ## airflow/providers/amazon/aws/operators/sagemaker.py: ## @@ -1523,3 +1524,246 @@ def execute(self, context: Context) -> str: arn = ans["ExperimentArn"]

[GitHub] [airflow] alexbegg commented on pull request #33416: Fixing status checks for FabAirflowSecurityManagerOverride

2023-08-15 Thread via GitHub
alexbegg commented on PR #33416: URL: https://github.com/apache/airflow/pull/33416#issuecomment-1679426859 @ferruzzi it looks like the PR for these files being added passed the status check before your PR changing adding rule D205, but then got merged after. This hopefully should fix

[GitHub] [airflow] timkpaine commented on pull request #30083: Account for change in UTC offset when performing next schedule calculations

2023-08-15 Thread via GitHub
timkpaine commented on PR #30083: URL: https://github.com/apache/airflow/pull/30083#issuecomment-1679399616 Let me know if there's anything else I can help with -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL

[GitHub] [airflow] vincbeck commented on a diff in pull request #33219: Added Amazon SageMaker Notebook hook and operators

2023-08-15 Thread via GitHub
vincbeck commented on code in PR #33219: URL: https://github.com/apache/airflow/pull/33219#discussion_r1294992979 ## airflow/providers/amazon/aws/operators/sagemaker.py: ## @@ -1523,3 +1524,246 @@ def execute(self, context: Context) -> str: arn = ans["ExperimentArn"]

[GitHub] [airflow] jens-scheffler-bosch opened a new pull request, #33417: Add typing to concrete taskflow examples

2023-08-15 Thread via GitHub
jens-scheffler-bosch opened a new pull request, #33417: URL: https://github.com/apache/airflow/pull/33417 I just saw the PR #33296 in Slack and really liked it - I was struggeling to find concrete examples myself multiple times. Thanks to @RNHTTR for making this! When taking a look

[GitHub] [airflow] vincbeck commented on a diff in pull request #33219: Added Amazon SageMaker Notebook hook and operators

2023-08-15 Thread via GitHub
vincbeck commented on code in PR #33219: URL: https://github.com/apache/airflow/pull/33219#discussion_r1294989565 ## airflow/providers/amazon/aws/operators/sagemaker.py: ## @@ -1523,3 +1524,246 @@ def execute(self, context: Context) -> str: arn = ans["ExperimentArn"]

[GitHub] [airflow] vincbeck commented on a diff in pull request #33219: Added Amazon SageMaker Notebook hook and operators

2023-08-15 Thread via GitHub
vincbeck commented on code in PR #33219: URL: https://github.com/apache/airflow/pull/33219#discussion_r1294992827 ## airflow/providers/amazon/aws/operators/sagemaker.py: ## @@ -1523,3 +1524,246 @@ def execute(self, context: Context) -> str: arn = ans["ExperimentArn"]

[GitHub] [airflow] vincbeck commented on a diff in pull request #33219: Added Amazon SageMaker Notebook hook and operators

2023-08-15 Thread via GitHub
vincbeck commented on code in PR #33219: URL: https://github.com/apache/airflow/pull/33219#discussion_r1294993349 ## tests/providers/amazon/aws/hooks/test_sagemaker_notebook.py: ## Review Comment: You can remove this file -- This is an automated message from the Apache

  1   2   >