[GitHub] [airflow] uranusjr commented on pull request #31987: Load just the first yielded event in the triggerer

2023-08-09 Thread via GitHub
uranusjr commented on PR #31987: URL: https://github.com/apache/airflow/pull/31987#issuecomment-1672602428 I experiemented this a bit. It seems like the most difficult part is actually to convince Mypy `run()` can either be implemented as an async generator function or a coroutine

[airflow] branch constraints-2-7 updated: Updating constraints. Github run id:5814087264

2023-08-09 Thread github-bot
This is an automated email from the ASF dual-hosted git repository. github-bot pushed a commit to branch constraints-2-7 in repository https://gitbox.apache.org/repos/asf/airflow.git The following commit(s) were added to refs/heads/constraints-2-7 by this push: new 0bb07c0366 Updating

[GitHub] [airflow] dstandish commented on pull request #31987: Load just the first yielded event in the triggerer

2023-08-09 Thread via GitHub
dstandish commented on PR #31987: URL: https://github.com/apache/airflow/pull/31987#issuecomment-1672559192 > An `isawaitable` check would cover this pretty easily. nice -- This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [airflow] uranusjr commented on pull request #31987: Load just the first yielded event in the triggerer

2023-08-09 Thread via GitHub
uranusjr commented on PR #31987: URL: https://github.com/apache/airflow/pull/31987#issuecomment-1672556380 > If it's coroutine we can await it and obtain the return. If it is asynciterator, we cannot await it but must iterate it. An `isawaitable` check would cover this pretty easily.

[GitHub] [airflow] uranusjr commented on pull request #33271: Change type of defualts for include_deferred

2023-08-09 Thread via GitHub
uranusjr commented on PR #33271: URL: https://github.com/apache/airflow/pull/33271#issuecomment-1672554384 Gosh no, this is exactly the thing I rejected in the original PR. Please don't. -- This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [airflow] uranusjr commented on a diff in pull request #32272: Fix rendering the mapped parameters when using `expand_kwargs` method

2023-08-09 Thread via GitHub
uranusjr commented on code in PR #32272: URL: https://github.com/apache/airflow/pull/32272#discussion_r1289520687 ## airflow/models/expandinput.py: ## @@ -217,6 +217,10 @@ class ListOfDictsExpandInput(NamedTuple): value: OperatorExpandKwargsArgument +def

[airflow] branch 31851 created (now a02a937d7f)

2023-08-09 Thread uranusjr
This is an automated email from the ASF dual-hosted git repository. uranusjr pushed a change to branch 31851 in repository https://gitbox.apache.org/repos/asf/airflow.git at a02a937d7f Format This branch includes the following new commits: new a02a937d7f Format The 1 revisions

[airflow] 01/01: Format

2023-08-09 Thread uranusjr
This is an automated email from the ASF dual-hosted git repository. uranusjr pushed a commit to branch 31851 in repository https://gitbox.apache.org/repos/asf/airflow.git commit a02a937d7f3dbf858d614ccf1027d65e28e67373 Author: Tzu-ping Chung AuthorDate: Thu Aug 10 11:46:46 2023 +0800

[GitHub] [airflow] uranusjr opened a new pull request, #33279: Allow 'airflow variables export' to print to stdout

2023-08-09 Thread via GitHub
uranusjr opened a new pull request, #33279: URL: https://github.com/apache/airflow/pull/33279 This is a continuation to #31854. Similar to connections, this change allows the CLI to export the variables to stdout using `airflow variables export -`. There are a couple of change

[GitHub] [airflow] Lee-W commented on issue #33257: Status of testing Providers that were prepared on August 09, 2023

2023-08-09 Thread via GitHub
Lee-W commented on issue #33257: URL: https://github.com/apache/airflow/issues/33257#issuecomment-1672472739 Tested with our example dags and everything works as expected. Thanks! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [airflow] saurabh-net commented on a diff in pull request #32256: Add Dataplex Data Quality operators.

2023-08-09 Thread via GitHub
saurabh-net commented on code in PR #32256: URL: https://github.com/apache/airflow/pull/32256#discussion_r1289393490 ## airflow/providers/google/cloud/hooks/dataplex.py: ## @@ -361,3 +390,470 @@ def get_lake( metadata=metadata, ) return result + +

[GitHub] [airflow] saurabh-net commented on a diff in pull request #32256: Add Dataplex Data Quality operators.

2023-08-09 Thread via GitHub
saurabh-net commented on code in PR #32256: URL: https://github.com/apache/airflow/pull/32256#discussion_r1289392944 ## airflow/providers/google/cloud/hooks/dataplex.py: ## @@ -361,3 +390,470 @@ def get_lake( metadata=metadata, ) return result + +

[GitHub] [airflow] github-actions[bot] closed pull request #31854: Allow variables to be printed to STDOUT with airflow variables export.

2023-08-09 Thread via GitHub
github-actions[bot] closed pull request #31854: Allow variables to be printed to STDOUT with airflow variables export. URL: https://github.com/apache/airflow/pull/31854 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use

[GitHub] [airflow] vandonr-amz opened a new pull request, #33277: move Actions, Resources and Permissions views

2023-08-09 Thread via GitHub
vandonr-amz opened a new pull request, #33277: URL: https://github.com/apache/airflow/pull/33277 closes #32197 closes #32198 closes #32199 it's the three views under the separator in the security menu

[GitHub] [airflow] boring-cyborg[bot] commented on issue #33276: Bashoperator/ python operator can get values from `airflow config` command

2023-08-09 Thread via GitHub
boring-cyborg[bot] commented on issue #33276: URL: https://github.com/apache/airflow/issues/33276#issuecomment-1672317850 Thanks for opening your first issue here! Be sure to follow the issue template! If you are willing to raise PR to address this issue please do so, no need to wait for

[GitHub] [airflow] TECH-Airflow-Engineering-Team opened a new issue, #33276: Bashoperator/ python operator can get values from `airflow config` command

2023-08-09 Thread via GitHub
TECH-Airflow-Engineering-Team opened a new issue, #33276: URL: https://github.com/apache/airflow/issues/33276 ### Apache Airflow version Other Airflow 2 version (please specify below) ### What happened As a admin user shouldn't be able to get configuration value through

[GitHub] [airflow] stelsemeyer-m60 commented on pull request #32631: Parametrize poll_interval in KubernetesPodOperator

2023-08-09 Thread via GitHub
stelsemeyer-m60 commented on PR #32631: URL: https://github.com/apache/airflow/pull/32631#issuecomment-1672315005 > > > I see. The PR was closed due to no activity. > > > How do you plan to follow up on it? > > > > > > Would be great to get feedback on my last reply (in that

[GitHub] [airflow] hussein-awala commented on pull request #32272: Fix rendering the mapped parameters in the mapped operator

2023-08-09 Thread via GitHub
hussein-awala commented on PR #32272: URL: https://github.com/apache/airflow/pull/32272#issuecomment-1672298618 @uranusjr, I discovered that the issue is just related to the `expand_kwargs` method. In the `ListOfDictsExpandInput` class, the `resolve` method doesn't filter out string values

[GitHub] [airflow] vandonr-amz commented on issue #32197: AIP-56 - FAB AM - Resources view

2023-08-09 Thread via GitHub
vandonr-amz commented on issue #32197: URL: https://github.com/apache/airflow/issues/32197#issuecomment-1672252032 I'll do this too, but a bit later. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go

[GitHub] [airflow] vandonr-amz commented on issue #32199: AIP-56 - FAB AM - Permissions view

2023-08-09 Thread via GitHub
vandonr-amz commented on issue #32199: URL: https://github.com/apache/airflow/issues/32199#issuecomment-1672251030 I'll do this -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific

[GitHub] [airflow] RNHTTR commented on issue #33272: Add capability of Dataset Schedule backfilling on producer dags clear the state of consumer dags instead of creating new runs.

2023-08-09 Thread via GitHub
RNHTTR commented on issue #33272: URL: https://github.com/apache/airflow/issues/33272#issuecomment-1672215643 I think this makes sense. Be sure to check out the [contributors quick start guide](https://github.com/apache/airflow/blob/main/CONTRIBUTORS_QUICK_START.rst#note-to-starters) and

[airflow] branch constraints-main updated: Updating constraints. Github run id:5814021734

2023-08-09 Thread github-bot
This is an automated email from the ASF dual-hosted git repository. github-bot pushed a commit to branch constraints-main in repository https://gitbox.apache.org/repos/asf/airflow.git The following commit(s) were added to refs/heads/constraints-main by this push: new 00b3322900 Updating

[GitHub] [airflow] vandonr-amz commented on a diff in pull request #33213: Implement `is_authorized()` in auth manager

2023-08-09 Thread via GitHub
vandonr-amz commented on code in PR #33213: URL: https://github.com/apache/airflow/pull/33213#discussion_r1289054657 ## airflow/auth/managers/base_auth_manager.py: ## @@ -54,6 +58,67 @@ def get_user_id(self) -> str: def is_logged_in(self) -> bool: """Return

[GitHub] [airflow] potiuk commented on a diff in pull request #33266: Refactor: Simplify code in serialization

2023-08-09 Thread via GitHub
potiuk commented on code in PR #33266: URL: https://github.com/apache/airflow/pull/33266#discussion_r1289206526 ## airflow/serialization/serde.py: ## @@ -300,14 +300,13 @@ def _stringify(classname: str, version: int, value: T | None) -> str: s =

[GitHub] [airflow] potiuk commented on a diff in pull request #33266: Refactor: Simplify code in serialization

2023-08-09 Thread via GitHub
potiuk commented on code in PR #33266: URL: https://github.com/apache/airflow/pull/33266#discussion_r1289205428 ## airflow/serialization/serde.py: ## @@ -300,14 +300,13 @@ def _stringify(classname: str, version: int, value: T | None) -> str: s =

[airflow] 08/09: Change type of defualts for include_deferred (#33271)

2023-08-09 Thread ephraimanierobi
This is an automated email from the ASF dual-hosted git repository. ephraimanierobi pushed a commit to branch v2-7-test in repository https://gitbox.apache.org/repos/asf/airflow.git commit 2ff8d4d6133300b1c8017ed8f5d9e981921b95c9 Author: Jarek Potiuk AuthorDate: Wed Aug 9 23:00:21 2023 +0200

[airflow] 06/09: Gantt chart: Use earliest/oldest ti dates if different than dag run start/end (#33215)

2023-08-09 Thread ephraimanierobi
This is an automated email from the ASF dual-hosted git repository. ephraimanierobi pushed a commit to branch v2-7-test in repository https://gitbox.apache.org/repos/asf/airflow.git commit 14ee53527043978a42a1747e53edf81f40907e17 Author: Brent Bovenzi AuthorDate: Wed Aug 9 09:37:54 2023 -0400

[airflow] 02/09: Enable pools to consider deferred tasks (#32709)

2023-08-09 Thread ephraimanierobi
This is an automated email from the ASF dual-hosted git repository. ephraimanierobi pushed a commit to branch v2-7-test in repository https://gitbox.apache.org/repos/asf/airflow.git commit 494edad3e9241fbc4e6500a7fb792e04f3896493 Author: Usiel Riedl AuthorDate: Wed Aug 9 20:34:37 2023 +0800

[airflow] 03/09: wOrkaround failing dedlock when running backfill (#32991)

2023-08-09 Thread ephraimanierobi
This is an automated email from the ASF dual-hosted git repository. ephraimanierobi pushed a commit to branch v2-7-test in repository https://gitbox.apache.org/repos/asf/airflow.git commit e9c82d9319c0f27d40f49aca612eb1713e06f4ea Author: Jarek Potiuk AuthorDate: Wed Aug 9 13:00:48 2023 +0200

[airflow] 07/09: Remove add_task from TaskGroup (#33262)

2023-08-09 Thread ephraimanierobi
This is an automated email from the ASF dual-hosted git repository. ephraimanierobi pushed a commit to branch v2-7-test in repository https://gitbox.apache.org/repos/asf/airflow.git commit b0998038936d290e1ca5e237930c884fec1ce9e4 Author: Daniel Standish

[airflow] 01/09: Fix missing related PRs when cherry-picking (#33261)

2023-08-09 Thread ephraimanierobi
This is an automated email from the ASF dual-hosted git repository. ephraimanierobi pushed a commit to branch v2-7-test in repository https://gitbox.apache.org/repos/asf/airflow.git commit a57cc8b0c05c1fa0121cdc0321e7ae75c82d05a5 Author: Jarek Potiuk AuthorDate: Wed Aug 9 17:41:06 2023 +0200

[airflow] 09/09: Update RELEASE_NOTES.rst

2023-08-09 Thread ephraimanierobi
This is an automated email from the ASF dual-hosted git repository. ephraimanierobi pushed a commit to branch v2-7-test in repository https://gitbox.apache.org/repos/asf/airflow.git commit 803d0134ac81427154b9b8521c48a171d5aee0c3 Author: Ephraim Anierobi AuthorDate: Wed Aug 9 07:55:58 2023

[airflow] 05/09: Refactor: Simplify code in dag_processing (#33161)

2023-08-09 Thread ephraimanierobi
This is an automated email from the ASF dual-hosted git repository. ephraimanierobi pushed a commit to branch v2-7-test in repository https://gitbox.apache.org/repos/asf/airflow.git commit 4e00fe8ec01c9a2bbfea02ac7841d42eb45ea7f5 Author: Miroslav Šedivý <6774676+eum...@users.noreply.github.com>

[airflow] 04/09: Fix flaky sqlite tests with `test_xcom_map_nest` hopefully (#33145)

2023-08-09 Thread ephraimanierobi
This is an automated email from the ASF dual-hosted git repository. ephraimanierobi pushed a commit to branch v2-7-test in repository https://gitbox.apache.org/repos/asf/airflow.git commit 4e817c46b4c94d44b5c50665f23f19d7284b42be Author: Jarek Potiuk AuthorDate: Sat Aug 5 23:15:49 2023 +0200

[airflow] branch v2-7-test updated (2c43ca3ede -> 803d0134ac)

2023-08-09 Thread ephraimanierobi
This is an automated email from the ASF dual-hosted git repository. ephraimanierobi pushed a change to branch v2-7-test in repository https://gitbox.apache.org/repos/asf/airflow.git omit 2c43ca3ede Update RELEASE_NOTES.rst new a57cc8b0c0 Fix missing related PRs when cherry-picking

[airflow] branch main updated: Change type of defualts for include_deferred (#33271)

2023-08-09 Thread potiuk
This is an automated email from the ASF dual-hosted git repository. potiuk pushed a commit to branch main in repository https://gitbox.apache.org/repos/asf/airflow.git The following commit(s) were added to refs/heads/main by this push: new d989e9dba5 Change type of defualts for

[GitHub] [airflow] potiuk merged pull request #33271: Change type of defualts for include_deferred

2023-08-09 Thread via GitHub
potiuk merged PR #33271: URL: https://github.com/apache/airflow/pull/33271 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

[GitHub] [airflow] functicons opened a new pull request, #33274: Add parameter sftp_prefetch to SFTPToGCSOperator

2023-08-09 Thread via GitHub
functicons opened a new pull request, #33274: URL: https://github.com/apache/airflow/pull/33274 --- **^ Add meaningful description above** Read the **[Pull Request

[GitHub] [airflow] boring-cyborg[bot] commented on pull request #33274: Add parameter sftp_prefetch to SFTPToGCSOperator

2023-08-09 Thread via GitHub
boring-cyborg[bot] commented on PR #33274: URL: https://github.com/apache/airflow/pull/33274#issuecomment-1672126709 Congratulations on your first Pull Request and welcome to the Apache Airflow community! If you have any issues or are unsure about any anything please check our Contribution

[GitHub] [airflow] ephraimbuddy closed pull request #33103: Use task_id as backup if no task.label

2023-08-09 Thread via GitHub
ephraimbuddy closed pull request #33103: Use task_id as backup if no task.label URL: https://github.com/apache/airflow/pull/33103 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific

[GitHub] [airflow] dstandish commented on pull request #31987: Load just the first yielded event in the triggerer

2023-08-09 Thread via GitHub
dstandish commented on PR #31987: URL: https://github.com/apache/airflow/pull/31987#issuecomment-1672117025 > @dstandish We also need to think about triggers in the different providers and how to make them b/c; should we add an ugly Airflow version check in each trigger to know if we

[GitHub] [airflow] jens-scheffler-bosch commented on a diff in pull request #33017: Allow PythonVenvOperator using other index url

2023-08-09 Thread via GitHub
jens-scheffler-bosch commented on code in PR #33017: URL: https://github.com/apache/airflow/pull/33017#discussion_r1289166442 ## airflow/utils/python_virtualenv.py: ## @@ -76,11 +88,18 @@ def prepare_virtualenv( See virtualenv documentation for more information.

[airflow] branch main updated: Clean CODEOWNERS paths and add hussein-awala as owner to some files (#33269)

2023-08-09 Thread husseinawala
This is an automated email from the ASF dual-hosted git repository. husseinawala pushed a commit to branch main in repository https://gitbox.apache.org/repos/asf/airflow.git The following commit(s) were added to refs/heads/main by this push: new e11431258c Clean CODEOWNERS paths and add

[GitHub] [airflow] hussein-awala merged pull request #33269: Clean CODEOWNERS paths and add hussein-awala as owner to some files

2023-08-09 Thread via GitHub
hussein-awala merged PR #33269: URL: https://github.com/apache/airflow/pull/33269 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

[GitHub] [airflow] ephraimbuddy closed pull request #33263: Add mssql_drop_default when dropping include_deferred table

2023-08-09 Thread via GitHub
ephraimbuddy closed pull request #33263: Add mssql_drop_default when dropping include_deferred table URL: https://github.com/apache/airflow/pull/33263 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go

[GitHub] [airflow] eumiro opened a new pull request, #33273: Refactor: Simplify code in breeze

2023-08-09 Thread via GitHub
eumiro opened a new pull request, #33273: URL: https://github.com/apache/airflow/pull/33273 (no comment) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe,

[GitHub] [airflow] lfpll opened a new issue, #33272: Dataset Schedule backfilling on producer dags will backfill consumer dags.

2023-08-09 Thread via GitHub
lfpll opened a new issue, #33272: URL: https://github.com/apache/airflow/issues/33272 ### Description Right now if we back fill a dag that is a producer, the consumer dags will trigger a new dag with current date not related to the backfilled one. ### Use case/motivation

[GitHub] [airflow] boring-cyborg[bot] commented on issue #33272: Dataset Schedule backfilling on producer dags will backfill consumer dags.

2023-08-09 Thread via GitHub
boring-cyborg[bot] commented on issue #33272: URL: https://github.com/apache/airflow/issues/33272#issuecomment-1672086296 Thanks for opening your first issue here! Be sure to follow the issue template! If you are willing to raise PR to address this issue please do so, no need to wait for

[GitHub] [airflow] hussein-awala commented on pull request #31987: Load just the first yielded event in the triggerer

2023-08-09 Thread via GitHub
hussein-awala commented on PR #31987: URL: https://github.com/apache/airflow/pull/31987#issuecomment-1672078077 > But I guess it remains for us to decide whether to deprecate iterator, and if given iterator, whether to consume up to the return, or just stop at first event, or consume all

[GitHub] [airflow] potiuk opened a new pull request, #33271: Change type of defualts for include_deferred

2023-08-09 Thread via GitHub
potiuk opened a new pull request, #33271: URL: https://github.com/apache/airflow/pull/33271 We should change the defaults to client defaults for MSSQL compatibility The change in #32709 was not liked by MSSQL --- **^ Add meaningful description above**

[GitHub] [airflow] eumiro opened a new pull request, #33270: Refactor: Simplify code in www

2023-08-09 Thread via GitHub
eumiro opened a new pull request, #33270: URL: https://github.com/apache/airflow/pull/33270 (no comment) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe,

[GitHub] [airflow] hussein-awala opened a new pull request, #33269: Clean CODEOWNERS paths and add hussein-awala as owner to some files

2023-08-09 Thread via GitHub
hussein-awala opened a new pull request, #33269: URL: https://github.com/apache/airflow/pull/33269 --- **^ Add meaningful description above** Read the **[Pull Request

[GitHub] [airflow] eumiro opened a new pull request, #33268: Refactor: Simplify code in utils

2023-08-09 Thread via GitHub
eumiro opened a new pull request, #33268: URL: https://github.com/apache/airflow/pull/33268 (no comment) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe,

[GitHub] [airflow] hussein-awala commented on a diff in pull request #33174: Add `pod_template_dict` field to `KubernetesPodOperator`

2023-08-09 Thread via GitHub
hussein-awala commented on code in PR #33174: URL: https://github.com/apache/airflow/pull/33174#discussion_r1289115208 ## tests/providers/cncf/kubernetes/operators/test_pod.py: ## @@ -920,6 +921,64 @@ def test_pod_template_file_kwargs_override(self, randomize_name,

[airflow] branch main updated: Run MSSQL tests when PR contains DB migrations (#33265)

2023-08-09 Thread potiuk
This is an automated email from the ASF dual-hosted git repository. potiuk pushed a commit to branch main in repository https://gitbox.apache.org/repos/asf/airflow.git The following commit(s) were added to refs/heads/main by this push: new fcf0970bc3 Run MSSQL tests when PR contains DB

[GitHub] [airflow] potiuk merged pull request #33265: Run MSSQL tests when PR contains DB migrations

2023-08-09 Thread via GitHub
potiuk merged PR #33265: URL: https://github.com/apache/airflow/pull/33265 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

[GitHub] [airflow] covidium commented on a diff in pull request #33251: Add new RdsStartExportTaskOperator parameters

2023-08-09 Thread via GitHub
covidium commented on code in PR #33251: URL: https://github.com/apache/airflow/pull/33251#discussion_r1289107832 ## airflow/providers/amazon/aws/operators/rds.py: ## @@ -351,6 +351,8 @@ def __init__( s3_prefix: str = "", export_only: list[str] | None = None,

[airflow] branch main updated: Make the `OnFinishAction` enum inherit from str to support passing it to `KubernetesPodOperatpor` (#33228)

2023-08-09 Thread husseinawala
This is an automated email from the ASF dual-hosted git repository. husseinawala pushed a commit to branch main in repository https://gitbox.apache.org/repos/asf/airflow.git The following commit(s) were added to refs/heads/main by this push: new 1f6257110d Make the `OnFinishAction` enum

[GitHub] [airflow] hussein-awala merged pull request #33228: Make the `OnFinishAction` enum inherit from str to support passing it to `KubernetesPodOperatpor`

2023-08-09 Thread via GitHub
hussein-awala merged PR #33228: URL: https://github.com/apache/airflow/pull/33228 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

[GitHub] [airflow] eumiro opened a new pull request, #33267: Refactor: Simplify code in settings

2023-08-09 Thread via GitHub
eumiro opened a new pull request, #33267: URL: https://github.com/apache/airflow/pull/33267 (no comment) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe,

[GitHub] [airflow] vincbeck commented on a diff in pull request #33231: Improve fetching logs from AWS

2023-08-09 Thread via GitHub
vincbeck commented on code in PR #33231: URL: https://github.com/apache/airflow/pull/33231#discussion_r1289070433 ## airflow/providers/amazon/aws/hooks/logs.py: ## @@ -60,6 +61,7 @@ def get_log_events( log_group: str, log_stream_name: str, start_time:

[GitHub] [airflow] potiuk commented on pull request #33265: Run MSSQL tests when PR contains DB migrations

2023-08-09 Thread via GitHub
potiuk commented on PR #33265: URL: https://github.com/apache/airflow/pull/33265#issuecomment-1671971725 > I think it's time to raise the question if we really want MsSQL as backend. a lot of overhead and very little value. I will raise it in the mailing list soon. +1000 -- This

[GitHub] [airflow] eumiro opened a new pull request, #33266: Refactor: Simplify code in serialization

2023-08-09 Thread via GitHub
eumiro opened a new pull request, #33266: URL: https://github.com/apache/airflow/pull/33266 (no comment) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe,

[airflow] branch main updated: Refactor: Simplify code in dag_processing (#33161)

2023-08-09 Thread potiuk
This is an automated email from the ASF dual-hosted git repository. potiuk pushed a commit to branch main in repository https://gitbox.apache.org/repos/asf/airflow.git The following commit(s) were added to refs/heads/main by this push: new 2f16a46db5 Refactor: Simplify code in

[GitHub] [airflow] potiuk merged pull request #33161: Refactor: Simplify code in dag_processing

2023-08-09 Thread via GitHub
potiuk merged PR #33161: URL: https://github.com/apache/airflow/pull/33161 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

[GitHub] [airflow] dzhigimont commented on issue #32870: BigQueryValueCheckOperator does not accept non-default project_id, which lead to jobs with impersonation_chain to fail.

2023-08-09 Thread via GitHub
dzhigimont commented on issue #32870: URL: https://github.com/apache/airflow/issues/32870#issuecomment-1671951901 I want to work on this, assign me please -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above

[GitHub] [airflow] potiuk opened a new pull request, #33265: Run MSSQL tests when PR contains DB migrations

2023-08-09 Thread via GitHub
potiuk opened a new pull request, #33265: URL: https://github.com/apache/airflow/pull/33265 We skip MSSQL tests usually for stability, but when the PR contains migrations, we should always run them. --- **^ Add meaningful description above** Read the

[GitHub] [airflow] eumiro opened a new pull request, #33264: Refactor: Simplify dict manipulation in metrics

2023-08-09 Thread via GitHub
eumiro opened a new pull request, #33264: URL: https://github.com/apache/airflow/pull/33264 (no comment) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe,

[GitHub] [airflow] ephraimbuddy opened a new pull request, #33263: Add mssql_drop_default when dropping include_deferred table

2023-08-09 Thread via GitHub
ephraimbuddy opened a new pull request, #33263: URL: https://github.com/apache/airflow/pull/33263 Because the column has default, we should handle dropping default in MSSQL since it creates a separate object for default values -- This is an automated message from the Apache Git

[GitHub] [airflow] potiuk commented on a diff in pull request #33130: Refactor: Lazy loops, string operations, better sorting

2023-08-09 Thread via GitHub
potiuk commented on code in PR #33130: URL: https://github.com/apache/airflow/pull/33130#discussion_r1289016377 ## airflow/configuration.py: ## @@ -1336,7 +1325,7 @@ def getsection(self, section: str) -> ConfigOptionsDictType | None:

[GitHub] [airflow] potiuk commented on a diff in pull request #33234: Refactor: Simplify code in smaller providers

2023-08-09 Thread via GitHub
potiuk commented on code in PR #33234: URL: https://github.com/apache/airflow/pull/33234#discussion_r1289000634 ## airflow/providers/daskexecutor/executors/dask_executor.py: ## @@ -135,13 +135,13 @@ def end(self) -> None: assert self.client assert

[GitHub] [airflow] eumiro commented on a diff in pull request #33130: Refactor: Lazy loops, string operations, better sorting

2023-08-09 Thread via GitHub
eumiro commented on code in PR #33130: URL: https://github.com/apache/airflow/pull/33130#discussion_r1288996409 ## airflow/configuration.py: ## @@ -1336,7 +1325,7 @@ def getsection(self, section: str) -> ConfigOptionsDictType | None:

[airflow] branch main updated (eb07618d29 -> 2d2a1d699b)

2023-08-09 Thread dstandish
This is an automated email from the ASF dual-hosted git repository. dstandish pushed a change to branch main in repository https://gitbox.apache.org/repos/asf/airflow.git from eb07618d29 Fix missing related PRs when cherry-picking (#33261) add 2d2a1d699b Remove add_task from TaskGroup

[GitHub] [airflow] dstandish merged pull request #33262: Remove add_task from TaskGroup

2023-08-09 Thread via GitHub
dstandish merged PR #33262: URL: https://github.com/apache/airflow/pull/33262 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

[GitHub] [airflow] dstandish commented on pull request #33262: Remove add_task from TaskGroup

2023-08-09 Thread via GitHub
dstandish commented on PR #33262: URL: https://github.com/apache/airflow/pull/33262#issuecomment-1671904206 test failure looks unrelated, will merge -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go

[GitHub] [airflow] vandonr-amz commented on issue #21867: Retry for TaskGroup

2023-08-09 Thread via GitHub
vandonr-amz commented on issue #21867: URL: https://github.com/apache/airflow/issues/21867#issuecomment-1671902556 hey @weirdjh did you make any progress on this ? If not, do you mind if we unassign it from you ? -- This is an automated message from the Apache Git Service. To respond to

[GitHub] [airflow] ferruzzi commented on pull request #33222: Refactor: Simplify code in providers/amazon

2023-08-09 Thread via GitHub
ferruzzi commented on PR #33222: URL: https://github.com/apache/airflow/pull/33222#issuecomment-1671897274 I got half way through reviewing it last night and was going to finish this afternoon.Ah well. :P Thanks for the cleanup! -- This is an automated message from the Apache Git

[GitHub] [airflow] potiuk commented on pull request #32221: Add `DatabricksJobsCreateOperator`

2023-08-09 Thread via GitHub
potiuk commented on PR #32221: URL: https://github.com/apache/airflow/pull/32221#issuecomment-1671895020 See if those tests succed - I rebased your PR now and ping again please. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub

[GitHub] [airflow] potiuk commented on pull request #32221: Add `DatabricksJobsCreateOperator`

2023-08-09 Thread via GitHub
potiuk commented on PR #32221: URL: https://github.com/apache/airflow/pull/32221#issuecomment-1671893897 > hey @potiuk is anything holding this up from being merged? Is it just the type hint comment? Mostly just lack of ping after you fixed the tests to run the workflows I think.

[GitHub] [airflow] eumiro commented on a diff in pull request #33234: Refactor: Simplify code in smaller providers

2023-08-09 Thread via GitHub
eumiro commented on code in PR #33234: URL: https://github.com/apache/airflow/pull/33234#discussion_r1288982279 ## airflow/providers/http/hooks/http.py: ## @@ -340,31 +340,18 @@ async def run( if headers: _headers.update(headers) -if

[GitHub] [airflow] eumiro commented on a diff in pull request #33234: Refactor: Simplify code in smaller providers

2023-08-09 Thread via GitHub
eumiro commented on code in PR #33234: URL: https://github.com/apache/airflow/pull/33234#discussion_r1288976150 ## airflow/providers/http/hooks/http.py: ## @@ -340,31 +340,18 @@ async def run( if headers: _headers.update(headers) -if

[GitHub] [airflow] eumiro commented on a diff in pull request #33229: Refactor: Simplify code in providers/google

2023-08-09 Thread via GitHub
eumiro commented on code in PR #33229: URL: https://github.com/apache/airflow/pull/33229#discussion_r1288967639 ## airflow/providers/google/cloud/transfers/facebook_ads_to_gcs.py: ## @@ -226,7 +226,5 @@ def _flush_rows(self, converted_rows: list[Any] | None, object_name: str):

[GitHub] [airflow] amoghrajesh commented on pull request #23996: Removing magic numbers from exceptions

2023-08-09 Thread via GitHub
amoghrajesh commented on PR #23996: URL: https://github.com/apache/airflow/pull/23996#issuecomment-1671874955 @bernardocouto I am taking this task up and will lead it to completion here. -- This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [airflow] eumiro commented on a diff in pull request #33229: Refactor: Simplify code in providers/google

2023-08-09 Thread via GitHub
eumiro commented on code in PR #33229: URL: https://github.com/apache/airflow/pull/33229#discussion_r1288965368 ## airflow/providers/google/cloud/hooks/gcs.py: ## @@ -328,11 +328,16 @@ def download( # TODO: future improvement check file size before downloading,

[GitHub] [airflow] vincbeck commented on a diff in pull request #33231: Improve fetching logs from AWS

2023-08-09 Thread via GitHub
vincbeck commented on code in PR #33231: URL: https://github.com/apache/airflow/pull/33231#discussion_r1288965245 ## airflow/providers/amazon/aws/log/cloudwatch_task_handler.py: ## @@ -103,17 +106,29 @@ def _read(self, task_instance, try_number, metadata=None): log

[GitHub] [airflow] vincbeck commented on a diff in pull request #33231: Improve fetching logs from AWS

2023-08-09 Thread via GitHub
vincbeck commented on code in PR #33231: URL: https://github.com/apache/airflow/pull/33231#discussion_r1288954740 ## airflow/providers/amazon/aws/hooks/logs.py: ## @@ -60,6 +61,7 @@ def get_log_events( log_group: str, log_stream_name: str, start_time:

[GitHub] [airflow] vandonr-amz commented on a diff in pull request #33231: Improve fetching logs from AWS

2023-08-09 Thread via GitHub
vandonr-amz commented on code in PR #33231: URL: https://github.com/apache/airflow/pull/33231#discussion_r1288948715 ## airflow/providers/amazon/aws/log/cloudwatch_task_handler.py: ## @@ -103,17 +106,29 @@ def _read(self, task_instance, try_number, metadata=None):

[GitHub] [airflow] eladkal commented on a diff in pull request #33231: Improve fetching logs from AWS

2023-08-09 Thread via GitHub
eladkal commented on code in PR #33231: URL: https://github.com/apache/airflow/pull/33231#discussion_r1288944532 ## airflow/providers/amazon/aws/hooks/logs.py: ## @@ -60,6 +61,7 @@ def get_log_events( log_group: str, log_stream_name: str, start_time:

[GitHub] [airflow] stikkireddy commented on pull request #32221: Add `DatabricksJobsCreateOperator`

2023-08-09 Thread via GitHub
stikkireddy commented on PR #32221: URL: https://github.com/apache/airflow/pull/32221#issuecomment-1671781290 hey @potiuk is anything holding this up from being merged? Is it just the type hint comment? -- This is an automated message from the Apache Git Service. To respond to the

[GitHub] [airflow] ebdavison commented on issue #33260: BUG foreign key associated with column 'ab_permission_view_role.permission_view_id'

2023-08-09 Thread via GitHub
ebdavison commented on issue #33260: URL: https://github.com/apache/airflow/issues/33260#issuecomment-1671777030 I did an install using the python contraints as per the docs but still SQLAlchemy was not quite right `sudo -u airflow pip3 install "apache-airflow[celery]==2.3.4"

[GitHub] [airflow] adam133 commented on pull request #32903: allow DatabricksSubmitRunOperator to accept a pipeline name for a pipeline_task

2023-08-09 Thread via GitHub
adam133 commented on PR #32903: URL: https://github.com/apache/airflow/pull/32903#issuecomment-1671770111 > > Needs testing and tests. I just threw this together quickly because I had a bit to look at it. Will also need some documentation updates. > > Hello @adam133, I'm very

[GitHub] [airflow] vandonr-amz commented on a diff in pull request #32437: add deferrable mode to rds start & stop DB

2023-08-09 Thread via GitHub
vandonr-amz commented on code in PR #32437: URL: https://github.com/apache/airflow/pull/32437#discussion_r1288874173 ## airflow/providers/amazon/aws/operators/rds.py: ## @@ -20,15 +20,20 @@ import json import warnings from datetime import timedelta -from typing import

[GitHub] [airflow] Owen-CH-Leung commented on pull request #31389: Add istio support to k8s operator

2023-08-09 Thread via GitHub
Owen-CH-Leung commented on PR #31389: URL: https://github.com/apache/airflow/pull/31389#issuecomment-1671701891 There're failing helm tests with different executors...I'll try to fix them in coming few days -- This is an automated message from the Apache Git Service. To respond to the

[GitHub] [airflow] vincbeck commented on a diff in pull request #33251: Add new RdsStartExportTaskOperator parameters

2023-08-09 Thread via GitHub
vincbeck commented on code in PR #33251: URL: https://github.com/apache/airflow/pull/33251#discussion_r1288779402 ## airflow/providers/amazon/aws/operators/rds.py: ## @@ -351,6 +351,8 @@ def __init__( s3_prefix: str = "", export_only: list[str] | None = None,

[airflow] branch main updated: Fix missing related PRs when cherry-picking (#33261)

2023-08-09 Thread ephraimanierobi
This is an automated email from the ASF dual-hosted git repository. ephraimanierobi pushed a commit to branch main in repository https://gitbox.apache.org/repos/asf/airflow.git The following commit(s) were added to refs/heads/main by this push: new eb07618d29 Fix missing related PRs when

[GitHub] [airflow] ephraimbuddy merged pull request #33261: Fix missing related PRs when cherry-picking

2023-08-09 Thread via GitHub
ephraimbuddy merged PR #33261: URL: https://github.com/apache/airflow/pull/33261 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

[GitHub] [airflow] vincbeck commented on pull request #33231: Improve fetching logs form AWS

2023-08-09 Thread via GitHub
vincbeck commented on PR #33231: URL: https://github.com/apache/airflow/pull/33231#issuecomment-1671645608 > Changes look good! But maybe some unit tests for the changes to `get_cloudwatch_logs` and maybe `get_log_events`? Good call! I added some unit tests for `get_cloudwatch_logs`.

[GitHub] [airflow] potiuk opened a new pull request, #33261: Fix missing related PRs when cherry-picking

2023-08-09 Thread via GitHub
potiuk opened a new pull request, #33261: URL: https://github.com/apache/airflow/pull/33261 The `airflow-github` tooling had a check if PR has been already merged, but the PR did not work well when there was already a PR cherry-picked that referred the PR in question in their log message.

[GitHub] [airflow] ebdavison opened a new issue, #33260: BUG foreign key associated with column 'ab_permission_view_role.permission_view_id'

2023-08-09 Thread via GitHub
ebdavison opened a new issue, #33260: URL: https://github.com/apache/airflow/issues/33260 ### Apache Airflow version Other Airflow 2 version (please specify below) ### What happened Receiving an error starting webserver after upgrade to 2.4.0: ```

[GitHub] [airflow] boring-cyborg[bot] commented on issue #33260: BUG foreign key associated with column 'ab_permission_view_role.permission_view_id'

2023-08-09 Thread via GitHub
boring-cyborg[bot] commented on issue #33260: URL: https://github.com/apache/airflow/issues/33260#issuecomment-1671642129 Thanks for opening your first issue here! Be sure to follow the issue template! If you are willing to raise PR to address this issue please do so, no need to wait for

  1   2   3   >