Re: [I] White INFO log is hard to read [airflow]

2024-04-08 Thread via GitHub
tirkarthi commented on issue #38844: URL: https://github.com/apache/airflow/issues/38844#issuecomment-2044202208 Does the logs returned from API have ansi code that causes this? cc: @jscheffl -- This is an automated message from the Apache Git Service. To respond to the message,

Re: [I] clear a deferred task do not increment the tries [airflow]

2024-04-08 Thread via GitHub
tirkarthi commented on issue #38735: URL: https://github.com/apache/airflow/issues/38735#issuecomment-2044200109 When a task is deferred the `_try_number` is decremented so that when the trigger yields an event the task resumes execution where `_try_number` is incremented to use the same

Re: [I] timeout on Dynamic Task mapping : skipped inner task , task status is still success [airflow]

2024-04-08 Thread via GitHub
raphaelauv commented on issue #37332: URL: https://github.com/apache/airflow/issues/37332#issuecomment-2044184401 @ephraimbuddy could you please remo6the tag wait for response , thanks -- This is an automated message from the Apache Git Service. To respond to the message, please log on

Re: [PR] Make _get_ti compatible with RPC [airflow]

2024-04-08 Thread via GitHub
dstandish commented on code in PR #38570: URL: https://github.com/apache/airflow/pull/38570#discussion_r1556927388 ## airflow/serialization/pydantic/dag_run.py: ## @@ -78,7 +78,7 @@ def get_task_instances( def get_task_instance( self, task_id: str, -

Re: [PR] Make _get_ti compatible with RPC [airflow]

2024-04-08 Thread via GitHub
dstandish commented on code in PR #38570: URL: https://github.com/apache/airflow/pull/38570#discussion_r1556922160 ## airflow/cli/commands/task_command.py: ## @@ -156,8 +156,10 @@ def _get_dag_run( raise ValueError(f"unknown create_if_necessary value:

Re: [PR] Make _get_ti compatible with RPC [airflow]

2024-04-08 Thread via GitHub
dstandish commented on code in PR #38570: URL: https://github.com/apache/airflow/pull/38570#discussion_r1556918483 ## airflow/serialization/serialized_objects.py: ## @@ -1462,9 +1462,15 @@ def deserialize_dag(cls, encoded_dag: dict[str, Any]) -> SerializedDAG:

Re: [I] Unable to find Spark Support in helm chart [airflow]

2024-04-08 Thread via GitHub
boring-cyborg[bot] commented on issue #38851: URL: https://github.com/apache/airflow/issues/38851#issuecomment-2044123456 Thanks for opening your first issue here! Be sure to follow the issue template! If you are willing to raise PR to address this issue please do so, no need to wait for

[I] Unable to find Spark Support in helm chart [airflow]

2024-04-08 Thread via GitHub
anandlingaraj opened a new issue, #38851: URL: https://github.com/apache/airflow/issues/38851 ### Apache Airflow version 2.9.0 ### If "Other Airflow 2 version" selected, which one? apache/airflow:2.8.3 ### What happened? When selecting connection type for

Re: [PR] fix: try002 for provider cncf kubernetes [airflow]

2024-04-08 Thread via GitHub
eladkal commented on PR #38799: URL: https://github.com/apache/airflow/pull/38799#issuecomment-2044119678 There are conflics to resolve -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the

Re: [PR] Add four unit tests for aws/utils [airflow]

2024-04-08 Thread via GitHub
slycyberguy commented on code in PR #38820: URL: https://github.com/apache/airflow/pull/38820#discussion_r1556846390 ## tests/providers/amazon/aws/utils/test_sagemaker.py: ## Review Comment: I removed those tests. I figured they were necessary because those services were

Re: [PR] Paginate Airflow task logs [airflow]

2024-04-08 Thread via GitHub
uranusjr commented on code in PR #38807: URL: https://github.com/apache/airflow/pull/38807#discussion_r1556841592 ## airflow/providers/amazon/aws/log/s3_task_handler.py: ## @@ -178,7 +180,13 @@ def s3_read(self, remote_log_location: str, return_error: bool = False) -> str:

Re: [PR] Add the deferrable mode to the Dataflow sensors [airflow]

2024-04-08 Thread via GitHub
Lee-W commented on code in PR #37693: URL: https://github.com/apache/airflow/pull/37693#discussion_r1556794107 ## airflow/providers/google/cloud/sensors/dataflow.py: ## @@ -194,27 +244,72 @@ def poke(self, context: Context) -> bool: project_id=self.project_id,

Re: [PR] Always use the executemany method when inserting rows in DbApiHook as it's way much faster [airflow]

2024-04-08 Thread via GitHub
uranusjr commented on code in PR #38715: URL: https://github.com/apache/airflow/pull/38715#discussion_r1556827871 ## airflow/providers/common/sql/hooks/sql.py: ## @@ -166,6 +175,7 @@ def __init__(self, *args, schema: str | None = None, log_sql: bool = True, **kwa #

Re: [PR] Add the deferrable mode to the Dataflow sensors [airflow]

2024-04-08 Thread via GitHub
Lee-W commented on code in PR #37693: URL: https://github.com/apache/airflow/pull/37693#discussion_r1556763479 ## airflow/providers/google/cloud/sensors/dataflow.py: ## @@ -151,12 +205,14 @@ def __init__( self, *, job_id: str, -callback:

Re: [PR] Deferrable mode for Custom Training Job operators [airflow]

2024-04-08 Thread via GitHub
Lee-W commented on code in PR #38584: URL: https://github.com/apache/airflow/pull/38584#discussion_r1556745491 ## airflow/providers/google/cloud/operators/vertex_ai/custom_job.py: ## @@ -539,6 +564,94 @@ def on_kill(self) -> None: if self.hook:

Re: [PR] Fix AttributeError: 'DagRunNote' object has no attribute 'dag_id' [airflow]

2024-04-08 Thread via GitHub
MyLong commented on code in PR #38843: URL: https://github.com/apache/airflow/pull/38843#discussion_r1556746373 ## airflow/models/dagrun.py: ## @@ -1648,7 +1648,5 @@ def __init__(self, content, user_id=None): self.user_id = user_id def __repr__(self): -

Re: [I] timeout on Dynamic Task mapping : skipped inner task , task status is still success [airflow]

2024-04-08 Thread via GitHub
github-actions[bot] commented on issue #37332: URL: https://github.com/apache/airflow/issues/37332#issuecomment-2043917725 This issue has been automatically marked as stale because it has been open for 14 days with no response from the author. It will be closed in next 7 days if no further

Re: [PR] Make ImapHook mail body encoding configurable [airflow]

2024-04-08 Thread via GitHub
github-actions[bot] commented on PR #37517: URL: https://github.com/apache/airflow/pull/37517#issuecomment-2043917704 This pull request has been automatically marked as stale because it has not had recent activity. It will be closed in 5 days if no further activity occurs. Thank you for

Re: [I] GlueJobOperator failing with Invalid type for parameter GlueVersion for provider version >= 7.1.0. continution of #32404 [airflow]

2024-04-08 Thread via GitHub
don1uppa commented on issue #38848: URL: https://github.com/apache/airflow/issues/38848#issuecomment-2043872862 Thank you the literal option worked. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go

Re: [PR] Fix AttributeError: 'DagRunNote' object has no attribute 'dag_id' [airflow]

2024-04-08 Thread via GitHub
hussein-awala commented on code in PR #38843: URL: https://github.com/apache/airflow/pull/38843#discussion_r1556607661 ## airflow/models/dagrun.py: ## @@ -1648,7 +1648,5 @@ def __init__(self, content, user_id=None): self.user_id = user_id def __repr__(self): -

Re: [PR] Amazon Bedrock - Model Throughput Provisioning [airflow]

2024-04-08 Thread via GitHub
ferruzzi commented on code in PR #38850: URL: https://github.com/apache/airflow/pull/38850#discussion_r1556503642 ## airflow/providers/amazon/aws/operators/bedrock.py: ## @@ -25,7 +25,10 @@ from airflow.exceptions import AirflowException Review Comment: This comment is

Re: [PR] Amazon Bedrock - Model Throughput Provisioning [airflow]

2024-04-08 Thread via GitHub
ferruzzi commented on code in PR #38850: URL: https://github.com/apache/airflow/pull/38850#discussion_r1556503642 ## airflow/providers/amazon/aws/operators/bedrock.py: ## @@ -25,7 +25,10 @@ from airflow.exceptions import AirflowException Review Comment: This comment is

[PR] Amazon Bedrock - Model Throughput Provisioning [airflow]

2024-04-08 Thread via GitHub
ferruzzi opened a new pull request, #38850: URL: https://github.com/apache/airflow/pull/38850 Adds support for provisioning model throughput on Amazon Bedrock including: operators, sensors, waiters, and triggerer, as well as unit tests and system tests for all of the above, and doc

[PR] Amazon Bedrock - Clean up hook unit tests [airflow]

2024-04-08 Thread via GitHub
ferruzzi opened a new pull request, #38849: URL: https://github.com/apache/airflow/pull/38849 @vincbeck and @Taragolis requested some changes to the Bedrock hooks in https://github.com/apache/airflow/pull/38693 and I missed the opportunity to greatly simplify the related unit tests.

Re: [I] Default credentials cause warnings on Google Chrome after logging in to Airflow's UI [airflow]

2024-04-08 Thread via GitHub
Taragolis commented on issue #38797: URL: https://github.com/apache/airflow/issues/38797#issuecomment-2043738184 > but unfortunately, Chrome is not an open source, so I'm not sure what the address is in this case. AFAIK, Chrome bugs could be reported/tracked on Chromium bug tracker:

(airflow-site) branch gh-pages updated (613224189f -> 21f20bf658)

2024-04-08 Thread github-bot
This is an automated email from the ASF dual-hosted git repository. github-bot pushed a change to branch gh-pages in repository https://gitbox.apache.org/repos/asf/airflow-site.git discard 613224189f Rewritten history to remove past gh-pages deployments new 21f20bf658 Rewritten history to

(airflow) branch constraints-main updated: Updating constraints. Github run id:8606550300

2024-04-08 Thread github-bot
This is an automated email from the ASF dual-hosted git repository. github-bot pushed a commit to branch constraints-main in repository https://gitbox.apache.org/repos/asf/airflow.git The following commit(s) were added to refs/heads/constraints-main by this push: new 2c0a5b33b8 Updating

(airflow-site) branch main updated: Fix typo in 2.9 blog post (#1000)

2024-04-08 Thread jedcunningham
This is an automated email from the ASF dual-hosted git repository. jedcunningham pushed a commit to branch main in repository https://gitbox.apache.org/repos/asf/airflow-site.git The following commit(s) were added to refs/heads/main by this push: new 265ea2fea6 Fix typo in 2.9 blog post

Re: [PR] Fix typo in 2.9 blog post [airflow-site]

2024-04-08 Thread via GitHub
jedcunningham merged PR #1000: URL: https://github.com/apache/airflow-site/pull/1000 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

(airflow-site) branch 2.9.0-blog-post-typo deleted (was d2db4716ad)

2024-04-08 Thread jedcunningham
This is an automated email from the ASF dual-hosted git repository. jedcunningham pushed a change to branch 2.9.0-blog-post-typo in repository https://gitbox.apache.org/repos/asf/airflow-site.git was d2db4716ad Fix typo in 2.9 blog post The revisions that were on this branch are still

Re: [PR] Make _run_task_by_local_task_job compatible with internal API [airflow]

2024-04-08 Thread via GitHub
dstandish commented on PR #38563: URL: https://github.com/apache/airflow/pull/38563#issuecomment-2043674071 ok renaming this one to `Add session that blows up when using internal API` because, with the addition of this, there's no longer a need to make the other changes I think. --

Re: [PR] Introduce new config variable to control whether DAG processor outputs to stdout [airflow]

2024-04-08 Thread via GitHub
dimon222 commented on PR #37439: URL: https://github.com/apache/airflow/pull/37439#issuecomment-2043662173 How to select output ONLY to stdout? If I write to both locations I risk getting OOM in container environment and I don't want to persist it perpetually in PVC. -- This is an

(airflow-site) branch gh-pages updated (12ec5916dd -> 613224189f)

2024-04-08 Thread github-bot
This is an automated email from the ASF dual-hosted git repository. github-bot pushed a change to branch gh-pages in repository https://gitbox.apache.org/repos/asf/airflow-site.git discard 12ec5916dd Rewritten history to remove past gh-pages deployments new 613224189f Rewritten history to

Re: [PR] fix: try002 for provider imap [airflow]

2024-04-08 Thread via GitHub
Taragolis commented on code in PR #38804: URL: https://github.com/apache/airflow/pull/38804#discussion_r1556428173 ## airflow/providers/imap/hooks/imap.py: ## @@ -219,7 +219,7 @@ def _retrieve_mails_attachments_by_name( self, name: str, check_regex: bool, latest_only:

(airflow) branch main updated: fix: try002 for provider common sql (#38800)

2024-04-08 Thread taragolis
This is an automated email from the ASF dual-hosted git repository. taragolis pushed a commit to branch main in repository https://gitbox.apache.org/repos/asf/airflow.git The following commit(s) were added to refs/heads/main by this push: new 94153d70ac fix: try002 for provider common sql

Re: [PR] fix: try002 for provider common sql [airflow]

2024-04-08 Thread via GitHub
Taragolis merged PR #38800: URL: https://github.com/apache/airflow/pull/38800 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

Re: [I] GlueJobOperator failing with Invalid type for parameter GlueVersion for provider version >= 7.1.0. continution of #32404 [airflow]

2024-04-08 Thread via GitHub
Taragolis commented on issue #38848: URL: https://github.com/apache/airflow/issues/38848#issuecomment-2043607725 ``render_template_as_native_obj=True,`` will force to use `jinja2.nativetypes.NativeEnvironment` which are enforced to convert some to python types, see example:

[PR] Fix typo in 2.9 blog post [airflow-site]

2024-04-08 Thread via GitHub
jedcunningham opened a new pull request, #1000: URL: https://github.com/apache/airflow-site/pull/1000 (no comment) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To

(airflow-site) 01/01: Fix typo in 2.9 blog post

2024-04-08 Thread jedcunningham
This is an automated email from the ASF dual-hosted git repository. jedcunningham pushed a commit to branch 2.9.0-blog-post-typo in repository https://gitbox.apache.org/repos/asf/airflow-site.git commit d2db4716ad049c85cd63c460e2e57cddfb612a02 Author: Jed Cunningham AuthorDate: Mon Apr 8

(airflow-site) branch 2.9.0-blog-post-typo created (now d2db4716ad)

2024-04-08 Thread jedcunningham
This is an automated email from the ASF dual-hosted git repository. jedcunningham pushed a change to branch 2.9.0-blog-post-typo in repository https://gitbox.apache.org/repos/asf/airflow-site.git at d2db4716ad Fix typo in 2.9 blog post This branch includes the following new commits:

(airflow) branch main updated: Amazon Bedrock - Model Customization Jobs (#38693)

2024-04-08 Thread ferruzzi
This is an automated email from the ASF dual-hosted git repository. ferruzzi pushed a commit to branch main in repository https://gitbox.apache.org/repos/asf/airflow.git The following commit(s) were added to refs/heads/main by this push: new 7ed31d5fdf Amazon Bedrock - Model Customization

(airflow-site) branch add-weilee-to-committer-list deleted (was 43cd4e89d8)

2024-04-08 Thread jedcunningham
This is an automated email from the ASF dual-hosted git repository. jedcunningham pushed a change to branch add-weilee-to-committer-list in repository https://gitbox.apache.org/repos/asf/airflow-site.git was 43cd4e89d8 add Wei Lee to committer list The revisions that were on this branch

(airflow-site) branch main updated: add Wei Lee to committer list (#996)

2024-04-08 Thread jedcunningham
This is an automated email from the ASF dual-hosted git repository. jedcunningham pushed a commit to branch main in repository https://gitbox.apache.org/repos/asf/airflow-site.git The following commit(s) were added to refs/heads/main by this push: new e1c24fbde7 add Wei Lee to committer

Re: [PR] add Wei Lee to committer list [airflow-site]

2024-04-08 Thread via GitHub
jedcunningham merged PR #996: URL: https://github.com/apache/airflow-site/pull/996 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

Re: [PR] Amazon Bedrock - Model Customization Jobs [airflow]

2024-04-08 Thread via GitHub
ferruzzi merged PR #38693: URL: https://github.com/apache/airflow/pull/38693 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

Re: [PR] Fix eks.py SyntaxWarning: invalid esape sequence '\s' [airflow]

2024-04-08 Thread via GitHub
vincbeck commented on PR #38734: URL: https://github.com/apache/airflow/pull/38734#issuecomment-2043541309 Hi @dabla, it seems this PR introduced a bug. All our system tests related to AWS EKS started to fail for the same reason. See error below: ``` ERROR [root] exec: failed to

Re: [I] Automate links to configuration of providers [airflow]

2024-04-08 Thread via GitHub
poorvirohidekar commented on issue #34306: URL: https://github.com/apache/airflow/issues/34306#issuecomment-2043511356 @potiuk - From the file mentioned, I understand that class AuthConfigurations is rendering a template configurations.rst.jinja2 and when I look at the

[I] GlueJobOperator failing with Invalid type for parameter GlueVersion for provider version >= 7.1.0. continution of #32404 [airflow]

2024-04-08 Thread via GitHub
don1uppa opened a new issue, #38848: URL: https://github.com/apache/airflow/issues/38848 ### Apache Airflow version Other Airflow 2 version (please specify below) ### If "Other Airflow 2 version" selected, which one? Version: v2.8.1 current mwaa release ### What

(airflow) branch main updated: Airflow 2.9.0 has been released (#38837)

2024-04-08 Thread ephraimanierobi
This is an automated email from the ASF dual-hosted git repository. ephraimanierobi pushed a commit to branch main in repository https://gitbox.apache.org/repos/asf/airflow.git The following commit(s) were added to refs/heads/main by this push: new 14a2b2cf98 Airflow 2.9.0 has been

Re: [PR] Airflow 2.9.0 has been released [airflow]

2024-04-08 Thread via GitHub
ephraimbuddy merged PR #38837: URL: https://github.com/apache/airflow/pull/38837 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

(airflow) branch main updated: Chart: Default Airflow version to 2.9.0 (#38839)

2024-04-08 Thread ephraimanierobi
This is an automated email from the ASF dual-hosted git repository. ephraimanierobi pushed a commit to branch main in repository https://gitbox.apache.org/repos/asf/airflow.git The following commit(s) were added to refs/heads/main by this push: new 1d70bbf683 Chart: Default Airflow version

Re: [PR] Chart: Default Airflow version to 2.9.0 [airflow]

2024-04-08 Thread via GitHub
ephraimbuddy merged PR #38839: URL: https://github.com/apache/airflow/pull/38839 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

Re: [I] Change the UI color for ExternalSensor tasks [airflow]

2024-04-08 Thread via GitHub
idantepper commented on issue #38845: URL: https://github.com/apache/airflow/issues/38845#issuecomment-2043379278 Hi, I would like to take this issue -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go

Re: [PR] Amazon Bedrock - Model Customization Jobs [airflow]

2024-04-08 Thread via GitHub
ferruzzi commented on code in PR #38693: URL: https://github.com/apache/airflow/pull/38693#discussion_r1556213468 ## airflow/providers/amazon/aws/operators/bedrock.py: ## @@ -91,3 +98,155 @@ def execute(self, context: Context) -> dict[str, str | int]:

Re: [PR] Amazon Bedrock - Model Customization Jobs [airflow]

2024-04-08 Thread via GitHub
ferruzzi commented on code in PR #38693: URL: https://github.com/apache/airflow/pull/38693#discussion_r1556206876 ## airflow/providers/amazon/aws/operators/bedrock.py: ## @@ -91,3 +98,155 @@ def execute(self, context: Context) -> dict[str, str | int]:

Re: [PR] Amazon Bedrock - Model Customization Jobs [airflow]

2024-04-08 Thread via GitHub
ferruzzi commented on code in PR #38693: URL: https://github.com/apache/airflow/pull/38693#discussion_r1556206876 ## airflow/providers/amazon/aws/operators/bedrock.py: ## @@ -91,3 +98,155 @@ def execute(self, context: Context) -> dict[str, str | int]:

Re: [PR] Amazon Bedrock - Model Customization Jobs [airflow]

2024-04-08 Thread via GitHub
ferruzzi commented on code in PR #38693: URL: https://github.com/apache/airflow/pull/38693#discussion_r1556199352 ## airflow/providers/amazon/aws/sensors/bedrock.py: ## @@ -0,0 +1,111 @@ +# +# Licensed to the Apache Software Foundation (ASF) under one +# or more contributor

Re: [PR] Amazon Bedrock - Model Customization Jobs [airflow]

2024-04-08 Thread via GitHub
ferruzzi commented on code in PR #38693: URL: https://github.com/apache/airflow/pull/38693#discussion_r1556195811 ## airflow/providers/amazon/aws/hooks/bedrock.py: ## @@ -19,6 +19,37 @@ from airflow.providers.amazon.aws.hooks.base_aws import AwsBaseHook +class

Re: [PR] Make _run_task_by_local_task_job compatible with internal API [airflow]

2024-04-08 Thread via GitHub
dstandish commented on code in PR #38563: URL: https://github.com/apache/airflow/pull/38563#discussion_r1556179515 ## airflow/cli/commands/task_command.py: ## @@ -278,7 +279,15 @@ def _run_task_by_local_task_job(args, ti: TaskInstance | TaskInstancePydantic) -

Re: [PR] Setting use_beeline by default for hive cli connection [airflow]

2024-04-08 Thread via GitHub
eladkal commented on code in PR #38763: URL: https://github.com/apache/airflow/pull/38763#discussion_r1556176719 ## airflow/providers/apache/hive/CHANGELOG.rst: ## @@ -27,6 +27,19 @@ Changelog - +8.0.0 +. + + +Breaking changes + + +Changed the

(airflow) branch main updated: Add lineage_job_namespace and lineage_job_name OpenLineage macros (#38829)

2024-04-08 Thread mobuchowski
This is an automated email from the ASF dual-hosted git repository. mobuchowski pushed a commit to branch main in repository https://gitbox.apache.org/repos/asf/airflow.git The following commit(s) were added to refs/heads/main by this push: new 093ab7e755 Add lineage_job_namespace and

Re: [PR] Add lineage_job_namespace and lineage_job_name OpenLineage macros [airflow]

2024-04-08 Thread via GitHub
mobuchowski merged PR #38829: URL: https://github.com/apache/airflow/pull/38829 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

Re: [PR] Setting use_beeline by default for hive cli connection [airflow]

2024-04-08 Thread via GitHub
potiuk commented on PR #38763: URL: https://github.com/apache/airflow/pull/38763#issuecomment-2043244944 Approved with NIT's -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment.

Re: [PR] Setting use_beeline by default for hive cli connection [airflow]

2024-04-08 Thread via GitHub
potiuk commented on code in PR #38763: URL: https://github.com/apache/airflow/pull/38763#discussion_r1556155471 ## airflow/providers/apache/hive/provider.yaml: ## @@ -25,6 +25,7 @@ state: ready source-date-epoch: 1709554960 # note that those versions are maintained by release

Re: [PR] Setting use_beeline by default for hive cli connection [airflow]

2024-04-08 Thread via GitHub
potiuk commented on code in PR #38763: URL: https://github.com/apache/airflow/pull/38763#discussion_r1556154848 ## airflow/providers/apache/hive/provider.yaml: ## @@ -25,6 +25,7 @@ state: ready source-date-epoch: 1709554960 # note that those versions are maintained by release

(airflow) branch main updated: Improve editable airflow installation by adding preinstalled deps (#38764)

2024-04-08 Thread potiuk
This is an automated email from the ASF dual-hosted git repository. potiuk pushed a commit to branch main in repository https://gitbox.apache.org/repos/asf/airflow.git The following commit(s) were added to refs/heads/main by this push: new d409b8be96 Improve editable airflow installation

Re: [PR] Improve editable airflow installation by adding preinstalled deps [airflow]

2024-04-08 Thread via GitHub
potiuk merged PR #38764: URL: https://github.com/apache/airflow/pull/38764 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

Re: [PR] Give `on_task_instance_failed` access to the error that caused the failure [airflow]

2024-04-08 Thread via GitHub
potiuk commented on code in PR #38155: URL: https://github.com/apache/airflow/pull/38155#discussion_r1556148549 ## airflow/models/taskinstance.py: ## @@ -1409,7 +1410,9 @@ class TaskInstance(Base, LoggingMixin): cascade="all, delete, delete-orphan", ) note =

Re: [I] Databricks provider does not support Azure managed identity if more than one identity exists (e.g. ADF, AKS) [airflow]

2024-04-08 Thread via GitHub
potiuk commented on issue #38762: URL: https://github.com/apache/airflow/issues/38762#issuecomment-2043197124 > Perhaps one of the project maintainers could kindly suggest it if they have open channels of communication with Microsoft about their commercial offerings? Not as far as I

Re: [PR] Replace dill package to use cloudpickle [airflow]

2024-04-08 Thread via GitHub
potiuk commented on code in PR #38531: URL: https://github.com/apache/airflow/pull/38531#discussion_r1556118149 ## airflow/models/taskinstance.py: ## @@ -1287,7 +1287,7 @@ class TaskInstance(Base, LoggingMixin): queued_dttm = Column(UtcDateTime) queued_by_job_id =

Re: [I] Databricks provider does not support Azure managed identity if more than one identity exists (e.g. ADF, AKS) [airflow]

2024-04-08 Thread via GitHub
jtv8 commented on issue #38762: URL: https://github.com/apache/airflow/issues/38762#issuecomment-2043178028 Hi @ghjklw! I agree that it would be better to leverage the abstractions in the official Microsoft libraries - the current implementation is far too low level. However, that would

(airflow) branch main updated (285f037dbc -> 76477b0977)

2024-04-08 Thread ephraimanierobi
This is an automated email from the ASF dual-hosted git repository. ephraimanierobi pushed a change to branch main in repository https://gitbox.apache.org/repos/asf/airflow.git from 285f037dbc Remove decorator from rendering fields example (#38827) add 76477b0977 Update version added

Re: [PR] Implement start/end/debug for multiple executors in scheduler job [airflow]

2024-04-08 Thread via GitHub
o-nikolas commented on code in PR #38514: URL: https://github.com/apache/airflow/pull/38514#discussion_r1556103085 ## airflow/jobs/job.py: ## @@ -104,12 +106,13 @@ class Job(Base, LoggingMixin): Only makes sense for SchedulerJob and BackfillJob instances. """ -

Re: [PR] Update version added field in config.yml [airflow]

2024-04-08 Thread via GitHub
ephraimbuddy merged PR #38840: URL: https://github.com/apache/airflow/pull/38840 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

Re: [PR] Chart: Default Airflow version to 2.9.0 [airflow]

2024-04-08 Thread via GitHub
ephraimbuddy commented on code in PR #38839: URL: https://github.com/apache/airflow/pull/38839#discussion_r1556101539 ## chart/newsfragments/38478.significant.rst: ## Review Comment: Thanks. Updated ## chart/newsfragments/38478.significant.rst: ##

Re: [PR] Deferrable mode for Custom Training Job operators [airflow]

2024-04-08 Thread via GitHub
potiuk commented on code in PR #38584: URL: https://github.com/apache/airflow/pull/38584#discussion_r1556100397 ## airflow/providers/google/cloud/operators/vertex_ai/custom_job.py: ## @@ -539,6 +564,94 @@ def on_kill(self) -> None: if self.hook:

Re: [PR] Deferrable mode for Custom Training Job operators [airflow]

2024-04-08 Thread via GitHub
potiuk commented on code in PR #38584: URL: https://github.com/apache/airflow/pull/38584#discussion_r1556098638 ## airflow/providers/google/cloud/operators/vertex_ai/custom_job.py: ## @@ -539,6 +564,94 @@ def on_kill(self) -> None: if self.hook:

Re: [PR] Remove decorator from rendering fields example [airflow]

2024-04-08 Thread via GitHub
potiuk merged PR #38827: URL: https://github.com/apache/airflow/pull/38827 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

(airflow) branch main updated: Remove decorator from rendering fields example (#38827)

2024-04-08 Thread potiuk
This is an automated email from the ASF dual-hosted git repository. potiuk pushed a commit to branch main in repository https://gitbox.apache.org/repos/asf/airflow.git The following commit(s) were added to refs/heads/main by this push: new 285f037dbc Remove decorator from rendering fields

Re: [PR] Update ACL during job reset [airflow]

2024-04-08 Thread via GitHub
subham611 commented on code in PR #38741: URL: https://github.com/apache/airflow/pull/38741#discussion_r1556066734 ## airflow/providers/databricks/hooks/databricks.py: ## @@ -655,6 +656,16 @@ def get_repo_by_path(self, path: str) -> str | None: return None +def

Re: [PR] Add Weights and Biases Provider [airflow]

2024-04-08 Thread via GitHub
odaneau-astro closed pull request #38846: Add Weights and Biases Provider URL: https://github.com/apache/airflow/pull/38846 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To

Re: [PR] Paginate Airflow task logs [airflow]

2024-04-08 Thread via GitHub
RNHTTR commented on code in PR #38807: URL: https://github.com/apache/airflow/pull/38807#discussion_r1556051546 ## airflow/providers/amazon/aws/log/s3_task_handler.py: ## @@ -178,7 +180,13 @@ def s3_read(self, remote_log_location: str, return_error: bool = False) -> str:

Re: [PR] Correctly select task in DAG Graph View when clicking on its name [airflow]

2024-04-08 Thread via GitHub
bbovenzi commented on code in PR #38782: URL: https://github.com/apache/airflow/pull/38782#discussion_r1556047065 ## airflow/www/static/js/dag/details/graph/DagNode.tsx: ## @@ -126,10 +125,6 @@ const DagNode = ({ label={taskName} isOpen={isOpen}

Re: [I] cannot start 'airflow standalone'; pendulum.tz.timezone(tz) -> TypeError [airflow]

2024-04-08 Thread via GitHub
dhshah13 commented on issue #38714: URL: https://github.com/apache/airflow/issues/38714#issuecomment-2043070747 and degrade python version to 3.11- it is more stable. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use

Re: [I] [2.9.0] DB migrate throws na error on encrypt_trigger_kwargs [airflow]

2024-04-08 Thread via GitHub
potiuk commented on issue #38836: URL: https://github.com/apache/airflow/issues/38836#issuecomment-2043068257 cc: @hussein-awala -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific

(airflow) branch main updated (87fc581262 -> bfaa4f2012)

2024-04-08 Thread vincbeck
This is an automated email from the ASF dual-hosted git repository. vincbeck pushed a change to branch main in repository https://gitbox.apache.org/repos/asf/airflow.git from 87fc581262 Remove false negative SparkInK8s internal deprecations (#38777) add bfaa4f2012 fix: COMMAND string

Re: [PR] Fix eks.py SyntaxWarning: invalid esape sequence '\s' [airflow]

2024-04-08 Thread via GitHub
vincbeck merged PR #38734: URL: https://github.com/apache/airflow/pull/38734 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

Re: [PR] Amazon Bedrock - Model Customization Jobs [airflow]

2024-04-08 Thread via GitHub
vincbeck commented on code in PR #38693: URL: https://github.com/apache/airflow/pull/38693#discussion_r1556043966 ## airflow/providers/amazon/aws/sensors/bedrock.py: ## @@ -0,0 +1,111 @@ +# +# Licensed to the Apache Software Foundation (ASF) under one +# or more contributor

Re: [PR] fix: update `get_uri` to return URI in sqlAlchemy URI format [airflow]

2024-04-08 Thread via GitHub
rawwar commented on PR #38831: URL: https://github.com/apache/airflow/pull/38831#issuecomment-2043040070 @Taragolis , based on your comment [here](https://github.com/apache/airflow/issues/38195#issuecomment-2042644747) > As far as i remember, we recently discuss this case in slack. I

Re: [I] Connection to Postgres failed due to special characters in a password [airflow]

2024-04-08 Thread via GitHub
rawwar commented on issue #38187: URL: https://github.com/apache/airflow/issues/38187#issuecomment-2043038768 @Taragolis , based on your comment [here](https://github.com/apache/airflow/issues/38195#issuecomment-2042644747) > As far as i remember, we recently discuss this case in

[I] Change the UI color for ExternalSensor tasks [airflow]

2024-04-08 Thread via GitHub
Timelessprod opened a new issue, #38845: URL: https://github.com/apache/airflow/issues/38845 ### Description With the recent update of the graph display in DAGs, there's a visibility issue regarding external task sensors. Indeed those tasks have a `#19647e` tile color while the

Re: [I] Change the UI color for ExternalSensor tasks [airflow]

2024-04-08 Thread via GitHub
boring-cyborg[bot] commented on issue #38845: URL: https://github.com/apache/airflow/issues/38845#issuecomment-2043032243 Thanks for opening your first issue here! Be sure to follow the issue template! If you are willing to raise PR to address this issue please do so, no need to wait for

[I] White INFO log is hard to read [airflow]

2024-04-08 Thread via GitHub
romanzdk opened a new issue, #38844: URL: https://github.com/apache/airflow/issues/38844 ### Apache Airflow version main (development) ### If "Other Airflow 2 version" selected, which one? 2.9.0 ### What happened? White INFO log is hard to read

Re: [PR] Update dagrun.py to fix #38816 [airflow]

2024-04-08 Thread via GitHub
boring-cyborg[bot] commented on PR #38843: URL: https://github.com/apache/airflow/pull/38843#issuecomment-2043013963 Congratulations on your first Pull Request and welcome to the Apache Airflow community! If you have any issues or are unsure about any anything please check our

[PR] Update dagrun.py to fix #38816 [airflow]

2024-04-08 Thread via GitHub
MyLong opened a new pull request, #38843: URL: https://github.com/apache/airflow/pull/38843 'DagRunNote' object has no attribute dag_id\dagrun_id\run_id\map_index so replace those attributes to existing attributes dag_run_id\content -- This is an automated message from the Apache Git

Re: [PR] OpenAI Chat & Assistant hook functions [airflow]

2024-04-08 Thread via GitHub
Lee-W commented on code in PR #38736: URL: https://github.com/apache/airflow/pull/38736#discussion_r1556009587 ## airflow/providers/openai/hooks/openai.py: ## @@ -77,6 +89,165 @@ def get_conn(self) -> OpenAI: **openai_client_kwargs, ) +def

Re: [PR] fix: try002 for provider google [airflow]

2024-04-08 Thread via GitHub
dondaum commented on code in PR #38803: URL: https://github.com/apache/airflow/pull/38803#discussion_r1556001809 ## airflow/providers/google/cloud/hooks/dataflow.py: ## @@ -252,7 +252,7 @@ def _get_current_jobs(self) -> list[dict]: self._job_id = jobs[0]["id"]

Re: [I] Applying queue with SparkSubmitOperator is ignored from 2.6.0 onwards [airflow]

2024-04-08 Thread via GitHub
pateash commented on issue #38461: URL: https://github.com/apache/airflow/issues/38461#issuecomment-2042992467 Checking this, can any one please assign this to me. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the

Re: [PR] Expose AWS IAM missing param in Hashicorp secret [airflow]

2024-04-08 Thread via GitHub
Lee-W commented on code in PR #38536: URL: https://github.com/apache/airflow/pull/38536#discussion_r1555990209 ## airflow/providers/hashicorp/provider.yaml: ## @@ -55,6 +55,7 @@ versions: dependencies: - apache-airflow>=2.6.0 - hvac>=1.1.0 + -

Re: [PR] Bugfix to correct GCSHook being called even when not required with BeamRunPythonPipelineOperator [airflow]

2024-04-08 Thread via GitHub
zstrathe commented on code in PR #38716: URL: https://github.com/apache/airflow/pull/38716#discussion_r1555982325 ## airflow/providers/apache/beam/operators/beam.py: ## @@ -364,11 +364,13 @@ def execute(self, context: Context): def execute_sync(self, context: Context):

  1   2   3   >