[GitHub] [airflow] chuxiangfeng opened a new issue, #29303: KubernetesPodOperator not make task status to success due the task execution time is very short

2023-02-02 Thread via GitHub
chuxiangfeng opened a new issue, #29303: URL: https://github.com/apache/airflow/issues/29303 ### Apache Airflow version Other Airflow 2 version (please specify below) ### What happened airflow version:2.4.1 python:3.7.13 After the KubernetesPodOperator task is

[GitHub] [airflow] liuning89757 commented on issue #27300: Scheduler encounters database update error, then gets stuck in endless loop, yet still shows as healthy

2023-02-02 Thread via GitHub
liuning89757 commented on issue #27300: URL: https://github.com/apache/airflow/issues/27300#issuecomment-1413361554 Hi @potiuk @ashb @ephraimbuddy , hopes my findings may offer some clues about this I have met similar issue, almost each time when a dynamic task expand to 30+ indexes.

[GitHub] [airflow] mhenc commented on pull request #29306: AIP-44 Fix support for classmethods and method mapping

2023-02-02 Thread via GitHub
mhenc commented on PR #29306: URL: https://github.com/apache/airflow/pull/29306#issuecomment-1413396642 cc: @vincbeck -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To

[GitHub] [airflow] mhenc opened a new pull request, #29306: AIP-44 Fix support for classmethods and method mapping

2023-02-02 Thread via GitHub
mhenc opened a new pull request, #29306: URL: https://github.com/apache/airflow/pull/29306 This fixes two bugs: - support for classmethods - Inconsistent method name (send by client vs map on server) introduced in https://github.com/apache/airflow/pull/28693 -- This is an automated

[GitHub] [airflow] mhenc commented on pull request #28783: AIP-44 Don't use Internal API in some components

2023-02-02 Thread via GitHub
mhenc commented on PR #28783: URL: https://github.com/apache/airflow/pull/28783#issuecomment-1413411246 cc: @potiuk @vincbeck -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific

[GitHub] [airflow] Bowrna opened a new issue, #29304: Mismatch of taskinstances executed for dagrun result in API vs web UI

2023-02-02 Thread via GitHub
Bowrna opened a new issue, #29304: URL: https://github.com/apache/airflow/issues/29304 ### Apache Airflow version Other Airflow 2 version (please specify below) ### What happened airflow version: 2.4.2

[GitHub] [airflow] Taragolis opened a new issue, #29305: Migrate remaining tests to `pytest`

2023-02-02 Thread via GitHub
Taragolis opened a new issue, #29305: URL: https://github.com/apache/airflow/issues/29305 ### Body All tests for Apache Airflow are run using [`pytest`](http://doc.pytest.org/en/latest/). We have about 80 tests modules which still use

[GitHub] [airflow] Taragolis commented on pull request #29142: Decrypt SecureString value obtained by SsmHook

2023-02-02 Thread via GitHub
Taragolis commented on PR #29142: URL: https://github.com/apache/airflow/pull/29142#issuecomment-1413363531 @o-nikolas if you have a time could you also have a look (you have a binding approval power) and maybe we could merge it -- This is an automated message from the Apache Git

[GitHub] [airflow] Taragolis commented on issue #29175: Support for Redis Time series in Airflow common packages

2023-02-02 Thread via GitHub
Taragolis commented on issue #29175: URL: https://github.com/apache/airflow/issues/29175#issuecomment-1413481942 > Can this be added to next release please. Why not? If **someone** found that we could safety migrate to new version of [`redis`](https://pypi.org/project/redis/),

[GitHub] [airflow] masterlittle opened a new issue, #29307: Bigquery Operators in defferable mode fail if the location is not US.

2023-02-02 Thread via GitHub
masterlittle opened a new issue, #29307: URL: https://github.com/apache/airflow/issues/29307 ### Apache Airflow Provider(s) google ### Versions of Apache Airflow Providers apache-airflow-providers-google==8.8.0 ### Apache Airflow version 2.5.1 ###

[GitHub] [airflow] boring-cyborg[bot] commented on issue #29307: Bigquery Operators in defferable mode fail if the location is not US.

2023-02-02 Thread via GitHub
boring-cyborg[bot] commented on issue #29307: URL: https://github.com/apache/airflow/issues/29307#issuecomment-1413536222 Thanks for opening your first issue here! Be sure to follow the issue template! -- This is an automated message from the Apache Git Service. To respond to the

[GitHub] [airflow] Taragolis closed pull request #28459: Migrate remaining Always & CLI tests to `pytest`

2023-02-02 Thread via GitHub
Taragolis closed pull request #28459: Migrate remaining Always & CLI tests to `pytest` URL: https://github.com/apache/airflow/pull/28459 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific

[GitHub] [airflow] mhenc commented on issue #28784: AIP-44 Migrate DagFileProcessorManager._fetch_callbacks to Internal API

2023-02-02 Thread via GitHub
mhenc commented on issue #28784: URL: https://github.com/apache/airflow/issues/28784#issuecomment-1413570892 Can you sync with @vincbeck as he may have covered that part as well: https://github.com/apache/airflow/pull/28900 -- This is an automated message from the Apache Git Service.

[GitHub] [airflow] mhenc commented on a diff in pull request #28900: [WIP] Convert DagFileProcessor.execute_callbacks to Internal API

2023-02-02 Thread via GitHub
mhenc commented on code in PR #28900: URL: https://github.com/apache/airflow/pull/28900#discussion_r1094274691 ## airflow/models/dag.py: ## @@ -1314,23 +1356,28 @@ def handle_callback(self, dagrun, success=True, reason=None, session=NEW_SESSION :param reason:

[GitHub] [airflow] csp33 commented on issue #29273: Status of testing of Apache Airflow Helm Chart 1.8.0rc1

2023-02-02 Thread via GitHub
csp33 commented on issue #29273: URL: https://github.com/apache/airflow/issues/29273#issuecomment-1415273070 #27443 is also OK -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific

[GitHub] [airflow] csp33 commented on issue #29273: Status of testing of Apache Airflow Helm Chart 1.8.0rc1

2023-02-02 Thread via GitHub
csp33 commented on issue #29273: URL: https://github.com/apache/airflow/issues/29273#issuecomment-1415266941 Guess we can confirm that #27230, #27668, #28074 and #29074 are working✌

[GitHub] [airflow] ecerulm commented on pull request #29016: Fix leak sensitive field via V1EnvVar on exception

2023-02-02 Thread via GitHub
ecerulm commented on PR #29016: URL: https://github.com/apache/airflow/pull/29016#issuecomment-1415206145 I had to fix some formatting issues and some stuff from the rebase from main but now "all checks have passed". -- This is an automated message from the Apache Git Service. To

[GitHub] [airflow] Bowrna commented on issue #29304: Mismatch of taskinstances executed for dagrun result in API vs web UI

2023-02-02 Thread via GitHub
Bowrna commented on issue #29304: URL: https://github.com/apache/airflow/issues/29304#issuecomment-1415237521 @potiuk thanks for clarifying this part to me. yes it makes sense to me now. i have wrongly assumed that the api response is incorrect. -- This is an automated message from the

[GitHub] [airflow] csp33 commented on issue #29273: Status of testing of Apache Airflow Helm Chart 1.8.0rc1

2023-02-02 Thread via GitHub
csp33 commented on issue #29273: URL: https://github.com/apache/airflow/issues/29273#issuecomment-1415276663 All good in #27625 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific

[GitHub] [airflow] csp33 commented on issue #29273: Status of testing of Apache Airflow Helm Chart 1.8.0rc1

2023-02-02 Thread via GitHub
csp33 commented on issue #29273: URL: https://github.com/apache/airflow/issues/29273#issuecomment-1415270745 #27178 works as expected -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific

[GitHub] [airflow] kaxil commented on a diff in pull request #29260: Add deferrable mode to ExternalTaskSensor

2023-02-02 Thread via GitHub
kaxil commented on code in PR #29260: URL: https://github.com/apache/airflow/pull/29260#discussion_r1094545679 ## airflow/triggers/external_task.py: ## @@ -0,0 +1,121 @@ +# +# Licensed to the Apache Software Foundation (ASF) under one +# or more contributor license agreements.

[GitHub] [airflow] potiuk commented on issue #29304: Mismatch of taskinstances executed for dagrun result in API vs web UI

2023-02-02 Thread via GitHub
potiuk commented on issue #29304: URL: https://github.com/apache/airflow/issues/29304#issuecomment-1413714902 I believe this is because previously you had "query_snowflake" task in your DAG that has been renamed to "snowflake_query" (notice that the UI si "snowflake_query" and in the API

[GitHub] [airflow] potiuk closed issue #29304: Mismatch of taskinstances executed for dagrun result in API vs web UI

2023-02-02 Thread via GitHub
potiuk closed issue #29304: Mismatch of taskinstances executed for dagrun result in API vs web UI URL: https://github.com/apache/airflow/issues/29304 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go

[GitHub] [airflow] alexandermalyga opened a new pull request, #29314: Add ttlSecondsAfterFinished to migrateDatabaseJob and createUserJob

2023-02-02 Thread via GitHub
alexandermalyga opened a new pull request, #29314: URL: https://github.com/apache/airflow/pull/29314 related: #27561 Adds the [ttlSecondsAfterFinished](https://kubernetes.io/docs/concepts/workloads/controllers/ttlafterfinished/) parameter to `migrateDatabaseJob` and

[GitHub] [airflow] josh-fell commented on a diff in pull request #29166: Drop Connection.schema use in DbtCloudHook

2023-02-02 Thread via GitHub
josh-fell commented on code in PR #29166: URL: https://github.com/apache/airflow/pull/29166#discussion_r1094565675 ## airflow/providers/dbt/cloud/hooks/dbt.py: ## @@ -181,33 +181,46 @@ def __init__(self, dbt_cloud_conn_id: str = default_conn_name, *args, **kwargs)

[GitHub] [airflow] mhenc closed issue #28787: AIP-44 Migrate Dag.pickle to Internal API

2023-02-02 Thread via GitHub
mhenc closed issue #28787: AIP-44 Migrate Dag.pickle to Internal API URL: https://github.com/apache/airflow/issues/28787 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To

[GitHub] [airflow] mhenc commented on issue #28787: AIP-44 Migrate Dag.pickle to Internal API

2023-02-02 Thread via GitHub
mhenc commented on issue #28787: URL: https://github.com/apache/airflow/issues/28787#issuecomment-1413811254 Handled in https://github.com/apache/airflow/pull/29188 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the

[GitHub] [airflow] boring-cyborg[bot] commented on issue #29318: Return error when requirements fail to install

2023-02-02 Thread via GitHub
boring-cyborg[bot] commented on issue #29318: URL: https://github.com/apache/airflow/issues/29318#issuecomment-1413815891 Thanks for opening your first issue here! Be sure to follow the issue template! -- This is an automated message from the Apache Git Service. To respond to the

[GitHub] [airflow] mhenc opened a new issue, #29317: AIP-44 Migrate TaskInstance.are_dependencies_met to Internal API

2023-02-02 Thread via GitHub
mhenc opened a new issue, #29317: URL: https://github.com/apache/airflow/issues/29317 https://github.com/apache/airflow/blob/main/airflow/models/taskinstance.py#L1063 needed by https://github.com/apache/airflow/blob/main/airflow/models/taskinstance.py#L1252 for

[GitHub] [airflow] ScriptPup opened a new issue, #29318: Return error when requirements fail to install

2023-02-02 Thread via GitHub
ScriptPup opened a new issue, #29318: URL: https://github.com/apache/airflow/issues/29318 ### Description When the PythonVirtualenvOperator fails to install requirements, it shows something like this ` ``` subprocess.CalledProcessError: Command

[airflow] branch constraints-main updated: Updating constraints. Github run id:4075031839

2023-02-02 Thread github-bot
This is an automated email from the ASF dual-hosted git repository. github-bot pushed a commit to branch constraints-main in repository https://gitbox.apache.org/repos/asf/airflow.git The following commit(s) were added to refs/heads/constraints-main by this push: new 7ddc80db7e Updating

[GitHub] [airflow] ecerulm commented on pull request #29016: Fix leak sensitive field via V1EnvVar on exception

2023-02-02 Thread via GitHub
ecerulm commented on PR #29016: URL: https://github.com/apache/airflow/pull/29016#issuecomment-1413772510 Nevermind, I realized now that the `ConvertableToDict` approach does not work on Python 3.7 (as `typing.Protocol` was introduced in 3.8). So I guess I'll try your try-import approach

[GitHub] [airflow] syun64 commented on pull request #29093: Enable tagged metric names for existing Statsd metric publishing events | influxdb-statsd support

2023-02-02 Thread via GitHub
syun64 commented on PR #29093: URL: https://github.com/apache/airflow/pull/29093#issuecomment-1413839397 > Some nits. General logic lgtm. Thank you! I've made the suggested changes. -- This is an automated message from the Apache Git Service. To respond to the message, please log

[GitHub] [airflow] mhenc opened a new issue, #29308: AIP-44 Migrate TaskInstance.schedule_downstream_tasks to Internal API

2023-02-02 Thread via GitHub
mhenc opened a new issue, #29308: URL: https://github.com/apache/airflow/issues/29308 https://github.com/apache/airflow/blob/main/airflow/models/taskinstance.py#L2570 used in https://github.com/apache/airflow/blob/main/airflow/jobs/local_task_job.py#L220 -- This is an

[GitHub] [airflow] mhenc opened a new issue, #29311: AIP-44 Migrate TaskInstance.refresh_from_db to Internal API

2023-02-02 Thread via GitHub
mhenc opened a new issue, #29311: URL: https://github.com/apache/airflow/issues/29311 https://github.com/apache/airflow/blob/main/airflow/models/taskinstance.py#L770 Note it is used in multiple places. The place needed by AIP-44

[GitHub] [airflow] mhenc opened a new issue, #29312: AIP-44 Migrate TaskInstance.get_template_context to Internal API

2023-02-02 Thread via GitHub
mhenc opened a new issue, #29312: URL: https://github.com/apache/airflow/issues/29312 https://github.com/apache/airflow/blob/main/airflow/models/taskinstance.py#L1884 used in https://github.com/apache/airflow/blob/main/airflow/models/taskinstance.py#L1386 -- This is an

[GitHub] [airflow] kaxil opened a new pull request, #29313: Add triggers for ExternalTask

2023-02-02 Thread via GitHub
kaxil opened a new pull request, #29313: URL: https://github.com/apache/airflow/pull/29313 Contributes back two of the core Triggers from https://github.com/astronomer/astronomer-providers so that it can be used to create an operator /sensor or used within taskflow API ---

[GitHub] [airflow] kaxil commented on a diff in pull request #29260: Add deferrable mode to ExternalTaskSensor

2023-02-02 Thread via GitHub
kaxil commented on code in PR #29260: URL: https://github.com/apache/airflow/pull/29260#discussion_r1094547244 ## airflow/sensors/external_task.py: ## @@ -360,6 +362,93 @@ def _handle_execution_date_fn(self, context) -> Any: return kwargs_callable(logical_date,

[GitHub] [airflow] snjypl commented on a diff in pull request #29188: AIP-44 Migrate Dagbag.sync_to_db to internal API.

2023-02-02 Thread via GitHub
snjypl commented on code in PR #29188: URL: https://github.com/apache/airflow/pull/29188#discussion_r1094546495 ## airflow/dag_processing/processor.py: ## @@ -837,3 +832,26 @@ def process_file( self.log.exception("Error logging DAG warnings.") return

[GitHub] [airflow] kaxil commented on a diff in pull request #29300: Add deferrable BatchOperator

2023-02-02 Thread via GitHub
kaxil commented on code in PR #29300: URL: https://github.com/apache/airflow/pull/29300#discussion_r1094552773 ## generated/provider_dependencies.json: ## @@ -17,6 +17,7 @@ }, "amazon": { "deps": [ + "aiobotocore>=2.1.1", Review Comment: Same comment

[GitHub] [airflow] mhenc opened a new issue, #29315: AIP-44 Migrate BaseJob.heartbeat to InternalAPI

2023-02-02 Thread via GitHub
mhenc opened a new issue, #29315: URL: https://github.com/apache/airflow/issues/29315 https://github.com/apache/airflow/blob/main/airflow/jobs/base_job.py#L182 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use

[GitHub] [airflow] josh-fell commented on issue #26760: DAG on_failure_callback uses wrong context

2023-02-02 Thread via GitHub
josh-fell commented on issue #26760: URL: https://github.com/apache/airflow/issues/26760#issuecomment-1413808038 This has been coming up more frequently for users. I can take on related improvements here. -- This is an automated message from the Apache Git Service. To respond to the

[GitHub] [airflow] mhenc opened a new issue, #29316: AIP-44 Migrate BaseJob.run to Internal API

2023-02-02 Thread via GitHub
mhenc opened a new issue, #29316: URL: https://github.com/apache/airflow/issues/29316 https://github.com/apache/airflow/blob/main/airflow/jobs/base_job.py#L248 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL

[GitHub] [airflow] ZJ577624359 opened a new issue, #29319: DAG cannot execute

2023-02-02 Thread via GitHub
ZJ577624359 opened a new issue, #29319: URL: https://github.com/apache/airflow/issues/29319 ### Apache Airflow version Other Airflow 2 version (please specify below) ### What happened version 2.4 *** Falling back to local log *** Log file does not exist:

[GitHub] [airflow] boring-cyborg[bot] commented on issue #29319: DAG cannot execute

2023-02-02 Thread via GitHub
boring-cyborg[bot] commented on issue #29319: URL: https://github.com/apache/airflow/issues/29319#issuecomment-1413835027 Thanks for opening your first issue here! Be sure to follow the issue template! -- This is an automated message from the Apache Git Service. To respond to the

[GitHub] [airflow] norm commented on pull request #29309: Add `airflow db drop-archived` command

2023-02-02 Thread via GitHub
norm commented on PR #29309: URL: https://github.com/apache/airflow/pull/29309#issuecomment-1413846077 Think docs/apache-airflow/howto/usage-cli.rst needs to be updated too. (He repeated himself. :) -- This is an automated message from the Apache Git Service. To respond to the

[GitHub] [airflow] VladaZakharova commented on a diff in pull request #29260: Add deferrable mode to ExternalTaskSensor

2023-02-02 Thread via GitHub
VladaZakharova commented on code in PR #29260: URL: https://github.com/apache/airflow/pull/29260#discussion_r1094455152 ## airflow/sensors/external_task.py: ## @@ -360,6 +362,93 @@ def _handle_execution_date_fn(self, context) -> Any: return

[GitHub] [airflow] VladaZakharova commented on a diff in pull request #29260: Add deferrable mode to ExternalTaskSensor

2023-02-02 Thread via GitHub
VladaZakharova commented on code in PR #29260: URL: https://github.com/apache/airflow/pull/29260#discussion_r1094455152 ## airflow/sensors/external_task.py: ## @@ -360,6 +362,93 @@ def _handle_execution_date_fn(self, context) -> Any: return

[GitHub] [airflow] kaxil commented on a diff in pull request #28850: Add deferrable mode in RedshiftPauseClusterOperator

2023-02-02 Thread via GitHub
kaxil commented on code in PR #28850: URL: https://github.com/apache/airflow/pull/28850#discussion_r1094475545 ## airflow/providers/amazon/provider.yaml: ## @@ -52,6 +52,7 @@ dependencies: - apache-airflow>=2.3.0 - apache-airflow-providers-common-sql>=1.3.1 -

[GitHub] [airflow] kolfild26 commented on pull request #28998: write to DB a user who triggered a dag via API endpoint request

2023-02-02 Thread via GitHub
kolfild26 commented on PR #28998: URL: https://github.com/apache/airflow/pull/28998#issuecomment-1413759778 Above there was a conflict with the very fresh commit https://github.com/apache/airflow/commit/b94f36bf563f5c8372086cec63b74eadef638ef8 in `main`. It also changed the same test in

[GitHub] [airflow] kaxil merged pull request #29313: Add triggers for ExternalTask

2023-02-02 Thread via GitHub
kaxil merged PR #29313: URL: https://github.com/apache/airflow/pull/29313 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

[airflow] branch main updated (cd5a92c2af -> 6ec97dc649)

2023-02-02 Thread kaxilnaik
This is an automated email from the ASF dual-hosted git repository. kaxilnaik pushed a change to branch main in repository https://gitbox.apache.org/repos/asf/airflow.git from cd5a92c2af Fix airflow_dags_mount formatting (#29296) add 6ec97dc649 Add triggers for ExternalTask (#29313)

[GitHub] [airflow] potiuk commented on issue #29281: (psycopg2.errors.FeatureNotSupported) This ALTER TABLE command is not yet supported .While deploying airflow with Postgres

2023-02-02 Thread via GitHub
potiuk commented on issue #29281: URL: https://github.com/apache/airflow/issues/29281#issuecomment-1413864978 Postgres DB not library. Look for postgres support in Airflow. It start from postgres 11. (Find it in docs). No idea whar -- This is an automated message from the Apache Git

[GitHub] [airflow] ephraimbuddy opened a new pull request, #29309: Add `airflow db drop-archived` command

2023-02-02 Thread via GitHub
ephraimbuddy opened a new pull request, #29309: URL: https://github.com/apache/airflow/pull/29309 This command drops the archive tables directly As part of this, the _confirm_drop_archives function was made more interactive -- This is an automated message from the Apache

[GitHub] [airflow] mhenc closed issue #28786: AIP-44 Migrate dagbag.sync_to_db to Internal API

2023-02-02 Thread via GitHub
mhenc closed issue #28786: AIP-44 Migrate dagbag.sync_to_db to Internal API URL: https://github.com/apache/airflow/issues/28786 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment.

[GitHub] [airflow] mhenc commented on issue #28786: AIP-44 Migrate dagbag.sync_to_db to Internal API

2023-02-02 Thread via GitHub
mhenc commented on issue #28786: URL: https://github.com/apache/airflow/issues/28786#issuecomment-1413658445 Duplicate of https://github.com/apache/airflow/issues/28615 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use

[airflow] branch main updated (d80b583db0 -> aed5c371c4)

2023-02-02 Thread kaxilnaik
This is an automated email from the ASF dual-hosted git repository. kaxilnaik pushed a change to branch main in repository https://gitbox.apache.org/repos/asf/airflow.git from d80b583db0 Fix `breeze static_checks` type arg (#29292) add aed5c371c4 Fix webserver.service.ports formatting

[airflow] branch main updated (aed5c371c4 -> cd5a92c2af)

2023-02-02 Thread kaxilnaik
This is an automated email from the ASF dual-hosted git repository. kaxilnaik pushed a change to branch main in repository https://gitbox.apache.org/repos/asf/airflow.git from aed5c371c4 Fix webserver.service.ports formatting (#29297) add cd5a92c2af Fix airflow_dags_mount formatting

[GitHub] [airflow] kaxil merged pull request #29296: Fix airflow_dags_mount formatting

2023-02-02 Thread via GitHub
kaxil merged PR #29296: URL: https://github.com/apache/airflow/pull/29296 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

[GitHub] [airflow] kaxil merged pull request #29297: Fix webserver.service.ports formatting

2023-02-02 Thread via GitHub
kaxil merged PR #29297: URL: https://github.com/apache/airflow/pull/29297 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

[GitHub] [airflow] norm commented on a diff in pull request #29309: Add `airflow db drop-archived` command

2023-02-02 Thread via GitHub
norm commented on code in PR #29309: URL: https://github.com/apache/airflow/pull/29309#discussion_r1094533045 ## airflow/cli/commands/db_command.py: ## @@ -218,3 +218,12 @@ def export_cleaned(args): table_names=args.tables, drop_archives=args.drop_archives,

[GitHub] [airflow] ephraimbuddy commented on a diff in pull request #29313: Add triggers for ExternalTask

2023-02-02 Thread via GitHub
ephraimbuddy commented on code in PR #29313: URL: https://github.com/apache/airflow/pull/29313#discussion_r1094525886 ## airflow/triggers/external_task.py: ## @@ -0,0 +1,163 @@ +# Licensed to the Apache Software Foundation (ASF) under one +# or more contributor license

[GitHub] [airflow] vksunilk opened a new issue, #29344: Check Absence of files or objects via GCSObjectExistenceSensor

2023-02-02 Thread via GitHub
vksunilk opened a new issue, #29344: URL: https://github.com/apache/airflow/issues/29344 ### Description Currently GCSObjectExistenceSensor fails if the object is not present. We can add a parameter `check_for_absence`, which when true, should check for absence and should fail if

[GitHub] [airflow] potiuk commented on issue #29281: (psycopg2.errors.FeatureNotSupported) This ALTER TABLE command is not yet supported .While deploying airflow with Postgres

2023-02-02 Thread via GitHub
potiuk commented on issue #29281: URL: https://github.com/apache/airflow/issues/29281#issuecomment-1415078837 I think you failed to mention that you are not using Postgres but something else that claims that it is Postgres compatible (Yugabyte?). Apparent kk y this database is - as opposed

[GitHub] [airflow] potiuk closed issue #29281: (psycopg2.errors.FeatureNotSupported) This ALTER TABLE command is not yet supported .While deploying airflow with Postgres

2023-02-02 Thread via GitHub
potiuk closed issue #29281: (psycopg2.errors.FeatureNotSupported) This ALTER TABLE command is not yet supported .While deploying airflow with Postgres URL: https://github.com/apache/airflow/issues/29281 -- This is an automated message from the Apache Git Service. To respond to the message,

[GitHub] [airflow] rajaths010494 commented on a diff in pull request #29300: Add deferrable BatchOperator

2023-02-02 Thread via GitHub
rajaths010494 commented on code in PR #29300: URL: https://github.com/apache/airflow/pull/29300#discussion_r1095392580 ## generated/provider_dependencies.json: ## @@ -17,6 +17,7 @@ }, "amazon": { "deps": [ + "aiobotocore>=2.1.1", Review Comment: Sure will

[GitHub] [airflow] vksunilk opened a new pull request, #29346: Check Absence of files or objects via GCSObjectExistenceSensor

2023-02-02 Thread via GitHub
vksunilk opened a new pull request, #29346: URL: https://github.com/apache/airflow/pull/29346 --- **^ Add meaningful description above** Read the **[Pull Request Guidelines](https://github.com/apache/airflow/blob/main/CONTRIBUTING.rst#pull-request-guidelines)** for more

[GitHub] [airflow] pmercatoris opened a new issue, #29339: Failing to execute successive Dynamic Task Group Mapping

2023-02-02 Thread via GitHub
pmercatoris opened a new issue, #29339: URL: https://github.com/apache/airflow/issues/29339 ### Apache Airflow version 2.5.1 ### What happened I am trying to map a task group within another task group which is already mapped. However, when launching the dag, all tasks

[GitHub] [airflow] ephraimbuddy commented on pull request #29309: Add `airflow db drop-archived` command

2023-02-02 Thread via GitHub
ephraimbuddy commented on PR #29309: URL: https://github.com/apache/airflow/pull/29309#issuecomment-1414029554 > (He repeated himself. :) Haha :) 臘‍♂️ -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL

[GitHub] [airflow] alexandermalyga commented on a diff in pull request #29314: Add ttlSecondsAfterFinished to migrateDatabaseJob and createUserJob

2023-02-02 Thread via GitHub
alexandermalyga commented on code in PR #29314: URL: https://github.com/apache/airflow/pull/29314#discussion_r1094788115 ## chart/templates/jobs/create-user-job.yaml: ## @@ -48,6 +48,9 @@ metadata: {{- $annotations | toYaml | nindent 4 }} {{- end }} spec: + {{- if not

[GitHub] [airflow] emilymitacc commented on issue #29199: TaskFlow AirflowSkipException causes downstream step to fail when multiple_outputs is true

2023-02-02 Thread via GitHub
emilymitacc commented on issue #29199: URL: https://github.com/apache/airflow/issues/29199#issuecomment-1414044622 I also have this problem, please your help to solve it -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use

[GitHub] [airflow] jedcunningham merged pull request #29257: Initialize and validate conf early and intentionally

2023-02-02 Thread via GitHub
jedcunningham merged PR #29257: URL: https://github.com/apache/airflow/pull/29257 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

[airflow] branch main updated (91c0ce7666 -> 338a633fc9)

2023-02-02 Thread jedcunningham
This is an automated email from the ASF dual-hosted git repository. jedcunningham pushed a change to branch main in repository https://gitbox.apache.org/repos/asf/airflow.git from 91c0ce7666 Drop Connection.schema use in DbtCloudHook (#29166) add 338a633fc9 Initialize and validate conf

[airflow] branch main updated (56175e4afa -> 478fd82652)

2023-02-02 Thread bbovenzi
This is an automated email from the ASF dual-hosted git repository. bbovenzi pushed a change to branch main in repository https://gitbox.apache.org/repos/asf/airflow.git from 56175e4afa Helm deployment fails when postgresql.nameOverride is used (#29214) add 478fd82652 Expose

[GitHub] [airflow] ephraimbuddy commented on a diff in pull request #29306: AIP-44 Fix support for classmethods and method mapping

2023-02-02 Thread via GitHub
ephraimbuddy commented on code in PR #29306: URL: https://github.com/apache/airflow/pull/29306#discussion_r1094836563 ## airflow/api_internal/endpoints/rpc_api_endpoint.py: ## @@ -81,6 +81,7 @@ def internal_airflow_api(body: dict[str, Any]) -> APIResponse:

[GitHub] [airflow] bbovenzi merged pull request #28636: Expose updated_at filter for dag run and task instance endpoints

2023-02-02 Thread via GitHub
bbovenzi merged PR #28636: URL: https://github.com/apache/airflow/pull/28636 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

[GitHub] [airflow] vincbeck commented on a diff in pull request #29306: AIP-44 Fix support for classmethods and method mapping

2023-02-02 Thread via GitHub
vincbeck commented on code in PR #29306: URL: https://github.com/apache/airflow/pull/29306#discussion_r1094834580 ## airflow/api_internal/endpoints/rpc_api_endpoint.py: ## @@ -81,6 +81,7 @@ def internal_airflow_api(body: dict[str, Any]) -> APIResponse: log.debug("Calling

[GitHub] [airflow] alejandrofm opened a new issue, #29322: DAG list, sorting lost when switching page

2023-02-02 Thread via GitHub
alejandrofm opened a new issue, #29322: URL: https://github.com/apache/airflow/issues/29322 ### Apache Airflow version Other Airflow 2 version (please specify below) ### What happened Hi, I'm currently on Airflow 2.4.2 In /home when sorting by DAG/Owner/Next Run and

[GitHub] [airflow] alejandrofm opened a new issue, #29323: DAG dependencies graph not updating when deleting a DAG

2023-02-02 Thread via GitHub
alejandrofm opened a new issue, #29323: URL: https://github.com/apache/airflow/issues/29323 ### Apache Airflow version Other Airflow 2 version (please specify below) ### What happened ON Airflow 2.4.2 Dag dependencies graph show deleted DAGs that use to have

[GitHub] [airflow] josh-fell merged pull request #29166: Drop Connection.schema use in DbtCloudHook

2023-02-02 Thread via GitHub
josh-fell merged PR #29166: URL: https://github.com/apache/airflow/pull/29166 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

[GitHub] [airflow] CarrascoEr commented on issue #29199: TaskFlow AirflowSkipException causes downstream step to fail when multiple_outputs is true

2023-02-02 Thread via GitHub
CarrascoEr commented on issue #29199: URL: https://github.com/apache/airflow/issues/29199#issuecomment-1414041105 stuck with this too, hoping for this feature to be released soon -- This is an automated message from the Apache Git Service. To respond to the message, please log on to

[airflow] branch main updated: Drop Connection.schema use in DbtCloudHook (#29166)

2023-02-02 Thread joshfell
This is an automated email from the ASF dual-hosted git repository. joshfell pushed a commit to branch main in repository https://gitbox.apache.org/repos/asf/airflow.git The following commit(s) were added to refs/heads/main by this push: new 91c0ce7666 Drop Connection.schema use in

[GitHub] [airflow] uranusjr commented on pull request #29016: Fix leak sensitive field via V1EnvVar on exception

2023-02-02 Thread via GitHub
uranusjr commented on PR #29016: URL: https://github.com/apache/airflow/pull/29016#issuecomment-1414056451 We actually have `Protocol` available on 3.7 via `airflow.typing_compat`, but either way is fine as I mentioned above, so since you already made the change let’s just keep this. --

[GitHub] [airflow] bbovenzi commented on pull request #28713: Add a param for get_dags endpoint to list only unpaused dags

2023-02-02 Thread via GitHub
bbovenzi commented on PR #28713: URL: https://github.com/apache/airflow/pull/28713#issuecomment-1414100779 @hussein-awala Could you rebase and regenerate the yaml and typescript files please? -- This is an automated message from the Apache Git Service. To respond to the message, please

[GitHub] [airflow] vincbeck commented on a diff in pull request #28900: [WIP] Convert DagFileProcessor.execute_callbacks to Internal API

2023-02-02 Thread via GitHub
vincbeck commented on code in PR #28900: URL: https://github.com/apache/airflow/pull/28900#discussion_r1094867921 ## airflow/models/dag.py: ## @@ -1298,6 +1298,48 @@ def normalized_schedule_interval(self) -> ScheduleInterval: _schedule_interval =

[GitHub] [airflow] mhenc opened a new issue, #29320: AIP-44 Migrate TaskCommand._get_ti to Internal API

2023-02-02 Thread via GitHub
mhenc opened a new issue, #29320: URL: https://github.com/apache/airflow/issues/29320 https://github.com/apache/airflow/blob/main/airflow/cli/commands/task_command.py#L145 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and

[GitHub] [airflow] Taragolis commented on a diff in pull request #27616: Flush the session in `handle_failure()` prior to invoking callbacks

2023-02-02 Thread via GitHub
Taragolis commented on code in PR #27616: URL: https://github.com/apache/airflow/pull/27616#discussion_r1094709928 ## airflow/models/taskinstance.py: ## @@ -1808,13 +1808,13 @@ def handle_failure( except Exception: self.log.exception("Failed to

[GitHub] [airflow] jedcunningham commented on issue #29273: Status of testing of Apache Airflow Helm Chart 1.8.0rc1

2023-02-02 Thread via GitHub
jedcunningham commented on issue #29273: URL: https://github.com/apache/airflow/issues/29273#issuecomment-1414065922 @JulesTriomphe, I fully intend to do releases more quickly than we have in the past. So "sometime soon" meaning in the next month, yes, in the next week or so, no. --

[GitHub] [airflow] keu opened a new issue, #29321: Airflow doesn't show an error when there are DAGs with the same ID

2023-02-02 Thread via GitHub
keu opened a new issue, #29321: URL: https://github.com/apache/airflow/issues/29321 ### Apache Airflow version 2.5.1 ### What happened Create two files with DAGs Both DAGs should have the same ID Airflow will constantly update DAG structure/code resulting in dozens

[GitHub] [airflow] boring-cyborg[bot] commented on issue #29321: Airflow doesn't show an error when there are DAGs with the same ID

2023-02-02 Thread via GitHub
boring-cyborg[bot] commented on issue #29321: URL: https://github.com/apache/airflow/issues/29321#issuecomment-1413902029 Thanks for opening your first issue here! Be sure to follow the issue template! -- This is an automated message from the Apache Git Service. To respond to the

[GitHub] [airflow] tsoud commented on a diff in pull request #29158: Update pipeline.rst - Fix query in `merge_data()` task

2023-02-02 Thread via GitHub
tsoud commented on code in PR #29158: URL: https://github.com/apache/airflow/pull/29158#discussion_r1092087567 ## docs/apache-airflow/tutorial/pipeline.rst: ## @@ -170,7 +170,7 @@ Here we select completely unique records from the retrieved data, then we check

[airflow] branch main updated (338a633fc9 -> 56175e4afa)

2023-02-02 Thread jedcunningham
This is an automated email from the ASF dual-hosted git repository. jedcunningham pushed a change to branch main in repository https://gitbox.apache.org/repos/asf/airflow.git from 338a633fc9 Initialize and validate conf early and intentionally (#29257) add 56175e4afa Helm deployment

[GitHub] [airflow] jedcunningham merged pull request #29214: Helm deployment fails when postgresql.nameOverride is used

2023-02-02 Thread via GitHub
jedcunningham merged PR #29214: URL: https://github.com/apache/airflow/pull/29214 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

[GitHub] [airflow] morkot commented on issue #29273: Status of testing of Apache Airflow Helm Chart 1.8.0rc1

2023-02-02 Thread via GitHub
morkot commented on issue #29273: URL: https://github.com/apache/airflow/issues/29273#issuecomment-1414076234 #27671 has been checked. Changes are there, as expected. You can tick that one off. -- This is an automated message from the Apache Git Service. To respond to the message, please

[GitHub] [airflow] boring-cyborg[bot] commented on pull request #29338: Fixed link to airflow variable in docs

2023-02-02 Thread via GitHub
boring-cyborg[bot] commented on PR #29338: URL: https://github.com/apache/airflow/pull/29338#issuecomment-1414094278 Congratulations on your first Pull Request and welcome to the Apache Airflow community! If you have any issues or are unsure about any anything please check our Contribution

[GitHub] [airflow] Adaverse opened a new pull request, #29338: Fixed link to airflow variable in docs

2023-02-02 Thread via GitHub
Adaverse opened a new pull request, #29338: URL: https://github.com/apache/airflow/pull/29338 Background: While going through the Dynamic DAG Generation doc -> https://airflow.apache.org/docs/apache-airflow/stable/howto/dynamic-dag-generation.html, found out one of the links to best

[GitHub] [airflow] hussein-awala commented on pull request #28713: Add a param for get_dags endpoint to list only unpaused dags

2023-02-02 Thread via GitHub
hussein-awala commented on PR #28713: URL: https://github.com/apache/airflow/pull/28713#issuecomment-1414122546 @bbovenzi done -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific

[GitHub] [airflow] JulesTriomphe commented on issue #29273: Status of testing of Apache Airflow Helm Chart 1.8.0rc1

2023-02-02 Thread via GitHub
JulesTriomphe commented on issue #29273: URL: https://github.com/apache/airflow/issues/29273#issuecomment-1414140194 Very cool, thanks for the clarification @jedcunningham ! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and

[GitHub] [airflow] potiuk commented on a diff in pull request #27758: Enable trigger logging in webserver

2023-02-02 Thread via GitHub
potiuk commented on code in PR #27758: URL: https://github.com/apache/airflow/pull/27758#discussion_r1094910327 ## docs/apache-airflow/administration-and-deployment/logging-monitoring/logging-tasks.rst: ## @@ -47,11 +44,14 @@ These patterns can be adjusted by

[GitHub] [airflow] eladkal commented on pull request #28338: New AWS sensor — DynamoDBValueSensor

2023-02-02 Thread via GitHub
eladkal commented on PR #28338: URL: https://github.com/apache/airflow/pull/28338#issuecomment-1414243753 > @ferruzzi Am I to use `tests/providers/amazon/aws/providers` for my unit test? It should be in `tests/providers/amazon/aws/sensors/test_dynamodb.py` If file doesnt

  1   2   >