Re: [PR] Enable none default ftp port [airflow]

2024-04-21 Thread via GitHub
kerlion commented on PR #39048: URL: https://github.com/apache/airflow/pull/39048#issuecomment-2068328606 Hi, is this ready for merging, anything else need I to do? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the

Re: [PR] [FEAT] added notebook error in databricks deferrable handler [airflow]

2024-04-21 Thread via GitHub
Lee-W commented on code in PR #39110: URL: https://github.com/apache/airflow/pull/39110#discussion_r1573991574 ## airflow/providers/databricks/operators/databricks.py: ## @@ -156,17 +144,37 @@ def _handle_deferrable_databricks_operator_execution(operator, hook, log, contex

Re: [PR] Don't clear downstream tasks when marked status is failure and downstream is True [airflow]

2024-04-21 Thread via GitHub
github-actions[bot] commented on PR #23079: URL: https://github.com/apache/airflow/pull/23079#issuecomment-2068278429 This pull request has been automatically marked as stale because it has not had recent activity. It will be closed in 5 days if no further activity occurs. Thank you for

Re: [PR] add dns_config and diagnostics parameters to AzureContainerInstancesOperator [airflow]

2024-04-21 Thread via GitHub
eladkal commented on code in PR #39156: URL: https://github.com/apache/airflow/pull/39156#discussion_r1573930933 ## airflow/providers/microsoft/azure/operators/container_instances.py: ## @@ -113,6 +117,13 @@ class AzureContainerInstancesOperator(BaseOperator):

Re: [PR] add dns_config and diagnostics parameters to AzureContainerInstancesOperator [airflow]

2024-04-21 Thread via GitHub
eladkal commented on code in PR #39156: URL: https://github.com/apache/airflow/pull/39156#discussion_r1573930869 ## airflow/providers/microsoft/azure/provider.yaml: ## @@ -23,6 +23,7 @@ state: ready source-date-epoch: 1709555852 # note that those versions are maintained by

Re: [PR] add dns_config and diagnostics parameters to AzureContainerInstancesOperator [airflow]

2024-04-21 Thread via GitHub
eladkal commented on code in PR #39156: URL: https://github.com/apache/airflow/pull/39156#discussion_r1573930728 ## airflow/providers/microsoft/azure/CHANGELOG.rst: ## Review Comment: Please remove changea to this file. Its not needed for features/bug fixes as it is auto

Re: [PR] [FEAT] added notebook error in databricks deferrable handler [airflow]

2024-04-21 Thread via GitHub
gaurav7261 commented on code in PR #39110: URL: https://github.com/apache/airflow/pull/39110#discussion_r1573919570 ## airflow/providers/databricks/operators/databricks.py: ## @@ -156,17 +144,37 @@ def _handle_deferrable_databricks_operator_execution(operator, hook, log,

(airflow) branch main updated (59084fd1f4 -> aa5581e30d)

2024-04-21 Thread potiuk
This is an automated email from the ASF dual-hosted git repository. potiuk pushed a change to branch main in repository https://gitbox.apache.org/repos/asf/airflow.git from 59084fd1f4 fix: add retry logic in case of google auth refresh credential error (#38961) add aa5581e30d Update

Re: [PR] Update providers metadata 2024-04-21 [airflow]

2024-04-21 Thread via GitHub
potiuk merged PR #39152: URL: https://github.com/apache/airflow/pull/39152 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

[PR] add dns_config and diagnostics parameters to AzureContainerInstancesOperator [airflow]

2024-04-21 Thread via GitHub
Kylea650 opened a new pull request, #39156: URL: https://github.com/apache/airflow/pull/39156 Closes #39149 When creating a container using the AzureContainerInstanceOperator, it would be good to be able to pass a specific dns configuration and log analytics workspace. These are

Re: [PR] add dns_config and diagnostics parameters to AzureContainerInstancesOperator [airflow]

2024-04-21 Thread via GitHub
boring-cyborg[bot] commented on PR #39156: URL: https://github.com/apache/airflow/pull/39156#issuecomment-2068082033 Congratulations on your first Pull Request and welcome to the Apache Airflow community! If you have any issues or are unsure about any anything please check our

Re: [I] Pods are still running even after receiveing SIGTERM Terminating subprocesses [airflow]

2024-04-21 Thread via GitHub
paramjeet01 commented on issue #39096: URL: https://github.com/apache/airflow/issues/39096#issuecomment-2068081401 After debugging for a while , I have also found that one of our scheduler was failed with liveliness probe and restarted at the same time. So , I'm guessing that the scheduler

Re: [PR] Feature/iceberg provider [airflow]

2024-04-21 Thread via GitHub
eladkal commented on PR #39155: URL: https://github.com/apache/airflow/pull/39155#issuecomment-2068057896 @Fokko Do we want to keep Tabular provider (if there might be future tabular specific behavior?) or should we deprecate the whole tabular provider in favor of apache-iceberg provider?

[PR] Feature/iceberg provider [airflow]

2024-04-21 Thread via GitHub
romsharon98 opened a new pull request, #39155: URL: https://github.com/apache/airflow/pull/39155 --- **^ Add meaningful description above** Read the **[Pull Request

Re: [PR] Implement `CloudDataTransferServiceRunJobOperator` [airflow]

2024-04-21 Thread via GitHub
boraberke commented on code in PR #39154: URL: https://github.com/apache/airflow/pull/39154#discussion_r1573738991 ## airflow/providers/google/cloud/hooks/cloud_storage_transfer_service.py: ## @@ -343,6 +343,32 @@ def delete_transfer_job(self, job_name: str, project_id: str)

Re: [I] Google Cloud Transfer operators: add an operator to run a transfer job [airflow]

2024-04-21 Thread via GitHub
boraberke commented on issue #22016: URL: https://github.com/apache/airflow/issues/22016#issuecomment-2068029794 Hi, I have implemented a basic operator named `CloudDataTransferServiceRunJobOperator` in #39154. It might be also useful to add a sensor

Re: [PR] Implement `CloudDataTransferServiceRunJobOperator` [airflow]

2024-04-21 Thread via GitHub
boraberke commented on code in PR #39154: URL: https://github.com/apache/airflow/pull/39154#discussion_r1573734935 ## airflow/providers/google/cloud/hooks/cloud_storage_transfer_service.py: ## @@ -343,6 +343,32 @@ def delete_transfer_job(self, job_name: str, project_id: str)

Re: [PR] Implement `CloudDataTransferServiceRunJobOperator` [airflow]

2024-04-21 Thread via GitHub
eladkal commented on code in PR #39154: URL: https://github.com/apache/airflow/pull/39154#discussion_r1573734490 ## airflow/providers/google/cloud/hooks/cloud_storage_transfer_service.py: ## @@ -343,6 +343,32 @@ def delete_transfer_job(self, job_name: str, project_id: str) ->

[PR] Implement `CloudDataTransferServiceRunJobOperator` [airflow]

2024-04-21 Thread via GitHub
boraberke opened a new pull request, #39154: URL: https://github.com/apache/airflow/pull/39154 This PR adds `CloudDataTransferServiceRunJobOperator` which runs a pre-configured storage data transfer service using it's `job_name`. Closes: #22016 -- This is an automated message from

Re: [I] Many running task instances are cleared by the new scheduler when an old scheduler is terminated and its health check server is periodically requested [airflow]

2024-04-21 Thread via GitHub
paramjeet01 commented on issue #39088: URL: https://github.com/apache/airflow/issues/39088#issuecomment-2068007395 @tanvn @RNHTTR , Could this issue be related to the one I raised : https://github.com/apache/airflow/issues/39096 We are running 8 schedulers and we see this issue

Re: [PR] fix comment about the condition for emitting dagrun.first_task_scheduling_delay [airflow]

2024-04-21 Thread via GitHub
tanvn commented on PR #39153: URL: https://github.com/apache/airflow/pull/39153#issuecomment-2067972894 @syun64 @potiuk @uranusjr Please take a look at your convenience. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and

[PR] fix comment about the condition for emitting dagrun.first_task_scheduling_delay [airflow]

2024-04-21 Thread via GitHub
tanvn opened a new pull request, #39153: URL: https://github.com/apache/airflow/pull/39153 ## Motivations The comment about the condition for emitting dagrun.first_task_scheduling_delay metric is not correct about `clear_number`. This PR fixed the comment.

(airflow-site) branch gh-pages updated (d7b5f62401 -> 6d90437fce)

2024-04-21 Thread github-bot
This is an automated email from the ASF dual-hosted git repository. github-bot pushed a change to branch gh-pages in repository https://gitbox.apache.org/repos/asf/airflow-site.git discard d7b5f62401 Rewritten history to remove past gh-pages deployments new 6d90437fce Rewritten history to

Re: [PR] gitSync: add envFrom to support proxy settings from k8s secret [airflow]

2024-04-21 Thread via GitHub
romsharon98 commented on PR #39031: URL: https://github.com/apache/airflow/pull/39031#issuecomment-2067953964 look good, can you add some tests? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to

Re: [I] Status of testing Providers that were prepared on April 16, 2024 [airflow]

2024-04-21 Thread via GitHub
eladkal closed issue #39063: Status of testing Providers that were prepared on April 16, 2024 URL: https://github.com/apache/airflow/issues/39063 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the

Re: [I] Status of testing Providers that were prepared on April 16, 2024 [airflow]

2024-04-21 Thread via GitHub
eladkal commented on issue #39063: URL: https://github.com/apache/airflow/issues/39063#issuecomment-2067950179 Thank you everyone. Providers are released. I invite everyone to help improve providers for the next release, a list of open issues can be found

Re: [PR] Add metrics_allow_list and metrics_block_list to statsd deployments env [airflow]

2024-04-21 Thread via GitHub
romsharon98 commented on PR #38697: URL: https://github.com/apache/airflow/pull/38697#issuecomment-2067948902 @eladkal looks good, can we merge it? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go

[PR] Update providers metadata 2024-04-21 [airflow]

2024-04-21 Thread via GitHub
eladkal opened a new pull request, #39152: URL: https://github.com/apache/airflow/pull/39152 (no comment) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe,

(airflow) annotated tag providers-fab/1.0.4 updated (13df6569d6 -> 8b0b834c4d)

2024-04-21 Thread eladkal
This is an automated email from the ASF dual-hosted git repository. eladkal pushed a change to annotated tag providers-fab/1.0.4 in repository https://gitbox.apache.org/repos/asf/airflow.git *** WARNING: tag providers-fab/1.0.4 was modified! *** from 13df6569d6 (commit) to 8b0b834c4d

(airflow) annotated tag providers-cncf-kubernetes/8.1.1 updated (13df6569d6 -> 50bacc60e3)

2024-04-21 Thread eladkal
This is an automated email from the ASF dual-hosted git repository. eladkal pushed a change to annotated tag providers-cncf-kubernetes/8.1.1 in repository https://gitbox.apache.org/repos/asf/airflow.git *** WARNING: tag providers-cncf-kubernetes/8.1.1 was modified! *** from 13df6569d6

Re: [PR] Add documentation for packages - 2024-04-16 [airflow-site]

2024-04-21 Thread via GitHub
eladkal merged PR #1007: URL: https://github.com/apache/airflow-site/pull/1007 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

(airflow-site) branch add-documentation-2024-04-16 deleted (was 3c2f9b0b95)

2024-04-21 Thread eladkal
This is an automated email from the ASF dual-hosted git repository. eladkal pushed a change to branch add-documentation-2024-04-16 in repository https://gitbox.apache.org/repos/asf/airflow-site.git was 3c2f9b0b95 Add documentation for packages - 2024-04-16 The revisions that were on this

(airflow) annotated tag providers-databricks/6.3.0 updated (13df6569d6 -> 0c2fed7228)

2024-04-21 Thread eladkal
This is an automated email from the ASF dual-hosted git repository. eladkal pushed a change to annotated tag providers-databricks/6.3.0 in repository https://gitbox.apache.org/repos/asf/airflow.git *** WARNING: tag providers-databricks/6.3.0 was modified! *** from 13df6569d6 (commit)

svn commit: r68683 - /dev/airflow/providers/ /release/airflow/providers/

2024-04-21 Thread eladkal
Author: eladkal Date: Sun Apr 21 07:51:58 2024 New Revision: 68683 Log: Release Airflow Providers on 2024-04-21 Added: release/airflow/providers/apache_airflow_providers_cncf_kubernetes-8.1.1-py3-none-any.whl - copied unchanged from r68682,

[PR] `S3DeleteObjectsOperator` Added ability to filter keys by last modified time [airflow]

2024-04-21 Thread via GitHub
idantepper opened a new pull request, #39151: URL: https://github.com/apache/airflow/pull/39151 expose cabability from the s3 hook to the delete operator --- **^ Add meaningful description above** Read the **[Pull Request

Re: [I] Statsd aggregation issue on some metrics with multiple schedulers running [airflow]

2024-04-21 Thread via GitHub
github-actions[bot] commented on issue #26601: URL: https://github.com/apache/airflow/issues/26601#issuecomment-2067932490 This issue has been closed because it has not received response from the issue author. -- This is an automated message from the Apache Git Service. To respond to the

Re: [I] Statsd aggregation issue on some metrics with multiple schedulers running [airflow]

2024-04-21 Thread via GitHub
github-actions[bot] closed issue #26601: Statsd aggregation issue on some metrics with multiple schedulers running URL: https://github.com/apache/airflow/issues/26601 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the

Re: [I] Not all zombies are caused by missed heartbeat [airflow]

2024-04-21 Thread via GitHub
github-actions[bot] commented on issue #29455: URL: https://github.com/apache/airflow/issues/29455#issuecomment-2067932474 This issue has been automatically marked as stale because it has been open for 365 days without any activity. There has been several Airflow releases since last

Re: [PR] Have `BaseOperator` get DAG from current task group if necessary [airflow]

2024-04-21 Thread via GitHub
eladkal commented on PR #35968: URL: https://github.com/apache/airflow/pull/35968#issuecomment-2067923087 @sean-rose can you fix the tests? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the

(airflow) branch main updated: fix: add retry logic in case of google auth refresh credential error (#38961)

2024-04-21 Thread eladkal
This is an automated email from the ASF dual-hosted git repository. eladkal pushed a commit to branch main in repository https://gitbox.apache.org/repos/asf/airflow.git The following commit(s) were added to refs/heads/main by this push: new 59084fd1f4 fix: add retry logic in case of google

Re: [I] `BigQueryInsertJobOperator` sometimes fails to acquire impersonated credentials when in deferred mode [airflow]

2024-04-21 Thread via GitHub
eladkal closed issue #38532: `BigQueryInsertJobOperator` sometimes fails to acquire impersonated credentials when in deferred mode URL: https://github.com/apache/airflow/issues/38532 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to

Re: [PR] fix: add retry logic in case of google auth refresh credential error [airflow]

2024-04-21 Thread via GitHub
eladkal merged PR #38961: URL: https://github.com/apache/airflow/pull/38961 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

(airflow) branch main updated: Add priorityclassname to jobs (#39133)

2024-04-21 Thread eladkal
This is an automated email from the ASF dual-hosted git repository. eladkal pushed a commit to branch main in repository https://gitbox.apache.org/repos/asf/airflow.git The following commit(s) were added to refs/heads/main by this push: new f31271b497 Add priorityclassname to jobs (#39133)

Re: [PR] Chart: add priorityClassName to jobs [airflow]

2024-04-21 Thread via GitHub
eladkal merged PR #39133: URL: https://github.com/apache/airflow/pull/39133 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: