Re: [PR] Enable none default ftp port [airflow]
kerlion commented on PR #39048: URL: https://github.com/apache/airflow/pull/39048#issuecomment-2068328606 Hi, is this ready for merging, anything else need I to do? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@airflow.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [PR] [FEAT] added notebook error in databricks deferrable handler [airflow]
Lee-W commented on code in PR #39110:
URL: https://github.com/apache/airflow/pull/39110#discussion_r1573991574
##
airflow/providers/databricks/operators/databricks.py:
##
@@ -156,17 +144,37 @@ def
_handle_deferrable_databricks_operator_execution(operator, hook, log, contex
log.info("%s completed successfully.", operator.task_id)
-def _handle_deferrable_databricks_operator_completion(event: dict, log:
Logger) -> None:
+def _get_databricks_notebook_error(run_info: dict, hook: DatabricksHook,
run_state: RunState) -> str:
+task_run_id = None
+if "tasks" in run_info:
+for task in run_info["tasks"]:
+if task.get("state", {}).get("result_state", "") == "FAILED":
+task_run_id = task["run_id"]
+if task_run_id is not None:
+run_output = hook.get_run_output(task_run_id)
+if "error" in run_output:
+notebook_error = run_output["error"]
+else:
+notebook_error = run_state.state_message
+else:
+notebook_error = run_state.state_message
+return notebook_error
+
+
+def _handle_deferrable_databricks_operator_completion(event: dict, log:
Logger, hook: DatabricksHook) -> None:
validate_trigger_event(event)
run_state = RunState.from_json(event["run_state"])
run_page_url = event["run_page_url"]
+run_id = event["run_id"]
log.info("View run status, Spark UI, and logs at %s", run_page_url)
if run_state.is_successful:
log.info("Job run completed successfully.")
return
+run_info = hook.get_run(run_id)
Review Comment:
We can run those logic in
https://github.com/apache/airflow/blob/4374a4987218a05913253b368f45b5fe0ef21493/airflow/providers/databricks/triggers/databricks.py#L83
and then yield the information back to
https://github.com/apache/airflow/blob/4374a4987218a05913253b368f45b5fe0ef21493/airflow/providers/databricks/operators/databricks.py#L118
to handle
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscr...@airflow.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Re: [PR] Don't clear downstream tasks when marked status is failure and downstream is True [airflow]
github-actions[bot] commented on PR #23079: URL: https://github.com/apache/airflow/pull/23079#issuecomment-2068278429 This pull request has been automatically marked as stale because it has not had recent activity. It will be closed in 5 days if no further activity occurs. Thank you for your contributions. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@airflow.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [PR] add dns_config and diagnostics parameters to AzureContainerInstancesOperator [airflow]
eladkal commented on code in PR #39156: URL: https://github.com/apache/airflow/pull/39156#discussion_r1573930933 ## airflow/providers/microsoft/azure/operators/container_instances.py: ## @@ -113,6 +117,13 @@ class AzureContainerInstancesOperator(BaseOperator): memory_in_gb=14.0, cpu=4.0, gpu=GpuResource(count=1, sku="K80"), +dns_config=["10.0.0.10", "10.0.0.11"], Review Comment: Can you add unit test to cover these changes? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@airflow.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [PR] add dns_config and diagnostics parameters to AzureContainerInstancesOperator [airflow]
eladkal commented on code in PR #39156: URL: https://github.com/apache/airflow/pull/39156#discussion_r1573930869 ## airflow/providers/microsoft/azure/provider.yaml: ## @@ -23,6 +23,7 @@ state: ready source-date-epoch: 1709555852 # note that those versions are maintained by release manager - do not update them manually versions: + - 10.0.1 Review Comment: Needs to be removed. Release manager decide on the version -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@airflow.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [PR] add dns_config and diagnostics parameters to AzureContainerInstancesOperator [airflow]
eladkal commented on code in PR #39156: URL: https://github.com/apache/airflow/pull/39156#discussion_r1573930728 ## airflow/providers/microsoft/azure/CHANGELOG.rst: ## Review Comment: Please remove changea to this file. Its not needed for features/bug fixes as it is auto generated by release manager -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@airflow.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [PR] [FEAT] added notebook error in databricks deferrable handler [airflow]
gaurav7261 commented on code in PR #39110:
URL: https://github.com/apache/airflow/pull/39110#discussion_r1573919570
##
airflow/providers/databricks/operators/databricks.py:
##
@@ -156,17 +144,37 @@ def
_handle_deferrable_databricks_operator_execution(operator, hook, log, contex
log.info("%s completed successfully.", operator.task_id)
-def _handle_deferrable_databricks_operator_completion(event: dict, log:
Logger) -> None:
+def _get_databricks_notebook_error(run_info: dict, hook: DatabricksHook,
run_state: RunState) -> str:
+task_run_id = None
+if "tasks" in run_info:
+for task in run_info["tasks"]:
+if task.get("state", {}).get("result_state", "") == "FAILED":
+task_run_id = task["run_id"]
+if task_run_id is not None:
+run_output = hook.get_run_output(task_run_id)
+if "error" in run_output:
+notebook_error = run_output["error"]
+else:
+notebook_error = run_state.state_message
+else:
+notebook_error = run_state.state_message
+return notebook_error
+
+
+def _handle_deferrable_databricks_operator_completion(event: dict, log:
Logger, hook: DatabricksHook) -> None:
validate_trigger_event(event)
run_state = RunState.from_json(event["run_state"])
run_page_url = event["run_page_url"]
+run_id = event["run_id"]
log.info("View run status, Spark UI, and logs at %s", run_page_url)
if run_state.is_successful:
log.info("Job run completed successfully.")
return
+run_info = hook.get_run(run_id)
Review Comment:
Hi @Lee-W when i'm changing to async it demands the entire method
`_handle_deferrable_databricks_operator_completion` to be async in that case,
am I missing something here, can you just guide me a little bit, I will fix this
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscr...@airflow.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
(airflow) branch main updated (59084fd1f4 -> aa5581e30d)
This is an automated email from the ASF dual-hosted git repository. potiuk pushed a change to branch main in repository https://gitbox.apache.org/repos/asf/airflow.git from 59084fd1f4 fix: add retry logic in case of google auth refresh credential error (#38961) add aa5581e30d Update providers metadata 2024-04-21 (#39152) No new revisions were added by this update. Summary of changes: generated/provider_metadata.json | 12 1 file changed, 12 insertions(+)
Re: [PR] Update providers metadata 2024-04-21 [airflow]
potiuk merged PR #39152: URL: https://github.com/apache/airflow/pull/39152 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@airflow.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[PR] add dns_config and diagnostics parameters to AzureContainerInstancesOperator [airflow]
Kylea650 opened a new pull request, #39156: URL: https://github.com/apache/airflow/pull/39156 Closes #39149 When creating a container using the AzureContainerInstanceOperator, it would be good to be able to pass a specific dns configuration and log analytics workspace. These are already supported by the azure management api. Non-breaking change that adds some nice extra functionality. In some cases, providing an explicit dns config is required for private containers when existing VNet DNS is not picked up. (see https://stackoverflow.com/questions/64700687/how-to-get-azure-container-instances-using-my-dns-server). Adding the ability to send logs to an existing log analytics workspace is a nice quality of life addition for sending non-airflow, application specific logs. Happy to add more tests if required. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@airflow.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [PR] add dns_config and diagnostics parameters to AzureContainerInstancesOperator [airflow]
boring-cyborg[bot] commented on PR #39156: URL: https://github.com/apache/airflow/pull/39156#issuecomment-2068082033 Congratulations on your first Pull Request and welcome to the Apache Airflow community! If you have any issues or are unsure about any anything please check our Contributors' Guide (https://github.com/apache/airflow/blob/main/contributing-docs/README.rst) Here are some useful points: - Pay attention to the quality of your code (ruff, mypy and type annotations). Our [pre-commits]( https://github.com/apache/airflow/blob/main/contributing-docs/08_static_code_checks.rst#prerequisites-for-pre-commit-hooks) will help you with that. - In case of a new feature add useful documentation (in docstrings or in `docs/` directory). Adding a new operator? Check this short [guide](https://github.com/apache/airflow/blob/main/docs/apache-airflow/howto/custom-operator.rst) Consider adding an example DAG that shows how users should use it. - Consider using [Breeze environment](https://github.com/apache/airflow/blob/main/dev/breeze/doc/README.rst) for testing locally, it's a heavy docker but it ships with a working Airflow and a lot of integrations. - Be patient and persistent. It might take some time to get a review or get the final approval from Committers. - Please follow [ASF Code of Conduct](https://www.apache.org/foundation/policies/conduct) for all communication including (but not limited to) comments on Pull Requests, Mailing list and Slack. - Be sure to read the [Airflow Coding style]( https://github.com/apache/airflow/blob/main/contributing-docs/05_pull_requests.rst#coding-style-and-best-practices). - Always keep your Pull Requests rebased, otherwise your build might fail due to changes not related to your commits. Apache Airflow is a community-driven project and together we are making it better 🚀. In case of doubts contact the developers at: Mailing List: d...@airflow.apache.org Slack: https://s.apache.org/airflow-slack -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@airflow.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [I] Pods are still running even after receiveing SIGTERM Terminating subprocesses [airflow]
paramjeet01 commented on issue #39096: URL: https://github.com/apache/airflow/issues/39096#issuecomment-2068081401 After debugging for a while , I have also found that one of our scheduler was failed with liveliness probe and restarted at the same time. So , I'm guessing that the scheduler restart caused SIGTERM killing and the task wasn't adopted by any schedulers. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@airflow.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [PR] Feature/iceberg provider [airflow]
eladkal commented on PR #39155: URL: https://github.com/apache/airflow/pull/39155#issuecomment-2068057896 @Fokko Do we want to keep Tabular provider (if there might be future tabular specific behavior?) or should we deprecate the whole tabular provider in favor of apache-iceberg provider? (difference between deprecate specific classes vs the whole provider) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@airflow.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[PR] Feature/iceberg provider [airflow]
romsharon98 opened a new pull request, #39155:
URL: https://github.com/apache/airflow/pull/39155
---
**^ Add meaningful description above**
Read the **[Pull Request
Guidelines](https://github.com/apache/airflow/blob/main/contributing-docs/05_pull_requests.rst#pull-request-guidelines)**
for more information.
In case of fundamental code changes, an Airflow Improvement Proposal
([AIP](https://cwiki.apache.org/confluence/display/AIRFLOW/Airflow+Improvement+Proposals))
is needed.
In case of a new dependency, check compliance with the [ASF 3rd Party
License Policy](https://www.apache.org/legal/resolved.html#category-x).
In case of backwards incompatible changes please leave a note in a
newsfragment file, named `{pr_number}.significant.rst` or
`{issue_number}.significant.rst`, in
[newsfragments](https://github.com/apache/airflow/tree/main/newsfragments).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscr...@airflow.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Re: [PR] Implement `CloudDataTransferServiceRunJobOperator` [airflow]
boraberke commented on code in PR #39154:
URL: https://github.com/apache/airflow/pull/39154#discussion_r1573738991
##
airflow/providers/google/cloud/hooks/cloud_storage_transfer_service.py:
##
@@ -343,6 +343,32 @@ def delete_transfer_job(self, job_name: str, project_id:
str) -> None:
.execute(num_retries=self.num_retries)
)
+@GoogleBaseHook.fallback_to_default_project_id
+def run_transfer_job(self, job_name: str, project_id: str) -> dict:
+"""Run Google Storage Transfer Service job.
+
+:param job_name: (Required) Name of the job to be fetched
+:param project_id: (Optional) the ID of the project that owns the
Transfer
+Job. If set to None or missing, the default project_id from the
Google Cloud
+connection is used.
+:return: If successful, Operation. See:
+
https://cloud.google.com/storage-transfer/docs/reference/rest/v1/Operation
+
+.. seealso::
https://cloud.google.com/storage-transfer/docs/reference/rest/v1/transferJobs/run
+
+"""
+return (
+self.get_conn()
+.transferJobs()
+.run(
+jobName=job_name,
+body={
+PROJECT_ID: project_id,
+},
+)
+.execute(num_retries=self.num_retries)
+)
Review Comment:
now added a unit test for `run_transfer_job` similar to the others.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscr...@airflow.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Re: [I] Google Cloud Transfer operators: add an operator to run a transfer job [airflow]
boraberke commented on issue #22016: URL: https://github.com/apache/airflow/issues/22016#issuecomment-2068029794 Hi, I have implemented a basic operator named `CloudDataTransferServiceRunJobOperator` in #39154. It might be also useful to add a sensor `CloudDataTransferServiceRunJobSensor` similar to [BigQueryDataTransferServiceTransferRunSensor](https://github.com/apache/airflow/blob/59084fd1f4c200986433f9ff60b28cd6f8a0bcc1/airflow/providers/google/cloud/sensors/bigquery_dts.py#L38) so that this job can be polled after triggered. What do you think? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@airflow.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [PR] Implement `CloudDataTransferServiceRunJobOperator` [airflow]
boraberke commented on code in PR #39154:
URL: https://github.com/apache/airflow/pull/39154#discussion_r1573734935
##
airflow/providers/google/cloud/hooks/cloud_storage_transfer_service.py:
##
@@ -343,6 +343,32 @@ def delete_transfer_job(self, job_name: str, project_id:
str) -> None:
.execute(num_retries=self.num_retries)
)
+@GoogleBaseHook.fallback_to_default_project_id
+def run_transfer_job(self, job_name: str, project_id: str) -> dict:
+"""Run Google Storage Transfer Service job.
+
+:param job_name: (Required) Name of the job to be fetched
+:param project_id: (Optional) the ID of the project that owns the
Transfer
+Job. If set to None or missing, the default project_id from the
Google Cloud
+connection is used.
+:return: If successful, Operation. See:
+
https://cloud.google.com/storage-transfer/docs/reference/rest/v1/Operation
+
+.. seealso::
https://cloud.google.com/storage-transfer/docs/reference/rest/v1/transferJobs/run
+
+"""
+return (
+self.get_conn()
+.transferJobs()
+.run(
+jobName=job_name,
+body={
+PROJECT_ID: project_id,
+},
+)
+.execute(num_retries=self.num_retries)
+)
Review Comment:
sure, working on it now!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscr...@airflow.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Re: [PR] Implement `CloudDataTransferServiceRunJobOperator` [airflow]
eladkal commented on code in PR #39154:
URL: https://github.com/apache/airflow/pull/39154#discussion_r1573734490
##
airflow/providers/google/cloud/hooks/cloud_storage_transfer_service.py:
##
@@ -343,6 +343,32 @@ def delete_transfer_job(self, job_name: str, project_id:
str) -> None:
.execute(num_retries=self.num_retries)
)
+@GoogleBaseHook.fallback_to_default_project_id
+def run_transfer_job(self, job_name: str, project_id: str) -> dict:
+"""Run Google Storage Transfer Service job.
+
+:param job_name: (Required) Name of the job to be fetched
+:param project_id: (Optional) the ID of the project that owns the
Transfer
+Job. If set to None or missing, the default project_id from the
Google Cloud
+connection is used.
+:return: If successful, Operation. See:
+
https://cloud.google.com/storage-transfer/docs/reference/rest/v1/Operation
+
+.. seealso::
https://cloud.google.com/storage-transfer/docs/reference/rest/v1/transferJobs/run
+
+"""
+return (
+self.get_conn()
+.transferJobs()
+.run(
+jobName=job_name,
+body={
+PROJECT_ID: project_id,
+},
+)
+.execute(num_retries=self.num_retries)
+)
Review Comment:
can you add unit test to cover this new function?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscr...@airflow.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[PR] Implement `CloudDataTransferServiceRunJobOperator` [airflow]
boraberke opened a new pull request, #39154: URL: https://github.com/apache/airflow/pull/39154 This PR adds `CloudDataTransferServiceRunJobOperator` which runs a pre-configured storage data transfer service using it's `job_name`. Closes: #22016 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@airflow.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [I] Many running task instances are cleared by the new scheduler when an old scheduler is terminated and its health check server is periodically requested [airflow]
paramjeet01 commented on issue #39088: URL: https://github.com/apache/airflow/issues/39088#issuecomment-2068007395 @tanvn @RNHTTR , Could this issue be related to the one I raised : https://github.com/apache/airflow/issues/39096 We are running 8 schedulers and we see this issue happening often , where the task is killed by SIGTERM and the pod is still running. When the task is retried again , we have error log stating the labels already exists and we see the pod is still running. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@airflow.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [PR] fix comment about the condition for emitting dagrun.first_task_scheduling_delay [airflow]
tanvn commented on PR #39153: URL: https://github.com/apache/airflow/pull/39153#issuecomment-2067972894 @syun64 @potiuk @uranusjr Please take a look at your convenience. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@airflow.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[PR] fix comment about the condition for emitting dagrun.first_task_scheduling_delay [airflow]
tanvn opened a new pull request, #39153:
URL: https://github.com/apache/airflow/pull/39153
## Motivations
The comment about the condition for emitting
dagrun.first_task_scheduling_delay metric is not correct about `clear_number`.
This PR fixed the comment.
---
**^ Add meaningful description above**
Read the **[Pull Request
Guidelines](https://github.com/apache/airflow/blob/main/contributing-docs/05_pull_requests.rst#pull-request-guidelines)**
for more information.
In case of fundamental code changes, an Airflow Improvement Proposal
([AIP](https://cwiki.apache.org/confluence/display/AIRFLOW/Airflow+Improvement+Proposals))
is needed.
In case of a new dependency, check compliance with the [ASF 3rd Party
License Policy](https://www.apache.org/legal/resolved.html#category-x).
In case of backwards incompatible changes please leave a note in a
newsfragment file, named `{pr_number}.significant.rst` or
`{issue_number}.significant.rst`, in
[newsfragments](https://github.com/apache/airflow/tree/main/newsfragments).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscr...@airflow.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
(airflow-site) branch gh-pages updated (d7b5f62401 -> 6d90437fce)
This is an automated email from the ASF dual-hosted git repository.
github-bot pushed a change to branch gh-pages
in repository https://gitbox.apache.org/repos/asf/airflow-site.git
discard d7b5f62401 Rewritten history to remove past gh-pages deployments
new 6d90437fce Rewritten history to remove past gh-pages deployments
This update added new revisions after undoing existing revisions.
That is to say, some revisions that were in the old version of the
branch are not in the new version. This situation occurs
when a user --force pushes a change and generates a repository
containing something like this:
* -- * -- B -- O -- O -- O (d7b5f62401)
\
N -- N -- N refs/heads/gh-pages (6d90437fce)
You should already have received notification emails for all of the O
revisions, and so the following emails describe only the N revisions
from the common base, B.
Any revisions marked "omit" are not gone; other references still
refer to them. Any revisions marked "discard" are gone forever.
The 1 revisions listed above as "new" are entirely new to this
repository and will be described in separate emails. The revisions
listed as "add" were already present in the repository and have only
been added to this reference.
Summary of changes:
_gen/packages-metadata.json| 15 +-
blog/airflow-1.10.10/index.html| 4 +-
blog/airflow-1.10.12/index.html| 4 +-
blog/airflow-1.10.8-1.10.9/index.html | 4 +-
blog/airflow-2.2.0/index.html | 4 +-
blog/airflow-2.3.0/index.html | 4 +-
blog/airflow-2.4.0/index.html | 4 +-
blog/airflow-2.5.0/index.html | 4 +-
blog/airflow-2.6.0/index.html | 4 +-
blog/airflow-2.7.0/index.html | 4 +-
blog/airflow-2.8.0/index.html | 4 +-
blog/airflow-2.9.0/index.html | 4 +-
blog/airflow-survey-2020/index.html| 4 +-
blog/airflow-survey-2022/index.html| 4 +-
blog/airflow-survey/index.html | 4 +-
blog/airflow-two-point-oh-is-here/index.html | 4 +-
blog/airflow_summit_2021/index.html| 4 +-
blog/airflow_summit_2022/index.html| 4 +-
blog/announcing-new-website/index.html | 4 +-
blog/apache-airflow-for-newcomers/index.html | 4 +-
.../index.html | 4 +-
.../index.html | 4 +-
.../index.html | 4 +-
.../index.html | 4 +-
blog/fab-oid-vulnerability/index.html | 4 +-
.../index.html | 4 +-
blog/introducing_setup_teardown/index.html | 4 +-
.../index.html | 4 +-
.../{8.1.0 => 8.1.1}/.buildinfo| 0
.../providers/cncf/kubernetes/callbacks/index.html | 8 +-
.../cncf/kubernetes/decorators/index.html | 8 +-
.../kubernetes/decorators/kubernetes/index.html| 8 +-
.../providers/cncf/kubernetes/executors/index.html | 8 +-
.../executors/kubernetes_executor/index.html | 8 +-
.../executors/kubernetes_executor_types/index.html | 8 +-
.../executors/kubernetes_executor_utils/index.html | 8 +-
.../executors/local_kubernetes_executor/index.html | 8 +-
.../providers/cncf/kubernetes/hooks/index.html | 8 +-
.../cncf/kubernetes/hooks/kubernetes/index.html| 8 +-
.../airflow/providers/cncf/kubernetes/index.html | 10 +-
.../providers/cncf/kubernetes/k8s_model/index.html | 8 +-
.../cncf/kubernetes/kube_client/index.html | 8 +-
.../cncf/kubernetes/kube_config/index.html | 8 +-
.../kubernetes_executor_templates/index.html | 8 +-
.../kubernetes_helper_functions/index.html | 8 +-
.../operators/custom_object_launcher/index.html| 8 +-
.../providers/cncf/kubernetes/operators/index.html | 8 +-
.../cncf/kubernetes/operators/job/index.html | 10 +-
.../kubernetes/operators/kubernetes_pod/index.html | 8 +-
.../cncf/kubernetes/operators/pod/index.html | 10 +-
.../cncf/kubernetes/operators/resource/index.html | 8 +-
.../operators/spark_kubernetes/index.html | 8 +-
.../cncf/kubernetes/pod_generator/index.html | 8 +-
.../kubernetes/pod_generator_deprecated/index.html | 8 +-
.../kubernetes/pod_launcher_deprecated/index.html | 8 +-
.../pod_template_file_examples/index.html | 8 +-
.../kubernetes/python_kubernetes_script/index.html | 8 +-
.../resource_convert/configmap/index.html | 8 +-
.../resource_convert/env_variable/index.html | 8 +-
.../cncf/kubernetes/resource_convert/index.html| 8 +-
.../kubernetes/resource_convert/secret/index.html |
Re: [PR] gitSync: add envFrom to support proxy settings from k8s secret [airflow]
romsharon98 commented on PR #39031: URL: https://github.com/apache/airflow/pull/39031#issuecomment-2067953964 look good, can you add some tests? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@airflow.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [I] Status of testing Providers that were prepared on April 16, 2024 [airflow]
eladkal closed issue #39063: Status of testing Providers that were prepared on April 16, 2024 URL: https://github.com/apache/airflow/issues/39063 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@airflow.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [I] Status of testing Providers that were prepared on April 16, 2024 [airflow]
eladkal commented on issue #39063: URL: https://github.com/apache/airflow/issues/39063#issuecomment-2067950179 Thank you everyone. Providers are released. I invite everyone to help improve providers for the next release, a list of open issues can be found [here](https://github.com/apache/airflow/issues?q=is%3Aopen+is%3Aissue+label%3Aarea%3Aproviders). -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@airflow.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [PR] Add metrics_allow_list and metrics_block_list to statsd deployments env [airflow]
romsharon98 commented on PR #38697: URL: https://github.com/apache/airflow/pull/38697#issuecomment-2067948902 @eladkal looks good, can we merge it? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@airflow.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[PR] Update providers metadata 2024-04-21 [airflow]
eladkal opened a new pull request, #39152: URL: https://github.com/apache/airflow/pull/39152 (no comment) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@airflow.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
(airflow) annotated tag providers-fab/1.0.4 updated (13df6569d6 -> 8b0b834c4d)
This is an automated email from the ASF dual-hosted git repository. eladkal pushed a change to annotated tag providers-fab/1.0.4 in repository https://gitbox.apache.org/repos/asf/airflow.git *** WARNING: tag providers-fab/1.0.4 was modified! *** from 13df6569d6 (commit) to 8b0b834c4d (tag) tagging 13df6569d6cc131fbf096cedd46dc32b0a6cf6b2 (commit) replaces providers-yandex/3.10.0 by Elad Kalif on Sun Apr 21 10:55:14 2024 +0300 - Log - Release 2024-04-21 of providers --- No new revisions were added by this update. Summary of changes:
(airflow) annotated tag providers-cncf-kubernetes/8.1.1 updated (13df6569d6 -> 50bacc60e3)
This is an automated email from the ASF dual-hosted git repository. eladkal pushed a change to annotated tag providers-cncf-kubernetes/8.1.1 in repository https://gitbox.apache.org/repos/asf/airflow.git *** WARNING: tag providers-cncf-kubernetes/8.1.1 was modified! *** from 13df6569d6 (commit) to 50bacc60e3 (tag) tagging 13df6569d6cc131fbf096cedd46dc32b0a6cf6b2 (commit) replaces providers-yandex/3.10.0 by Elad Kalif on Sun Apr 21 10:55:14 2024 +0300 - Log - Release 2024-04-21 of providers --- No new revisions were added by this update. Summary of changes:
Re: [PR] Add documentation for packages - 2024-04-16 [airflow-site]
eladkal merged PR #1007: URL: https://github.com/apache/airflow-site/pull/1007 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@airflow.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
(airflow-site) branch add-documentation-2024-04-16 deleted (was 3c2f9b0b95)
This is an automated email from the ASF dual-hosted git repository. eladkal pushed a change to branch add-documentation-2024-04-16 in repository https://gitbox.apache.org/repos/asf/airflow-site.git was 3c2f9b0b95 Add documentation for packages - 2024-04-16 The revisions that were on this branch are still contained in other references; therefore, this change does not discard any commits from the repository.
(airflow) annotated tag providers-databricks/6.3.0 updated (13df6569d6 -> 0c2fed7228)
This is an automated email from the ASF dual-hosted git repository. eladkal pushed a change to annotated tag providers-databricks/6.3.0 in repository https://gitbox.apache.org/repos/asf/airflow.git *** WARNING: tag providers-databricks/6.3.0 was modified! *** from 13df6569d6 (commit) to 0c2fed7228 (tag) tagging 13df6569d6cc131fbf096cedd46dc32b0a6cf6b2 (commit) replaces providers-yandex/3.10.0 by Elad Kalif on Sun Apr 21 10:55:14 2024 +0300 - Log - Release 2024-04-21 of providers --- No new revisions were added by this update. Summary of changes:
svn commit: r68683 - /dev/airflow/providers/ /release/airflow/providers/
Author: eladkal Date: Sun Apr 21 07:51:58 2024 New Revision: 68683 Log: Release Airflow Providers on 2024-04-21 Added: release/airflow/providers/apache_airflow_providers_cncf_kubernetes-8.1.1-py3-none-any.whl - copied unchanged from r68682, dev/airflow/providers/apache_airflow_providers_cncf_kubernetes-8.1.1-py3-none-any.whl release/airflow/providers/apache_airflow_providers_cncf_kubernetes-8.1.1-py3-none-any.whl.asc - copied unchanged from r68682, dev/airflow/providers/apache_airflow_providers_cncf_kubernetes-8.1.1-py3-none-any.whl.asc release/airflow/providers/apache_airflow_providers_cncf_kubernetes-8.1.1-py3-none-any.whl.sha512 - copied unchanged from r68682, dev/airflow/providers/apache_airflow_providers_cncf_kubernetes-8.1.1-py3-none-any.whl.sha512 release/airflow/providers/apache_airflow_providers_cncf_kubernetes-8.1.1.tar.gz - copied unchanged from r68682, dev/airflow/providers/apache_airflow_providers_cncf_kubernetes-8.1.1.tar.gz release/airflow/providers/apache_airflow_providers_cncf_kubernetes-8.1.1.tar.gz.asc - copied unchanged from r68682, dev/airflow/providers/apache_airflow_providers_cncf_kubernetes-8.1.1.tar.gz.asc release/airflow/providers/apache_airflow_providers_cncf_kubernetes-8.1.1.tar.gz.sha512 - copied unchanged from r68682, dev/airflow/providers/apache_airflow_providers_cncf_kubernetes-8.1.1.tar.gz.sha512 release/airflow/providers/apache_airflow_providers_databricks-6.3.0-py3-none-any.whl - copied unchanged from r68682, dev/airflow/providers/apache_airflow_providers_databricks-6.3.0-py3-none-any.whl release/airflow/providers/apache_airflow_providers_databricks-6.3.0-py3-none-any.whl.asc - copied unchanged from r68682, dev/airflow/providers/apache_airflow_providers_databricks-6.3.0-py3-none-any.whl.asc release/airflow/providers/apache_airflow_providers_databricks-6.3.0-py3-none-any.whl.sha512 - copied unchanged from r68682, dev/airflow/providers/apache_airflow_providers_databricks-6.3.0-py3-none-any.whl.sha512 release/airflow/providers/apache_airflow_providers_databricks-6.3.0.tar.gz - copied unchanged from r68682, dev/airflow/providers/apache_airflow_providers_databricks-6.3.0.tar.gz release/airflow/providers/apache_airflow_providers_databricks-6.3.0.tar.gz.asc - copied unchanged from r68682, dev/airflow/providers/apache_airflow_providers_databricks-6.3.0.tar.gz.asc release/airflow/providers/apache_airflow_providers_databricks-6.3.0.tar.gz.sha512 - copied unchanged from r68682, dev/airflow/providers/apache_airflow_providers_databricks-6.3.0.tar.gz.sha512 release/airflow/providers/apache_airflow_providers_fab-1.0.4-py3-none-any.whl - copied unchanged from r68682, dev/airflow/providers/apache_airflow_providers_fab-1.0.4-py3-none-any.whl release/airflow/providers/apache_airflow_providers_fab-1.0.4-py3-none-any.whl.asc - copied unchanged from r68682, dev/airflow/providers/apache_airflow_providers_fab-1.0.4-py3-none-any.whl.asc release/airflow/providers/apache_airflow_providers_fab-1.0.4-py3-none-any.whl.sha512 - copied unchanged from r68682, dev/airflow/providers/apache_airflow_providers_fab-1.0.4-py3-none-any.whl.sha512 release/airflow/providers/apache_airflow_providers_fab-1.0.4.tar.gz - copied unchanged from r68682, dev/airflow/providers/apache_airflow_providers_fab-1.0.4.tar.gz release/airflow/providers/apache_airflow_providers_fab-1.0.4.tar.gz.asc - copied unchanged from r68682, dev/airflow/providers/apache_airflow_providers_fab-1.0.4.tar.gz.asc release/airflow/providers/apache_airflow_providers_fab-1.0.4.tar.gz.sha512 - copied unchanged from r68682, dev/airflow/providers/apache_airflow_providers_fab-1.0.4.tar.gz.sha512 Removed: dev/airflow/providers/apache_airflow_providers_cncf_kubernetes-8.1.1-py3-none-any.whl dev/airflow/providers/apache_airflow_providers_cncf_kubernetes-8.1.1-py3-none-any.whl.asc dev/airflow/providers/apache_airflow_providers_cncf_kubernetes-8.1.1-py3-none-any.whl.sha512 dev/airflow/providers/apache_airflow_providers_cncf_kubernetes-8.1.1.tar.gz dev/airflow/providers/apache_airflow_providers_cncf_kubernetes-8.1.1.tar.gz.asc dev/airflow/providers/apache_airflow_providers_cncf_kubernetes-8.1.1.tar.gz.sha512 dev/airflow/providers/apache_airflow_providers_databricks-6.3.0-py3-none-any.whl dev/airflow/providers/apache_airflow_providers_databricks-6.3.0-py3-none-any.whl.asc dev/airflow/providers/apache_airflow_providers_databricks-6.3.0-py3-none-any.whl.sha512 dev/airflow/providers/apache_airflow_providers_databricks-6.3.0.tar.gz dev/airflow/providers/apache_airflow_providers_databricks-6.3.0.tar.gz.asc dev/airflow/providers/apache_airflow_providers_databricks-6.3.0.tar.gz.sha512 dev/airflow/providers/apache_airflow_providers_fab-1.0.4-py3-none-any.whl dev/airflow/providers/apache_airflow_pr
[PR] `S3DeleteObjectsOperator` Added ability to filter keys by last modified time [airflow]
idantepper opened a new pull request, #39151:
URL: https://github.com/apache/airflow/pull/39151
expose cabability from the s3 hook to the delete operator
---
**^ Add meaningful description above**
Read the **[Pull Request
Guidelines](https://github.com/apache/airflow/blob/main/contributing-docs/05_pull_requests.rst#pull-request-guidelines)**
for more information.
In case of fundamental code changes, an Airflow Improvement Proposal
([AIP](https://cwiki.apache.org/confluence/display/AIRFLOW/Airflow+Improvement+Proposals))
is needed.
In case of a new dependency, check compliance with the [ASF 3rd Party
License Policy](https://www.apache.org/legal/resolved.html#category-x).
In case of backwards incompatible changes please leave a note in a
newsfragment file, named `{pr_number}.significant.rst` or
`{issue_number}.significant.rst`, in
[newsfragments](https://github.com/apache/airflow/tree/main/newsfragments).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscr...@airflow.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Re: [I] Statsd aggregation issue on some metrics with multiple schedulers running [airflow]
github-actions[bot] commented on issue #26601: URL: https://github.com/apache/airflow/issues/26601#issuecomment-2067932490 This issue has been closed because it has not received response from the issue author. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@airflow.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [I] Statsd aggregation issue on some metrics with multiple schedulers running [airflow]
github-actions[bot] closed issue #26601: Statsd aggregation issue on some metrics with multiple schedulers running URL: https://github.com/apache/airflow/issues/26601 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@airflow.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [I] Not all zombies are caused by missed heartbeat [airflow]
github-actions[bot] commented on issue #29455: URL: https://github.com/apache/airflow/issues/29455#issuecomment-2067932474 This issue has been automatically marked as stale because it has been open for 365 days without any activity. There has been several Airflow releases since last activity on this issue. Kindly asking to recheck the report against latest Airflow version and let us know if the issue is reproducible. The issue will be closed in next 30 days if no further activity occurs from the issue author. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@airflow.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
