This is an automated email from the ASF dual-hosted git repository.
potiuk pushed a commit to branch v3-0-test
in repository https://gitbox.apache.org/repos/asf/airflow.git
The following commit(s) were added to refs/heads/v3-0-test by this push:
new 3515299543e [v3-0-test] Lower bind pyspark and pydruid to relatively
new versions (#50205) (#50207)
3515299543e is described below
commit 3515299543e8fc752920d0574858b3f11a6af840
Author: github-actions[bot]
<41898282+github-actions[bot]@users.noreply.github.com>
AuthorDate: Mon May 5 16:42:35 2025 +0200
[v3-0-test] Lower bind pyspark and pydruid to relatively new versions
(#50205) (#50207)
PySpark and PyDruid lower-binding we had was limiting both to
versions that were as old as 2018 and sometimes it cause that
those really old versions have been downloaded and installed by
UV during installation - in order to perform dependency resolution.
Those PyDruid and PySpark versions were sometimes causing
failures / broken pipe when downloading them.
Bumping lower limits to relatively modern (2023) versions should
solve the problem.
(cherry picked from commit 8a5effd9ffd242035a3f277db4a23a64190cf334)
Co-authored-by: Jarek Potiuk <[email protected]>
---
providers/apache/druid/pyproject.toml | 2 +-
providers/apache/spark/pyproject.toml | 2 +-
2 files changed, 2 insertions(+), 2 deletions(-)
diff --git a/providers/apache/druid/pyproject.toml
b/providers/apache/druid/pyproject.toml
index 87cedc07f2a..2e3c911e979 100644
--- a/providers/apache/druid/pyproject.toml
+++ b/providers/apache/druid/pyproject.toml
@@ -59,7 +59,7 @@ requires-python = "~=3.9"
dependencies = [
"apache-airflow>=2.9.0",
"apache-airflow-providers-common-sql>=1.26.0",
- "pydruid>=0.4.1",
+ "pydruid>=0.6.6",
]
# The optional dependencies should be modified in place in the generated file
diff --git a/providers/apache/spark/pyproject.toml
b/providers/apache/spark/pyproject.toml
index 446a7fab53c..29a1ba70663 100644
--- a/providers/apache/spark/pyproject.toml
+++ b/providers/apache/spark/pyproject.toml
@@ -59,7 +59,7 @@ requires-python = "~=3.9"
dependencies = [
"apache-airflow>=2.9.0",
"apache-airflow-providers-common-compat>=1.5.0",
- "pyspark>=3.1.3",
+ "pyspark>=3.4.0",
"grpcio-status>=1.59.0",
]
