This is an automated email from the ASF dual-hosted git repository.
potiuk pushed a commit to branch main
in repository https://gitbox.apache.org/repos/asf/airflow.git
The following commit(s) were added to refs/heads/main by this push:
new 92a82eb9b8 Consistent python version checks and troubleshooting
(#42944)
92a82eb9b8 is described below
commit 92a82eb9b8f33abd91e2eadd798b79433fcfb495
Author: Jarek Potiuk <[email protected]>
AuthorDate: Fri Oct 11 21:22:05 2024 +0200
Consistent python version checks and troubleshooting (#42944)
Follow up after #42766 and #42936.
* We do not have to check for minimum Python version for Python 3.9
any more (as we do not support 3.8 any more)
* While Python 3.12 is not yet fully supported by Apache Beam, we
should still not allow it for releasing providers, but all other
commands should support Python 3.12
* When you had pre-commit installed with Python 3.8 before, various
errors might appear when running pre-commit. Troubleshooting
was added quoting the errors and explaining what to do.
---
dev/breeze/doc/04_troubleshooting.rst | 32 ++++++++++++++++++++++
.../commands/release_management_commands.py | 2 +-
.../src/airflow_breeze/utils/python_versions.py | 12 ++++----
.../airflow/providers/apache/beam/provider.yaml | 5 +++-
4 files changed, 42 insertions(+), 9 deletions(-)
diff --git a/dev/breeze/doc/04_troubleshooting.rst
b/dev/breeze/doc/04_troubleshooting.rst
index fd0b1dfa40..fd5f92b03e 100644
--- a/dev/breeze/doc/04_troubleshooting.rst
+++ b/dev/breeze/doc/04_troubleshooting.rst
@@ -72,6 +72,38 @@ describe your problem.
stated in `This comment
<https://github.com/moby/moby/issues/43361#issuecomment-1227617516>`_ and
allows to
run Breeze with no problems.
+Cannot import name 'cache' or Python >=3.9 required
+---------------------------------------------------
+
+When you see this error:
+
+.. code-block::
+
+ ImportError: cannot import name 'cache' from 'functools'
(/Users/jarek/Library/Application
Support/hatch/pythons/3.8/python/lib/python3.8/functools.py)
+
+or
+
+.. code-block::
+
+ ERROR: Package 'blacken-docs' requires a different Python: 3.8.18 not in
'>=3.9'
+
+
+It means that your pre-commit hook is installed with (already End-Of-Life)
Python 3.8 and you should reinstall
+it and clean pre-commit cache.
+
+This can be done (if you use ``pipx`` to install ``pre-commit``):
+
+.. code-block:: bash
+
+ pipx uninstall pre-commit
+ pipx install pre-commit --python $(which python3.9) --force
+ pre-commit clean
+ pre-commit install
+
+If you installed ``pre-commit`` differently, you should remove and reinstall
+it (and clean cache) in the way you installed it.
+
+
Bad Interpreter Error
---------------------
diff --git
a/dev/breeze/src/airflow_breeze/commands/release_management_commands.py
b/dev/breeze/src/airflow_breeze/commands/release_management_commands.py
index b0a1ca2877..95ecf30144 100644
--- a/dev/breeze/src/airflow_breeze/commands/release_management_commands.py
+++ b/dev/breeze/src/airflow_breeze/commands/release_management_commands.py
@@ -797,7 +797,7 @@ def prepare_provider_packages(
skip_tag_check: bool,
version_suffix_for_pypi: str,
):
- check_python_version()
+ check_python_version(release_provider_packages=True)
perform_environment_checks()
fix_ownership_using_docker()
cleanup_python_generated_files()
diff --git a/dev/breeze/src/airflow_breeze/utils/python_versions.py
b/dev/breeze/src/airflow_breeze/utils/python_versions.py
index c7d514f5ff..d144139b06 100644
--- a/dev/breeze/src/airflow_breeze/utils/python_versions.py
+++ b/dev/breeze/src/airflow_breeze/utils/python_versions.py
@@ -45,14 +45,12 @@ def get_python_version_list(python_versions: str) ->
list[str]:
return python_version_list
-def check_python_version():
- error = False
- if not sys.version_info >= (3, 9):
- get_console().print("[error]At least Python 3.9 is required to prepare
reproducible archives.\n")
- error = True
- if error:
+def check_python_version(release_provider_packages: bool = False):
+ if not sys.version_info < (3, 12) and release_provider_packages:
+ get_console().print("[error]Python 3.12 is not supported.\n")
get_console().print(
- "[warning]Please reinstall Breeze using Python 3.9 - 3.11
environment.[/]\n\n"
+ "[warning]Please reinstall Breeze using Python 3.9 - 3.11
environment because not all "
+ "provider packages support Python 3.12 yet.[/]\n\n"
"For example:\n\n"
"pipx uninstall apache-airflow-breeze\n"
"pipx install --python $(which python3.9) -e ./dev/breeze
--force\n"
diff --git a/providers/src/airflow/providers/apache/beam/provider.yaml
b/providers/src/airflow/providers/apache/beam/provider.yaml
index ae0c4d3700..684882f75b 100644
--- a/providers/src/airflow/providers/apache/beam/provider.yaml
+++ b/providers/src/airflow/providers/apache/beam/provider.yaml
@@ -72,7 +72,10 @@ additional-extras:
- apache-beam[gcp]
# Apache Beam currently does not support Python 3.12
-# There is an issue tracking it https://github.com/apache/beam/issues/29149
+# There is an issue tracking it https://github.com/apache/beam/issues/29149.
+# While the original issue above is closed, Apache Beam still does not support
Python 3.12
+# because the dill version used by them break our PythonVirtualenvOperator
when dill is enabled
+# See https://github.com/apache/beam/issues/32617
excluded-python-versions: ['3.12']
integrations: