Taragolis commented on code in PR #37444:
URL: https://github.com/apache/airflow/pull/37444#discussion_r1491043471
##########
dev/breeze/src/airflow_breeze/commands/release_management_commands.py:
##########
@@ -374,6 +404,58 @@ def _build_airflow_packages_with_hatch(
)
+def _dist_packages(
+ *, package_format: str, build_type: Literal["airflow", "providers"]
+) -> Iterator[DistributionPackageInfo]:
+ if build_type == "airflow":
+ default_glob_pattern = "apache[_-]airflow-[0-9]"
+ else:
+ default_glob_pattern = "apache[_-]airflow[_-]providers"
+
+ if package_format in ["sdist", "both"]:
+ for file in DIST_DIR.glob(f"{default_glob_pattern}*tar.gz"):
+ if not file.is_file():
+ continue
+ yield DistributionPackageInfo.from_sdist(filepath=file)
+ if package_format in ["wheel", "both"]:
+ for file in DIST_DIR.glob(f"{default_glob_pattern}*whl"):
+ if not file.is_file():
+ continue
+ yield DistributionPackageInfo.from_wheel(filepath=file)
+
+
+def _check_sdist_to_wheel(dist_info: DistributionPackageInfo):
+ if dist_info.dist_type != "sdist":
+ return
+
+ with tempfile.TemporaryDirectory() as tmp_dir_name:
+ venv_path = Path(tmp_dir_name) / ".venv"
+ venv.EnvBuilder(with_pip=True).create(venv_path)
+ python_path = venv_path / "bin" / "python"
+ if not python_path.exists():
+ msg = f"Python interpreter is not exist in path {python_path}"
+ raise FileNotFoundError(msg)
Review Comment:
I'm not sure should we create venv from the `run_command` or via
[`venv`](https://docs.python.org/3/library/venv.html) module.
Technically `venv` could be missing in some distributions, in this case
entire `release_management_commands` would fail on import statement.
##########
dev/breeze/src/airflow_breeze/commands/release_management_commands.py:
##########
@@ -711,8 +804,8 @@ def prepare_provider_packages(
sys.exit(0)
get_console().print("\n[success]Successfully built packages!\n\n")
get_console().print("\n[info]Packages available in dist:\n")
- for file in sorted(DIST_DIR.glob("apache*")):
- get_console().print(file.name)
+ for dist_info in _dist_packages(package_format=package_format,
build_type="providers"):
+ get_console().print(str(dist_info))
Review Comment:
For providers I do not add this check but maybe I should?
In case of `apache-airflow` it doesn't big overhead, build package from
sources take for about 1m - 1m 20s, and additional 10 seconds in negligible in
the other hand create new venv for each provider package might add huge
overhead in total.
##########
dev/breeze/src/airflow_breeze/commands/release_management_commands.py:
##########
@@ -406,8 +488,19 @@ def prepare_airflow_packages(
version_suffix_for_pypi=version_suffix_for_pypi,
)
get_console().print("[success]Successfully prepared Airflow packages:")
- for file in sorted(DIST_DIR.glob("apache_airflow*")):
- get_console().print(file.name)
+ packages = tuple(_dist_packages(package_format=package_format,
build_type="airflow"))
+ for dist_info in packages:
+ get_console().print(str(dist_info))
+ get_console().print()
+ if package_format == "wheel":
+ return
+
+ for dist_info in packages:
+ if dist_info.dist_type == "sdist":
+ get_console().print(
+ f"[info]Validate build wheel from sdist distribution:
{dist_info.filepath.name!r}[/]"
+ )
+ _check_sdist_to_wheel(dist_info)
Review Comment:
This check run pretty fast, for about 5-15 seconds, but I've also have a
filling that this check might be optional
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
