kacpermuda commented on code in PR #39609:
URL: https://github.com/apache/airflow/pull/39609#discussion_r1599739725
##########
airflow/settings.py:
##########
@@ -207,6 +209,35 @@ def configure_vars():
DONOT_MODIFY_HANDLERS = conf.getboolean("logging",
"donot_modify_handlers", fallback=False)
+def _run_openlineage_runtime_check():
+ """
+ Ensure compatibility of OpenLineage provider package and Airflow version.
+
+ Airflow 2.10.0 introduced some core changes (#39336) that made versions <=
1.8.0 of OpenLineage
+ provider incompatible with future Airflow versions (>= 2.10.0).
+ """
+ ol_package = "apache-airflow-providers-openlineage"
+ try:
+ ol_version = metadata.version(ol_package)
Review Comment:
Not sure i understand it tho. I mean, i used the importlib here now, but
isn't that what the `ProvidersManager` uses underneath when looking for
packages installed (airflow.utils.entry_points._get_grouped_entry_points)? I
thought that when `ProvidersManager().providers.get(ol_package).is_source` is
False then `ProvidersManager().providers.get(ol_package).version` , it's the
same as `importlib.metadata.version(ol_package)` - it simply detects whether or
not package has been installed.
So i could just write this, and it should work just fine, right?
```
ol_package = "apache-airflow-providers-openlineage"
ol_provider = ProvidersManager().providers.get(ol_package)
if not ol_provider:
return
err_msg = (
f"You have installed `{ol_package}` == `{ol_provider.version}` that
is not compatible with "
f"`apache-airflow` == `{airflow_version}`. "
f"For `apache-airflow` >= `2.10.0` you must use `{ol_package}` >=
`1.8.0`."
)
if ol_provider.version and Version(ol_provider.version) <
Version("1.8.0"):
if ol_provider.is_source:
log.warning(err_msg)
else:
raise RuntimeError(err_msg)
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
