potiuk commented on code in PR #74531:
URL: https://github.com/apache/airflow/pull/74531#discussion_r4308139333
##########
shared/plugins_manager/src/airflow_shared/plugins_manager/plugins_manager.py:
##########
@@ -260,7 +260,14 @@ def _load_entrypoint_plugins(
plugins: list[AirflowPlugin] = []
import_errors: dict[str, str] = {}
+ seen_entry_points: set[metadata.EntryPoint] = set()
for entry_point, dist in entry_points_with_dist("airflow.plugins"):
+ # A distribution is returned once per sys.path entry it is found on.
RHEL/Fedora venvs put
+ # both lib64/ and lib/ site-packages on sys.path with lib64 symlinked
to lib, so without this
+ # every entry-point plugin is loaded twice and the second copy is
reported as a duplicate.
+ if entry_point in seen_entry_points:
+ continue
+ seen_entry_points.add(entry_point)
Review Comment:
Comparing `EntryPoint`s (name, value, group) only catches byte-identical
copies. The same package found twice in different versions (e.g. a venv with
`--system-site-packages`, or a user-site copy) has different entry points, so
both still load and "already registered" still shows. It also silently
collapses two genuinely different packages that declare the same plugin, which
used to surface as an error.
The duplicates come from `_get_grouped_entry_points()` in `module_loading`,
which iterates `metadata.distributions()` without deduplicating. Keeping only
the first distribution per normalized name there, as
`importlib.metadata.entry_points()` does and as `providers_discovery.py`
already does with `canonicalize_name(dist.metadata["name"])`, would fix this
for every caller of `entry_points_with_dist` at once, and the special case here
could go.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
