kaxil opened a new pull request, #72854: URL: https://github.com/apache/airflow/pull/72854
## Summary One installed provider whose metadata this Airflow version does not recognise currently takes down provider discovery for every provider in the process, not just its own entry. [`provider_schema_validator.validate(provider_info)`](https://github.com/apache/airflow/blob/bdf2abcbc6/shared/providers_discovery/src/airflow_shared/providers_discovery/providers_discovery.py#L325) sits unwrapped inside the entry-point loop, and nothing up either call chain catches it. `grep -c ValidationError` returns 0 in both `providers_manager.py` and `providers_manager_runtime.py`. The consequences: - Every provider after the offender in entry-point order is never registered. - `initialize_providers_list()` raises. - `provider_info_cache` does not memoise failures, so it raises again on every subsequent access rather than settling into a degraded state. - Both entry points share this loop: the scheduler, API server and dag-processor via `airflow-core/src/airflow/providers_manager.py`, and workers via `task-sdk/src/airflow/sdk/providers_manager_runtime.py`. Because providers contribute configuration sections, the failure surfaces through config parsing, so a bare `import airflow` can raise. ## It also contradicts the documented contract The docstring on the very function doing the validation says the runtime schema is deliberately the lenient one: > Note that the schema is different at runtime than provider.yaml.schema.json. The development version of provider schema is more strict and changes together with the code. The runtime version is more relaxed (allows for additional properties) and verifies only the subset of fields that are needed at runtime. A schema violation aborting all discovery is the opposite of relaxed. #71104 declined to tighten this same runtime schema for exactly this reason, and #72853 initially added a `pattern` to it before backing that out on the same grounds. The fragility is what makes those tightenings unsafe, so removing it also unblocks legitimate schema work later. ## What changes The offending provider is logged with its package name and skipped; the rest load normally. The scope of the `try` is the single `validate()` call, so a genuine packaging error such as the package-name mismatch check immediately below it still fails loudly as before. `Exception` is caught rather than `jsonschema.exceptions.ValidationError` so that this shared module does not take a direct dependency on `jsonschema`, which is supplied by the caller that builds the validator. Any failure to validate means the metadata is unusable, which is the same outcome either way. ## Not addressed here `entry_point.load()()` on the line above is also unwrapped, so a provider that raises while producing its own metadata still aborts discovery the same way. That is a different failure mode with a different judgement call about whether an unloadable provider should be visible or silent, so it is left alone rather than folded in. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
