kaxil opened a new pull request, #72853: URL: https://github.com/apache/airflow/pull/72853
## Summary The three PydanticAI vendor hooks declared their `conn_type` with hyphens, which made them unreachable from every secrets backend. A hook is registered under the literal `connection-type` string from `provider.yaml`, but [`Connection.from_uri`](https://github.com/apache/airflow/blob/251cd91bb2/airflow-core/src/airflow/models/connection.py#L235) and `Connection.from_json` both run [`_normalize_conn_type`](https://github.com/apache/airflow/blob/251cd91bb2/airflow-core/src/airflow/models/connection.py#L228-L233), which rewrites `-` to `_` before the lookup. The key the hook is [registered under](https://github.com/apache/airflow/blob/251cd91bb2/airflow-core/src/airflow/providers_manager.py#L679-L706) could therefore never be produced by either parse path, and any task using one of these connections failed with: ``` AirflowException: Unknown hook type "pydanticai_azure" File ".../airflow/sdk/definitions/connection.py", line 223 in get_hook ``` A connection read straight out of the metadata DB kept its hyphen and resolved, which is why this went unnoticed. Even that path was only reliable with the secret cache off: [`get_connection_from_secrets`](https://github.com/apache/airflow/blob/251cd91bb2/airflow-core/src/airflow/models/connection.py#L521) caches `conn.get_uri()` and rebuilds the connection from that URI on the next read, so with `secrets.use_cache = True` a DB-stored hyphenated connection worked on first use and failed on every subsequent one. Closes #72316. Supersedes #72341, which identified the same root cause. ## The blast radius is wider than the report The report describes URI-valued backends such as HashiCorp Vault. `from_json` normalizes identically, and so does the `AIRFLOW_CONN_*` environment-variable backend, which is enabled by default and is how most deployments define connections. All six combinations below failed before this change and resolve after it:  Both stored shapes are exercised because both occur in the wild. JSON carries the canonical underscore `conn_type`; a URI's scheme is hyphenated because RFC 3986 forbids `_` in a scheme, so `Connection.get_uri()` writes hyphens and `_normalize_conn_type` converts back on read. Hyphenated URIs already sitting in a secrets backend keep working. ## Why rename rather than also register the hyphenated form Three ways to close this, and why the rename wins: - **Add the hyphenated names to `_normalize_conn_type`'s alias table**, where `postgresql` → `postgres` lives. That puts provider-specific names in airflow-core, for one provider's benefit. - **Register a second hook class per vendor under the hyphenated key.** This does not fix the reported path at all: a hyphenated URI or JSON value already normalizes onto the underscore key, so the extra registration only serves connections sitting hyphenated in the metadata DB. It also adds three unlabelled entries to the connection-type dropdown, because a `connection-types` entry with no `hook-name` falls back to the raw slug. - **Rename**, which matches every other multi-word hook in the tree (`azure_data_factory`, `google_cloud_platform`, `spark_connect`, `hive_cli`) and leaves exactly one canonical name per type. One entry per type after the rename, each with its own label:  ## Migration `default_conn_name` values were already underscored (`pydanticai_azure_default`) and are unchanged, so only `conn_type` moves. A connection already stored with the hyphenated type needs its type corrected once: ```bash airflow connections delete my_azure_conn airflow connections add my_azure_conn \ --conn-type pydanticai_azure \ --conn-host https://my-resource.openai.azure.com \ --conn-password '<api-key>' \ --conn-extra '{"model": "azure:gpt-4o", "api_version": "2024-07-01-preview"}' ``` In the UI, re-pick the type on the existing connection. No compatibility shim is included: these three types are recent, the provider is pre-1.0, and a permanent second registration per vendor is not worth the dropdown noise it costs every user. ## Tradeoff worth stating Underscores bring back a warning whenever a connection is serialized to a URI: ``` Connection schemes (type: pydanticai_azure) shall not contain '_' according to RFC3986. ``` That warning is exactly why `pydantic_ai` became `pydanticai` in #62817. It is a log line only, the hyphenated URI it emits is valid and still round-trips, and `azure_data_factory` and `google_cloud_platform` have carried the same warning for years. Correct hook resolution is worth more than a clean log line. ## Guard against a repeat Nothing stopped a hyphen getting in: `connection-type` was declared as a bare `{"type": "string"}` in both provider schemas. This adds `"pattern": "^[a-z0-9_]+$"` there, so the same mistake now fails schema validation at authoring time, for any provider, rather than at a user's first task run. All 141 `connection-type` values across the provider tree already satisfy it, so it lands green. Two smaller guards alongside it. `TestConnTypeResolution` asserts that every `connection-type` this provider declares survives a URI round-trip, parametrized so it names every offender rather than aborting at the first. And the `connection-types` example in `contributing-docs/23_provider_hook_migration_to_yaml.rst` used the hyphenated form as its illustration, which is what a new provider author copies, so it is corrected too. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
