This is an automated email from the ASF dual-hosted git repository.
kaxil pushed a commit to branch main
in repository https://gitbox.apache.org/repos/asf/airflow.git
The following commit(s) were added to refs/heads/main by this push:
new b7ceab0372b Support Workload Identity Federation in Snowflake REST API
authentication (#75810)
b7ceab0372b is described below
commit b7ceab0372ba346a0962b9226e68c73aa583b6bd
Author: Omar Massfih <[email protected]>
AuthorDate: Thu Nov 5 15:53:44 2026 +0100
Support Workload Identity Federation in Snowflake REST API authentication
(#75810)
SnowflakeSqlApiHook, SnowflakeCortexAgentHook and the pydanticai_snowflake
connection type rejected connections that use Workload Identity Federation,
even though SnowflakeHook supports them. Authenticate those REST calls with
the workload's cloud identity token, sent as `Bearer WIF.<provider>.<token>`
with the WORKLOAD_IDENTITY_FEDERATION token type, and cache it until shortly
before it expires. A connection that also sets a private key still uses
key-pair authentication.
closes: #75808
Do not call snowflake.connector.wif_util, which is not a documented API of
the connector. Request the identity token directly: sts:GetWebIdentityToken
through boto3 for AWS, azure-identity for Azure (AKS workload identity or
the
managed identity), google-auth for GCP, and the connection's token or
token_file_path for OIDC. Each library is imported only for the provider the
connection names, so none becomes a dependency of the provider.
This also drops the snowflake-connector-python>=4.7.1 requirement for AWS.
* Harden Snowflake REST workload identity authentication
* Fix Snowflake REST workload identity account validation
* Align Snowflake REST workload identity credential selection with the
connector
* Fix Snowflake REST workload identity configuration and tests
WIF connections must report incorrect authenticator settings clearly, and
credential selection and tests must reflect the authentication path actually
used by the REST hooks.
Generated-by: OpenAI Codex
* Request hour-long AWS WIF tokens and document WIF on Snowflake REST hooks
Ask STS for a 3600 s GetWebIdentityToken lifetime instead of the 300 s
default, so the
deferred SQL API trigger refreshes the token on its event loop about once
an hour rather
than every few minutes. List WIF in the SnowflakeCortexAgentHook and
SnowflakeSqlApiHook
docstrings, name GetWebIdentityToken in the global-endpoint note, and test
a lowercase
authenticator value.
---------
Co-authored-by: Kaxil Naik <[email protected]>
---
.../docs/connections/pydantic_ai_snowflake.rst | 7 +-
providers/snowflake/docs/connections/snowflake.rst | 27 ++
.../snowflake/docs/operators/cortex_agent.rst | 7 +-
.../providers/snowflake/hooks/cortex_agent.py | 11 +-
.../airflow/providers/snowflake/hooks/sql_api.py | 5 +-
.../providers/snowflake/utils/_rest_auth.py | 213 ++++++++++-
.../unit/snowflake/hooks/test_cortex_agent.py | 23 +-
.../tests/unit/snowflake/hooks/test_sql_api.py | 24 +-
.../tests/unit/snowflake/utils/test__rest_auth.py | 422 ++++++++++++++++++++-
9 files changed, 703 insertions(+), 36 deletions(-)
diff --git a/providers/snowflake/docs/connections/pydantic_ai_snowflake.rst
b/providers/snowflake/docs/connections/pydantic_ai_snowflake.rst
index 6c6f7b812a0..8d51b94baf3 100644
--- a/providers/snowflake/docs/connections/pydantic_ai_snowflake.rst
+++ b/providers/snowflake/docs/connections/pydantic_ai_snowflake.rst
@@ -79,9 +79,14 @@ method the referenced Snowflake connection uses, set the
same way as for
a refresh token, a client credentials grant, or ``azure_conn_id``.
- **PAT (Programmatic Access Token)**: set ``authenticator`` to
``programmatic_access_token`` and
put the PAT value in the Snowflake connection's Password field.
-- **Key-pair JWT**: the default when neither of the above is set. Configure
+- **Key-pair JWT**: used when neither OAuth nor PAT is selected and a private
key is configured,
+ taking precedence over WIF. Configure
``private_key_file`` or ``private_key_content`` (optionally with a
passphrase in Password), and
set Login to the Snowflake user name the key is registered to.
+- **Workload Identity Federation (WIF)**: when no private key is configured,
set
+ ``authenticator`` to ``WORKLOAD_IDENTITY`` and
``workload_identity_provider`` to
+ ``AWS``, ``AZURE``, ``GCP`` or ``OIDC``. See
+ :ref:`Snowflake connection <howto/connection:snowflake>` for prerequisites
and host restrictions.
See the :doc:`Snowflake connection <snowflake>` page for the full field
reference.
diff --git a/providers/snowflake/docs/connections/snowflake.rst
b/providers/snowflake/docs/connections/snowflake.rst
index 51b2b749f3a..3ce101bcfd5 100644
--- a/providers/snowflake/docs/connections/snowflake.rst
+++ b/providers/snowflake/docs/connections/snowflake.rst
@@ -189,3 +189,30 @@ and is granted a role with access to the target objects.
"workload_identity_provider": "GCP"
}
}'
+
+``SnowflakeSqlApiHook``, ``SnowflakeCortexAgentHook`` and the
``pydanticai_snowflake`` connection type call
+Snowflake's REST APIs, which accept the workload's identity token directly
instead of a connector session.
+The token is requested from the cloud and reused until shortly before it
expires. A connection that also
+sets a private key uses key-pair authentication for these REST hooks. Each
provider needs its cloud's client
+library installed:
+
+* ``AWS``: ``boto3`` with ``botocore>=1.41.0`` for ``sts:GetWebIdentityToken``
support.
+ The workload's IAM role must be allowed to
+ call it, and AWS outbound identity federation must be enabled for the
account. This requirement applies
+ only to the REST hooks; ``SnowflakeHook`` uses a signed
``GetCallerIdentity`` request by default and
+ does not require outbound identity federation. ``GetWebIdentityToken`` is
not available on
+ the global STS endpoint. Region resolution uses ``AWS_REGION``, then
``AWS_DEFAULT_REGION``,
+ then EC2 instance metadata. If none resolves, authentication fails with a
configuration error.
+ Regional STS endpoints are used even when legacy global endpoint settings
are present.
+* ``AZURE``: ``azure-identity`` (included in the provider's
``microsoft.azure`` extra).
+ On AKS with workload identity the pod's federated identity is used,
+ otherwise the managed identity. Set ``MANAGED_IDENTITY_CLIENT_ID`` for a
user-assigned managed identity.
+* ``GCP``: ``google-auth``. The ID token comes from the metadata identity
endpoint and is issued
+ to the workload's service account, even when
``GOOGLE_APPLICATION_CREDENTIALS`` is set.
+* ``OIDC``: nothing extra; the token comes from the ``token`` or
``token_file_path`` extra.
+
+Before fetching a WIF token or returning a cached one, REST hooks require a
destination in
+``snowflakecomputing.com``, ``snowflakecomputing.cn`` or
``snowflakecomputing.mil``. Suffixes match
+DNS label boundaries. Administrators can add comma-separated trusted DNS
suffixes through the process
+environment variable ``SNOWFLAKE_WIF_ALLOWED_HOST_SUFFIXES`` for private
deployments. Connection
+extras cannot extend this allowlist. Supply a DNS hostname without a scheme,
port, path or user info.
diff --git a/providers/snowflake/docs/operators/cortex_agent.rst
b/providers/snowflake/docs/operators/cortex_agent.rst
index 8ab07d2971f..a0f45eb0d48 100644
--- a/providers/snowflake/docs/operators/cortex_agent.rst
+++ b/providers/snowflake/docs/operators/cortex_agent.rst
@@ -51,9 +51,14 @@ Authentication
credentials grant, or ``azure_conn_id``, as on the Snowflake connection
generally.
- **PAT (Programmatic Access Token)**: set ``authenticator`` to
``programmatic_access_token`` and
put the PAT value in the connection's Password field.
-- **Key-pair JWT**: the default when neither of the above is set. Configure
+- **Key-pair JWT**: used when neither OAuth nor PAT is selected and a private
key is configured,
+ taking precedence over WIF. Configure
``private_key_file`` or ``private_key_content`` (optionally with a
passphrase in Password), and
set Login to the Snowflake user name the key is registered to.
+- **Workload Identity Federation (WIF)**: when no private key is configured,
set
+ ``authenticator`` to ``WORKLOAD_IDENTITY`` and
``workload_identity_provider`` to
+ ``AWS``, ``AZURE``, ``GCP`` or ``OIDC``. See
+ :ref:`Snowflake connection <howto/connection:snowflake>` for prerequisites
and host restrictions.
See the :doc:`Snowflake connection </connections/snowflake>` page for the full
field reference.
A hook instance keeps its resolved key-pair JWT and reuses it within its
renewal window instead of
diff --git
a/providers/snowflake/src/airflow/providers/snowflake/hooks/cortex_agent.py
b/providers/snowflake/src/airflow/providers/snowflake/hooks/cortex_agent.py
index 999ed6d0b23..e6864fbad3d 100644
--- a/providers/snowflake/src/airflow/providers/snowflake/hooks/cortex_agent.py
+++ b/providers/snowflake/src/airflow/providers/snowflake/hooks/cortex_agent.py
@@ -113,16 +113,19 @@ class SnowflakeCortexAgentHook(SnowflakeHook,
BaseManagedAgentHook):
"""
Hook for interacting with Snowflake Cortex Agents.
- Authenticates the same three ways as ``SnowflakeSqlApiHook``, chosen by
the connection's
+ Authenticates the same four ways as ``SnowflakeSqlApiHook``, chosen by the
connection's
``authenticator`` extra:
1. OAuth: set ``authenticator`` to ``oauth`` and configure a refresh
token, client
credentials grant, or ``azure_conn_id``, as on ``SnowflakeHook``.
2. PAT (Programmatic Access Token): set ``authenticator`` to
``programmatic_access_token``
and put the PAT value in the connection ``password`` field.
- 3. Key-pair JWT: the default when neither of the above is set. Configure
- ``private_key_file`` or ``private_key_content`` (optionally with a
passphrase in
- ``password``), as on ``SnowflakeHook``.
+ 3. Key-pair JWT: used when neither of the above is set and a private key
is configured,
+ taking precedence over WIF. Configure ``private_key_file`` or
``private_key_content``
+ (optionally with a passphrase in ``password``), as on ``SnowflakeHook``.
+ 4. Workload Identity Federation (WIF): when no private key is configured,
set
+ ``authenticator`` to ``WORKLOAD_IDENTITY`` and
``workload_identity_provider`` to
+ ``AWS``, ``AZURE``, ``GCP`` or ``OIDC``.
The resolved token is cached and renewed the same way as
``SnowflakeSqlApiHook`` -- see
``SnowflakeRestTokenProvider``, one instance per hook, so a long-running
agent reuses the
diff --git
a/providers/snowflake/src/airflow/providers/snowflake/hooks/sql_api.py
b/providers/snowflake/src/airflow/providers/snowflake/hooks/sql_api.py
index 6a664bef4d0..a00544f55bf 100644
--- a/providers/snowflake/src/airflow/providers/snowflake/hooks/sql_api.py
+++ b/providers/snowflake/src/airflow/providers/snowflake/hooks/sql_api.py
@@ -59,6 +59,9 @@ class SnowflakeSqlApiHook(SnowflakeHook):
ID and client secret. With ``azure_conn_id``, the token comes from that
Azure connection instead.
3. PAT (Programmatic Access Token): set ``authenticator`` to
``programmatic_access_token`` in the
connection extras and put the PAT value in the connection ``password``
field.
+ 4. Workload Identity Federation (WIF): when no private key is configured,
set ``authenticator`` to
+ ``WORKLOAD_IDENTITY`` and ``workload_identity_provider`` to ``AWS``,
``AZURE``, ``GCP`` or
+ ``OIDC``. A configured private key takes precedence.
:param snowflake_conn_id: Reference to
:ref:`Snowflake connection id<howto/connection:snowflake>`
@@ -230,7 +233,7 @@ class SnowflakeSqlApiHook(SnowflakeHook):
return self.query_ids
def get_headers(self) -> dict[str, Any]:
- """Form auth headers based on OAuth token, PAT, or JWT token from
private key."""
+ """Form auth headers from an OAuth token, PAT, private-key JWT, or
workload identity token."""
if self._rest_token_provider is None:
self._rest_token_provider = SnowflakeRestTokenProvider(
self,
diff --git
a/providers/snowflake/src/airflow/providers/snowflake/utils/_rest_auth.py
b/providers/snowflake/src/airflow/providers/snowflake/utils/_rest_auth.py
index af02b36715f..45d666ad536 100644
--- a/providers/snowflake/src/airflow/providers/snowflake/utils/_rest_auth.py
+++ b/providers/snowflake/src/airflow/providers/snowflake/utils/_rest_auth.py
@@ -15,11 +15,12 @@
# specific language governing permissions and limitations
# under the License.
"""
-Shared Snowflake REST API authentication (OAuth, PAT, key-pair JWT).
+Shared Snowflake REST API authentication (OAuth, PAT, key-pair JWT, workload
identity).
Every Snowflake REST caller (the SQL API, Cortex Agents, the Cortex
chat-completions
-endpoint used by pydantic-ai) authenticates the same three ways: an OAuth
access token,
-a Programmatic Access Token (PAT), or a JWT signed with the connection's
private key.
+endpoint used by pydantic-ai) authenticates the same four ways: an OAuth
access token,
+a Programmatic Access Token (PAT), a JWT signed with the connection's private
key, or a
+Workload Identity Federation (WIF) token issued by the workload's cloud.
:class:`SnowflakeRestTokenProvider` produces those headers from a
:class:`SnowflakeHook`
so each caller does not need to duplicate the branching or the token caching.
@@ -30,12 +31,19 @@ touch those optional dependencies.
from __future__ import annotations
+import os
+import re
import threading
+import time
from collections.abc import Callable
from dataclasses import dataclass, field
from datetime import timedelta
+from pathlib import Path
from typing import TYPE_CHECKING, Any
+import jwt
+
+from airflow.providers.common.compat.sdk import
AirflowOptionalProviderFeatureException
from airflow.providers.snowflake.hooks.snowflake import
_validate_account_component
from airflow.providers.snowflake.utils.sql_api_generate_jwt import JWTGenerator
@@ -45,6 +53,17 @@ if TYPE_CHECKING:
from airflow.providers.snowflake.hooks.snowflake import SnowflakeHook
+# Re-fetch a workload identity token this many seconds before it expires.
+WIF_TOKEN_EXPIRY_BUFFER = 30
+# Audience Snowflake expects in the identity token: its domain for AWS and
GCP, and its Entra
+# application for Azure.
+WIF_AUDIENCE = "snowflakecomputing.com"
+WIF_AZURE_RESOURCE = "api://fd3f753b-eed3-462c-b6a7-a4b5bb650aad"
+# Lifetime to request for an AWS GetWebIdentityToken JWT. STS defaults to 300
seconds and allows up
+# to 3600; the longer lifetime keeps refreshes rare on the deferred SQL API
trigger's event loop.
+WIF_AWS_TOKEN_DURATION_SECONDS = 3600
+
+
@dataclass(frozen=True)
class SnowflakeRestToken:
"""A REST bearer token and the ``X-Snowflake-Authorization-Token-Type``
value it needs."""
@@ -61,9 +80,14 @@ class SnowflakeRestTokenProvider:
reads the token ``hook._get_conn_params()`` already resolved (which itself
refreshes an
expiring OAuth or Azure token, so this provider does not cache that branch
at all);
``authenticator == "programmatic_access_token"`` reads the PAT from the
connection password;
- anything else signs a key-pair JWT. The private key is loaded once and
kept. The
- ``JWTGenerator`` is built once and kept, because it renews its own token
on the
- ``token_renewal_delta`` schedule.
+ a connection without a private key that sets
``authenticator=WORKLOAD_IDENTITY`` and
+ ``workload_identity_provider`` uses Workload Identity Federation; anything
else signs a key-pair JWT.
+ The private key is loaded once and
+ kept. The ``JWTGenerator`` is built once and kept, because it renews its
own token on the
+ ``token_renewal_delta`` schedule. The workload identity token is requested
from the cloud
+ with its own client library (``boto3``, ``azure-identity`` or
``google-auth``, imported only
+ for the provider the connection names), cached until shortly before its
``exp`` claim, and
+ not cached when it carries none.
:param hook: The ``SnowflakeHook`` (or subclass) whose connection supplies
credentials.
:param token_life_time: Passed to the ``JWTGenerator`` for the key-pair
branch.
@@ -94,7 +118,10 @@ class SnowflakeRestTokenProvider:
private_key_loader or hook.get_private_key
)
self._private_key: PrivateKeyTypes | None = None
+ self._private_key_loaded = False
self._jwt_generator: JWTGenerator | None = None
+ self._wif_token: SnowflakeRestToken | None = None
+ self._wif_token_expires_at: float = 0.0
self._lock = threading.Lock()
def get_token(self) -> SnowflakeRestToken:
@@ -117,18 +144,21 @@ class SnowflakeRestTokenProvider:
)
return SnowflakeRestToken(token=pat,
token_type="PROGRAMMATIC_ACCESS_TOKEN")
- if self._private_key is None:
+ if not self._private_key_loaded:
self._private_key = self._private_key_loader()
+ self._private_key_loaded = True
if self._private_key is None:
if conn_config.get("workload_identity_provider"):
- raise ValueError(
- "Workload identity federation is not supported for
Snowflake REST APIs; use "
- "OAuth, PAT, or key-pair authentication instead. A
connection that also "
- "sets a private key uses key-pair authentication."
- )
+ if (conn_config.get("authenticator") or "").upper() !=
"WORKLOAD_IDENTITY":
+ raise ValueError(
+ "workload_identity_provider was set but
authenticator was not set to "
+ "WORKLOAD_IDENTITY"
+ )
+ return self._get_workload_identity_token(conn_config)
raise ValueError(
"Snowflake REST API authentication requires an OAuth
access token, a "
- "Programmatic Access Token (PAT), or a private key for
key-pair JWT auth; "
+ "Programmatic Access Token (PAT), a private key for
key-pair JWT auth, or a "
+ "workload_identity_provider for Workload Identity
Federation; "
"none is configured on this connection."
)
@@ -154,6 +184,61 @@ class SnowflakeRestTokenProvider:
token = self._jwt_generator.get_token()
return SnowflakeRestToken(token=token, token_type="KEYPAIR_JWT")
+ def _get_workload_identity_token(self, conn_config: dict[str, Any]) ->
SnowflakeRestToken:
+ """
+ Return a Workload Identity Federation token, fetching a new one when
the cached one expires.
+
+ The caller must hold ``self._lock``.
+ """
+ _validate_wif_host(conn_config)
+ if self._wif_token is not None and time.time() <
self._wif_token_expires_at:
+ return self._wif_token
+
+ provider_name = str(conn_config["workload_identity_provider"]).upper()
+ if provider_name == "AWS":
+ credential = _fetch_aws_identity_token()
+ elif provider_name == "AZURE":
+ credential = _fetch_azure_identity_token()
+ elif provider_name == "GCP":
+ credential = _fetch_gcp_identity_token()
+ elif provider_name == "OIDC":
+ credential = self._get_oidc_token(conn_config)
+ else:
+ raise ValueError(
+ f"Unknown workload_identity_provider {provider_name!r};
expected one of "
+ "AWS, AZURE, GCP or OIDC."
+ )
+
+ token = SnowflakeRestToken(
+ token=f"WIF.{provider_name}.{credential}",
+ token_type="WORKLOAD_IDENTITY_FEDERATION",
+ )
+ expires_at = self._get_jwt_expiry(credential)
+ self._wif_token = token if expires_at else None
+ self._wif_token_expires_at = (expires_at or 0.0) -
WIF_TOKEN_EXPIRY_BUFFER
+ return token
+
+ @staticmethod
+ def _get_oidc_token(conn_config: dict[str, Any]) -> str:
+ """Return the OIDC ID token from the connection ``token`` or
``token_file_path`` extra."""
+ if token_file_path := conn_config.get("token_file_path"):
+ return Path(token_file_path).read_text().strip()
+ if token := conn_config.get("token"):
+ return token
+ raise ValueError(
+ "Workload Identity Federation with the OIDC provider requires the
connection extra "
+ "'token' or 'token_file_path'."
+ )
+
+ @staticmethod
+ def _get_jwt_expiry(credential: str) -> float | None:
+ """Return the ``exp`` claim of ``credential`` without verifying it, or
None if it has none."""
+ try:
+ exp = jwt.decode(credential, options={"verify_signature":
False}).get("exp")
+ except jwt.PyJWTError:
+ return None
+ return float(exp) if isinstance(exp, int | float) else None
+
def build_auth_headers(self) -> dict[str, str]:
"""Return the two REST auth headers: ``Authorization`` and the
token-type header."""
token = self.get_token()
@@ -163,6 +248,108 @@ class SnowflakeRestTokenProvider:
}
+def _validate_wif_host(conn_config: dict[str, Any]) -> None:
+ """Restrict ambient workload credentials to trusted Snowflake
destinations."""
+ host =
get_cortex_base_url(conn_config).removeprefix("https://").lower().removesuffix(".")
+ suffixes = ["snowflakecomputing.com", "snowflakecomputing.cn",
"snowflakecomputing.mil"]
+ suffixes.extend(
+ entry.strip().lower().strip(".")
+ for entry in os.environ.get("SNOWFLAKE_WIF_ALLOWED_HOST_SUFFIXES",
"").split(",")
+ if entry.strip()
+ )
+ dns_name =
r"[a-z0-9_](?:[a-z0-9_-]{0,61}[a-z0-9_])?(?:\.[a-z0-9_](?:[a-z0-9_-]{0,61}[a-z0-9_])?)*"
+ if (
+ len(host) <= 253
+ and re.fullmatch(dns_name, host)
+ and any(
+ re.fullmatch(dns_name, suffix) and (host == suffix or
host.endswith(f".{suffix}"))
+ for suffix in suffixes
+ )
+ ):
+ return
+ raise ValueError("Workload Identity Federation requires a recognized
Snowflake host.")
+
+
+def _fetch_aws_identity_token() -> str:
+ """Return the JWT from regional STS for the workload's AWS identity."""
+ try:
+ import boto3
+ from botocore.session import get_session
+ from botocore.utils import InstanceMetadataRegionFetcher
+ except ImportError as e:
+ raise AirflowOptionalProviderFeatureException(
+ "Workload Identity Federation with the AWS provider requires
boto3. Install boto3."
+ ) from e
+ region = (
+ os.environ.get("AWS_REGION")
+ or os.environ.get("AWS_DEFAULT_REGION")
+ or InstanceMetadataRegionFetcher().retrieve_region()
+ )
+ if not region:
+ raise ValueError(
+ "AWS workload identity requires a region: set AWS_REGION or
AWS_DEFAULT_REGION, "
+ "or enable EC2 instance metadata region discovery."
+ )
+ session = get_session()
+ session.set_config_variable("sts_regional_endpoints", "regional")
+ sts = boto3.Session(botocore_session=session).client("sts",
region_name=region)
+ if not hasattr(sts, "get_web_identity_token"):
+ raise AirflowOptionalProviderFeatureException(
+ "Workload Identity Federation with the AWS provider requires a
boto3 version that "
+ "supports sts:GetWebIdentityToken (botocore>=1.41.0). Upgrade
boto3."
+ )
+ response = sts.get_web_identity_token(
+ Audience=[WIF_AUDIENCE],
+ SigningAlgorithm="ES384",
+ DurationSeconds=WIF_AWS_TOKEN_DURATION_SECONDS,
+ )
+ return response["WebIdentityToken"]
+
+
+def _fetch_azure_identity_token() -> str:
+ """
+ Return an Entra access token for Snowflake's application, issued to the
workload's identity.
+
+ Uses the AKS workload identity when the pod has one (the Azure Workload
Identity webhook sets
+ ``AZURE_CLIENT_ID``, ``AZURE_TENANT_ID`` and
``AZURE_FEDERATED_TOKEN_FILE``), and the managed
+ identity otherwise. A user-assigned managed identity is selected with
+ ``MANAGED_IDENTITY_CLIENT_ID``.
+ """
+ try:
+ from azure.identity import ManagedIdentityCredential,
WorkloadIdentityCredential
+ except ImportError as e:
+ raise AirflowOptionalProviderFeatureException(
+ "Workload Identity Federation with the AZURE provider requires
azure-identity. "
+ "Install apache-airflow-providers-snowflake[microsoft.azure]."
+ ) from e
+ credential: ManagedIdentityCredential | WorkloadIdentityCredential
+ if all(
+ os.environ.get(name) for name in ("AZURE_CLIENT_ID",
"AZURE_TENANT_ID", "AZURE_FEDERATED_TOKEN_FILE")
+ ):
+ credential = WorkloadIdentityCredential()
+ else:
+ credential =
ManagedIdentityCredential(client_id=os.environ.get("MANAGED_IDENTITY_CLIENT_ID"))
+ try:
+ return credential.get_token(f"{WIF_AZURE_RESOURCE}/.default").token
+ finally:
+ credential.close()
+
+
+def _fetch_gcp_identity_token() -> str:
+ """Return a Google ID token for Snowflake's audience, issued to the
workload's service account."""
+ try:
+ from google.auth.compute_engine import IDTokenCredentials
+ from google.auth.transport.requests import Request
+ except ImportError as e:
+ raise AirflowOptionalProviderFeatureException(
+ "Workload Identity Federation with the GCP provider requires
google-auth. Install google-auth."
+ ) from e
+ request = Request()
+ credential = IDTokenCredentials(request, WIF_AUDIENCE,
use_metadata_identity_endpoint=True)
+ credential.refresh(request)
+ return credential.token
+
+
def get_cortex_base_url(conn_config: dict[str, Any]) -> str:
"""
Return the base URL for a Snowflake account's Cortex REST endpoints.
diff --git
a/providers/snowflake/tests/unit/snowflake/hooks/test_cortex_agent.py
b/providers/snowflake/tests/unit/snowflake/hooks/test_cortex_agent.py
index e9d35906ef3..76c96721558 100644
--- a/providers/snowflake/tests/unit/snowflake/hooks/test_cortex_agent.py
+++ b/providers/snowflake/tests/unit/snowflake/hooks/test_cortex_agent.py
@@ -399,6 +399,7 @@ class TestSnowflakeCortexAgentHook:
assert headers["Authorization"] == "Bearer my-pat-value"
assert headers["X-Snowflake-Authorization-Token-Type"] ==
"PROGRAMMATIC_ACCESS_TOKEN"
+
@mock.patch("airflow.providers.snowflake.utils._rest_auth._fetch_gcp_identity_token",
autospec=True)
@mock.patch(f"{MODULE_PATH}.requests.request", autospec=True)
@mock.patch(f"{HOOK_PATH}.get_private_key", autospec=True,
return_value=None)
@mock.patch(f"{HOOK_PATH}._get_conn_params", autospec=True)
@@ -406,30 +407,30 @@ class TestSnowflakeCortexAgentHook:
f"{HOOK_PATH}._get_static_conn_params",
new_callable=mock.PropertyMock,
)
- def test_request_raises_for_workload_identity_connections(
+ def test_request_sends_workload_identity_headers(
self,
mock_static_conn_params,
mock_conn_params,
mock_get_private_key,
mock_request,
+ mock_fetch_token,
):
- """WORKLOAD_IDENTITY connections without a private key are not
supported for REST APIs --
- SnowflakeHook sets a ``token`` in extras for this authenticator, but
it is not one of the
- three REST-accepted credential types, so this must be an explicit,
actionable error rather
- than silently falling into (and failing) the key-pair branch."""
+ """WORKLOAD_IDENTITY connections without a private key authenticate
with a WIF token."""
mock_static_conn_params.return_value = STATIC_CONN_PARAMS
mock_conn_params.return_value = {
"account": ACCOUNT,
- "workload_identity_provider": "AWS",
- "token": "wif-token",
+ "authenticator": "WORKLOAD_IDENTITY",
+ "workload_identity_provider": "GCP",
}
+ mock_fetch_token.return_value = "gcp-id-token"
+ mock_request.return_value = create_response(json_body={"name":
AGENT_NAME})
hook = SnowflakeCortexAgentHook(snowflake_conn_id="mock_conn_id")
+ hook.describe_agent(database=DATABASE, schema=SCHEMA,
agent_name=AGENT_NAME)
- with pytest.raises(ValueError, match="Workload identity federation"):
- hook.describe_agent(database=DATABASE, schema=SCHEMA,
agent_name=AGENT_NAME)
-
- mock_request.assert_not_called()
+ headers = mock_request.call_args.kwargs["headers"]
+ assert headers["Authorization"] == "Bearer WIF.GCP.gcp-id-token"
+ assert headers["X-Snowflake-Authorization-Token-Type"] ==
"WORKLOAD_IDENTITY_FEDERATION"
@mock.patch(f"{MODULE_PATH}.requests.request", autospec=True)
@mock.patch(f"{HOOK_PATH}.get_private_key", autospec=True)
diff --git a/providers/snowflake/tests/unit/snowflake/hooks/test_sql_api.py
b/providers/snowflake/tests/unit/snowflake/hooks/test_sql_api.py
index 0bd6085cb46..46ec8757247 100644
--- a/providers/snowflake/tests/unit/snowflake/hooks/test_sql_api.py
+++ b/providers/snowflake/tests/unit/snowflake/hooks/test_sql_api.py
@@ -534,11 +534,33 @@ class TestSnowflakeSqlApiHook:
self, mock_get_token, mock_conn_param, mock_private_key
):
mock_get_token.return_value = "newT0k3n"
- mock_conn_param.return_value = {**CONN_PARAMS,
"workload_identity_provider": "AWS"}
+ mock_conn_param.return_value = {
+ **CONN_PARAMS,
+ "authenticator": "WORKLOAD_IDENTITY",
+ "workload_identity_provider": "AWS",
+ }
hook = SnowflakeSqlApiHook(snowflake_conn_id="mock_conn_id")
result = hook.get_headers()
assert result["X-Snowflake-Authorization-Token-Type"] == "KEYPAIR_JWT"
+
@mock.patch("airflow.providers.snowflake.utils._rest_auth._fetch_gcp_identity_token",
autospec=True)
+ @mock.patch(f"{HOOK_PATH}.get_private_key", autospec=True,
return_value=None)
+ @mock.patch(f"{HOOK_PATH}._get_conn_params", autospec=True)
+ def test_get_headers_uses_workload_identity_without_private_key(
+ self, mock_conn_param, mock_private_key, mock_fetch_token
+ ):
+ mock_conn_param.return_value = {
+ **CONN_PARAMS,
+ "region": "us-east-2.aws",
+ "authenticator": "WORKLOAD_IDENTITY",
+ "workload_identity_provider": "GCP",
+ }
+ mock_fetch_token.return_value = "gcp-id-token"
+ hook = SnowflakeSqlApiHook(snowflake_conn_id="mock_conn_id")
+ result = hook.get_headers()
+ assert result["Authorization"] == "Bearer WIF.GCP.gcp-id-token"
+ assert result["X-Snowflake-Authorization-Token-Type"] ==
"WORKLOAD_IDENTITY_FEDERATION"
+
@pytest.mark.parametrize(
"extra",
[
diff --git a/providers/snowflake/tests/unit/snowflake/utils/test__rest_auth.py
b/providers/snowflake/tests/unit/snowflake/utils/test__rest_auth.py
index 51d7fe24763..47e50dac3a4 100644
--- a/providers/snowflake/tests/unit/snowflake/utils/test__rest_auth.py
+++ b/providers/snowflake/tests/unit/snowflake/utils/test__rest_auth.py
@@ -16,18 +16,26 @@
# under the License.
from __future__ import annotations
+import sys
+import time
from datetime import UTC, datetime, timedelta
from unittest import mock
+import boto3
import jwt
import pytest
+from botocore.stub import Stubber
from cryptography.hazmat.backends import default_backend
from cryptography.hazmat.primitives.asymmetric import rsa
+from airflow.providers.common.compat.sdk import
AirflowOptionalProviderFeatureException
from airflow.providers.snowflake.hooks.snowflake import SnowflakeHook
from airflow.providers.snowflake.utils._rest_auth import (
SnowflakeRestToken,
SnowflakeRestTokenProvider,
+ _fetch_aws_identity_token,
+ _fetch_azure_identity_token,
+ _fetch_gcp_identity_token,
get_cortex_base_url,
)
@@ -40,6 +48,16 @@ CONN_PARAMS_PAT = {
"password": "my_pat_token_value",
}
CONN_PARAMS_KEYPAIR = {"account": "airflow", "authenticator": "snowflake",
"user": "user"}
+CONN_PARAMS_WIF = {
+ "account": "airflow",
+ "authenticator": "WORKLOAD_IDENTITY",
+ "workload_identity_provider": "GCP",
+}
+FETCH_TOKEN = MODULE_PATH + "._fetch_{}_identity_token"
+
+
+def _make_jwt(**claims) -> str:
+ return jwt.encode(claims, "secret", algorithm="HS256")
class TestSnowflakeRestTokenProvider:
@@ -148,7 +166,11 @@ class TestSnowflakeRestTokenProvider:
@mock.patch.object(SnowflakeHook, "_get_conn_params", autospec=True)
def test_get_token_workload_identity_with_private_key_uses_key_pair(self,
mock_conn_params):
- mock_conn_params.return_value = {**CONN_PARAMS_KEYPAIR,
"workload_identity_provider": "AWS"}
+ mock_conn_params.return_value = {
+ **CONN_PARAMS_KEYPAIR,
+ "authenticator": "WORKLOAD_IDENTITY",
+ "workload_identity_provider": "AWS",
+ }
key = rsa.generate_private_key(backend=default_backend(),
public_exponent=65537, key_size=2048)
hook = SnowflakeHook(snowflake_conn_id="mock_conn_id")
provider = SnowflakeRestTokenProvider(hook, private_key_loader=lambda:
key)
@@ -156,12 +178,163 @@ class TestSnowflakeRestTokenProvider:
assert provider.get_token().token_type == "KEYPAIR_JWT"
@mock.patch.object(SnowflakeHook, "_get_conn_params", autospec=True)
- def test_get_token_workload_identity_without_private_key_raises(self,
mock_conn_params):
- mock_conn_params.return_value = {**CONN_PARAMS_KEYPAIR,
"workload_identity_provider": "AWS"}
+ @mock.patch(FETCH_TOKEN.format("gcp"), autospec=True)
+ def test_private_key_absence_cached_across_wif_refresh(self, fetch,
conn_params, time_machine):
+ time_machine.move_to("2024-01-01T00:00:00Z", tick=False)
+ conn_params.return_value = CONN_PARAMS_WIF
+ fetch.return_value = _make_jwt(exp=int(time.time()) + 3600)
+ loader = mock.Mock(spec=SnowflakeHook().get_private_key,
return_value=None)
+ provider = SnowflakeRestTokenProvider(SnowflakeHook(),
private_key_loader=loader)
+ provider.get_token()
+ provider.get_token()
+ time_machine.shift(timedelta(hours=1))
+ provider.get_token()
+ loader.assert_called_once_with()
+ assert fetch.call_count == 2
+
+ @mock.patch.object(SnowflakeHook, "_get_conn_params", autospec=True)
+ def test_private_key_loader_retries_after_error(self, conn_params):
+ conn_params.return_value = {**CONN_PARAMS_WIF,
"workload_identity_provider": "OIDC", "token": "token"}
+ loader = mock.Mock(spec=SnowflakeHook().get_private_key,
side_effect=[OSError("unavailable"), None])
+ provider = SnowflakeRestTokenProvider(SnowflakeHook(),
private_key_loader=loader)
+ with pytest.raises(OSError, match="unavailable"):
+ provider.get_token()
+ assert provider.get_token().token == "WIF.OIDC.token"
+ assert loader.call_count == 2
+
+ @pytest.mark.parametrize("authenticator", ["WORKLOAD_IDENTITY",
"workload_identity"])
+ @pytest.mark.parametrize("provider_name", ["GCP", "AZURE", "AWS", "gcp"])
+ @mock.patch.object(SnowflakeHook, "_get_conn_params", autospec=True)
+ def test_get_token_workload_identity(self, mock_conn_params,
provider_name, authenticator):
+ credential = _make_jwt(exp=int(time.time()) + 3600)
+ mock_conn_params.return_value = {
+ **CONN_PARAMS_WIF,
+ "authenticator": authenticator,
+ "workload_identity_provider": provider_name,
+ }
+ hook = SnowflakeHook(snowflake_conn_id="mock_conn_id")
+ provider = SnowflakeRestTokenProvider(hook, private_key_loader=lambda:
None)
+
+ with mock.patch(
+ FETCH_TOKEN.format(provider_name.lower()), autospec=True,
return_value=credential
+ ) as fetch:
+ token = provider.get_token()
+
+ assert token.token == f"WIF.{provider_name.upper()}.{credential}"
+ assert token.token_type == "WORKLOAD_IDENTITY_FEDERATION"
+ fetch.assert_called_once_with()
+
+ @pytest.mark.parametrize("authenticator", [None, "", "snowflake",
"externalbrowser"])
+ @mock.patch(FETCH_TOKEN.format("gcp"), autospec=True)
+ @mock.patch.object(SnowflakeHook, "_get_conn_params", autospec=True)
+ def test_provider_extra_does_not_enable_wif(self, conn_params, fetch,
authenticator):
+ conn_params.return_value = {**CONN_PARAMS_WIF, "authenticator":
authenticator}
+ provider = SnowflakeRestTokenProvider(SnowflakeHook(),
private_key_loader=lambda: None)
+ with pytest.raises(
+ ValueError,
+ match="workload_identity_provider was set but authenticator was
not set to WORKLOAD_IDENTITY",
+ ):
+ provider.get_token()
+ fetch.assert_not_called()
+
+ @mock.patch.object(SnowflakeHook, "_get_conn_params", autospec=True)
+ def test_get_token_workload_identity_oidc_token(self, mock_conn_params):
+ mock_conn_params.return_value = {
+ **CONN_PARAMS_WIF,
+ "workload_identity_provider": "OIDC",
+ "token": "oidc-id-token",
+ }
+ hook = SnowflakeHook(snowflake_conn_id="mock_conn_id")
+ provider = SnowflakeRestTokenProvider(hook, private_key_loader=lambda:
None)
+
+ assert provider.get_token().token == "WIF.OIDC.oidc-id-token"
+
+ @mock.patch.object(SnowflakeHook, "_get_conn_params", autospec=True)
+ @pytest.mark.parametrize("static_token", [None, "leftover-static-token"])
+ def test_get_token_workload_identity_oidc_token_file(self,
mock_conn_params, tmp_path, static_token):
+ token_file = tmp_path / "token"
+ token_file.write_text("file-id-token\n")
+ mock_conn_params.return_value = {
+ **CONN_PARAMS_WIF,
+ "workload_identity_provider": "OIDC",
+ "token_file_path": str(token_file),
+ "token": static_token,
+ }
+ hook = SnowflakeHook(snowflake_conn_id="mock_conn_id")
+ provider = SnowflakeRestTokenProvider(hook, private_key_loader=lambda:
None)
+
+ assert provider.get_token().token == "WIF.OIDC.file-id-token"
+
+ @mock.patch.object(SnowflakeHook, "_get_conn_params", autospec=True)
+ def test_get_token_workload_identity_oidc_without_token_raises(self,
mock_conn_params):
+ mock_conn_params.return_value = {**CONN_PARAMS_WIF,
"workload_identity_provider": "OIDC"}
+ hook = SnowflakeHook(snowflake_conn_id="mock_conn_id")
+ provider = SnowflakeRestTokenProvider(hook, private_key_loader=lambda:
None)
+
+ with pytest.raises(ValueError, match="'token' or 'token_file_path'"):
+ provider.get_token()
+
+ @mock.patch.object(SnowflakeHook, "_get_conn_params", autospec=True)
+ def test_get_token_workload_identity_unknown_provider_raises(self,
mock_conn_params):
+ mock_conn_params.return_value = {**CONN_PARAMS_WIF,
"workload_identity_provider": "nope"}
+ hook = SnowflakeHook(snowflake_conn_id="mock_conn_id")
+ provider = SnowflakeRestTokenProvider(hook, private_key_loader=lambda:
None)
+
+ with pytest.raises(ValueError, match="AWS, AZURE, GCP or OIDC"):
+ provider.get_token()
+
+ @mock.patch.object(SnowflakeHook, "_get_conn_params", autospec=True)
+ def test_get_token_workload_identity_is_cached_until_expiry(self,
mock_conn_params):
+ mock_conn_params.return_value = CONN_PARAMS_WIF
+ hook = SnowflakeHook(snowflake_conn_id="mock_conn_id")
+ provider = SnowflakeRestTokenProvider(hook, private_key_loader=lambda:
None)
+
+ with mock.patch(
+ FETCH_TOKEN.format("gcp"), autospec=True,
return_value=_make_jwt(exp=int(time.time()) + 3600)
+ ) as fetch:
+ first = provider.get_token()
+ second = provider.get_token()
+
+ assert first == second
+ fetch.assert_called_once()
+
+ @pytest.mark.parametrize(
+ "claims",
+ [
+ pytest.param({"exp": 10}, id="expires-within-buffer"),
+ pytest.param({}, id="no-exp-claim"),
+ ],
+ )
+ @mock.patch.object(SnowflakeHook, "_get_conn_params", autospec=True)
+ def test_get_token_workload_identity_is_refetched_when_not_reusable(self,
mock_conn_params, claims):
+ if "exp" in claims:
+ claims = {"exp": int(time.time()) + claims["exp"]}
+ mock_conn_params.return_value = CONN_PARAMS_WIF
+ hook = SnowflakeHook(snowflake_conn_id="mock_conn_id")
+ provider = SnowflakeRestTokenProvider(hook, private_key_loader=lambda:
None)
+
+ with mock.patch(FETCH_TOKEN.format("gcp"), autospec=True,
return_value=_make_jwt(**claims)) as fetch:
+ provider.get_token()
+ provider.get_token()
+
+ assert fetch.call_count == 2
+
+ @mock.patch.object(SnowflakeHook, "_get_conn_params", autospec=True)
+ def test_get_token_workload_identity_credential_is_not_a_jwt(self,
mock_conn_params):
+ mock_conn_params.return_value = CONN_PARAMS_WIF
+ hook = SnowflakeHook(snowflake_conn_id="mock_conn_id")
+ provider = SnowflakeRestTokenProvider(hook, private_key_loader=lambda:
None)
+
+ with mock.patch(FETCH_TOKEN.format("gcp"), autospec=True,
return_value="opaque-access-token"):
+ assert provider.get_token().token == "WIF.GCP.opaque-access-token"
+
+ @mock.patch.object(SnowflakeHook, "_get_conn_params", autospec=True)
+ def test_get_token_without_any_credential_raises(self, mock_conn_params):
+ mock_conn_params.return_value = CONN_PARAMS_KEYPAIR
hook = SnowflakeHook(snowflake_conn_id="mock_conn_id")
provider = SnowflakeRestTokenProvider(hook, private_key_loader=lambda:
None)
- with pytest.raises(ValueError, match="Workload identity federation"):
+ with pytest.raises(ValueError, match="workload_identity_provider"):
provider.get_token()
@mock.patch(f"{MODULE_PATH}.JWTGenerator", autospec=True)
@@ -227,6 +400,247 @@ class TestSnowflakeRestTokenProvider:
assert headers["X-Snowflake-Authorization-Token-Type"] ==
"PROGRAMMATIC_ACCESS_TOKEN"
+class TestWifDestinations:
+ @pytest.mark.parametrize(
+ "host",
+ [
+ "attacker.example",
+ "evil-snowflakecomputing.com",
+ "a.snowflakecomputing.com.evil",
+ "[email protected]",
+ "a.snowflakecomputing.com:443",
+ "a.snowflakecomputing.com/path",
+ "a.snowflakecomputing.com?query",
+ "a.snowflakecomputing.com#fragment",
+ "a..snowflakecomputing.com",
+ "a.snowflakecomputing.com..",
+ "a.snowflakecomputing.com\n",
+ "a.snowflakecomputing.com%00",
+ "https://a.snowflakecomputing.com",
+ "a\\.snowflakecomputing.com",
+ ],
+ )
+ @pytest.mark.parametrize("cloud", ["AWS", "AZURE", "GCP", "OIDC"])
+ def test_rejects_before_fetch_or_file_read(self, host, cloud):
+ hook = mock.create_autospec(SnowflakeHook, instance=True)
+ hook._get_conn_params.return_value = {
+ **CONN_PARAMS_WIF,
+ "host": host,
+ "workload_identity_provider": cloud,
+ "token_file_path": "/does/not/exist",
+ }
+ provider = SnowflakeRestTokenProvider(hook, private_key_loader=lambda:
None)
+ with mock.patch(
+ FETCH_TOKEN.format(cloud.lower()) if cloud != "OIDC" else
f"{MODULE_PATH}.Path.read_text",
+ autospec=True,
+ ) as fetch:
+ with pytest.raises(ValueError, match="recognized Snowflake host"):
+ provider.get_token()
+ fetch.assert_not_called()
+
+ @pytest.mark.parametrize(
+ "conn_config",
+ [
+ {"host": "a.snowflakecomputing.com"},
+ {"host": "A.snowflakecomputing.cn."},
+ {"host": "a.snowflakecomputing.mil"},
+ {"host": "private.example"},
+ {"host": "a.private.example"},
+ {"host": "myorg-my_account.snowflakecomputing.com"},
+ {"account": "myorg-my_account"},
+ {"account": "_my_account_"},
+ {"account": "_"},
+ ],
+ )
+ @mock.patch(FETCH_TOKEN.format("gcp"), autospec=True,
return_value="id-token")
+ def test_allowed_destinations(self, fetch, conn_config, monkeypatch):
+ monkeypatch.setenv("SNOWFLAKE_WIF_ALLOWED_HOST_SUFFIXES", "
private.example,other.example ")
+ hook = mock.create_autospec(SnowflakeHook, instance=True)
+ hook._get_conn_params.return_value = {**CONN_PARAMS_WIF, **conn_config}
+ provider = SnowflakeRestTokenProvider(hook, private_key_loader=lambda:
None)
+ assert provider.get_token().token == "WIF.GCP.id-token"
+
+ @mock.patch(FETCH_TOKEN.format("gcp"), autospec=True)
+ def test_validates_cached_token_destination(self, fetch, monkeypatch):
+ monkeypatch.setenv("SNOWFLAKE_WIF_ALLOWED_HOST_SUFFIXES",
"private.example")
+ fetch.return_value = _make_jwt(exp=int(time.time()) + 3600)
+ hook = mock.create_autospec(SnowflakeHook, instance=True)
+ hook._get_conn_params.return_value = {**CONN_PARAMS_WIF, "host":
"private.example"}
+ provider = SnowflakeRestTokenProvider(hook, private_key_loader=lambda:
None)
+ provider.get_token()
+ monkeypatch.delenv("SNOWFLAKE_WIF_ALLOWED_HOST_SUFFIXES")
+ with pytest.raises(ValueError, match="recognized Snowflake host"):
+ provider.get_token()
+ fetch.assert_called_once()
+
+ def test_connection_cannot_extend_allowlist(self, monkeypatch):
+ monkeypatch.delenv("SNOWFLAKE_WIF_ALLOWED_HOST_SUFFIXES",
raising=False)
+ hook = mock.create_autospec(SnowflakeHook, instance=True)
+ hook._get_conn_params.return_value = {
+ **CONN_PARAMS_WIF,
+ "host": "evil.example",
+ "SNOWFLAKE_WIF_ALLOWED_HOST_SUFFIXES": "evil.example",
+ }
+ with pytest.raises(ValueError, match="recognized Snowflake host"):
+ SnowflakeRestTokenProvider(hook, private_key_loader=lambda:
None).get_token()
+
+
+class TestFetchIdentityTokens:
+ @pytest.mark.parametrize(
+ ("aws_region", "default_region", "metadata_region", "expected"),
+ [
+ ("eu-west-1", "eu-west-2", "eu-west-3", "eu-west-1"),
+ (None, "eu-west-2", "eu-west-3", "eu-west-2"),
+ (None, None, "eu-west-3", "eu-west-3"),
+ ("us-east-1", None, None, "us-east-1"),
+ ],
+ )
+ @mock.patch("botocore.utils.InstanceMetadataRegionFetcher", autospec=True)
+ def test_aws_requests_regional_web_identity_token(
+ self, metadata, aws_region, default_region, metadata_region, expected,
monkeypatch
+ ):
+ for name, value in (("AWS_REGION", aws_region), ("AWS_DEFAULT_REGION",
default_region)):
+ if value:
+ monkeypatch.setenv(name, value)
+ else:
+ monkeypatch.delenv(name, raising=False)
+ monkeypatch.setenv("AWS_STS_REGIONAL_ENDPOINTS", "legacy")
+ metadata.return_value.retrieve_region.return_value = metadata_region
+ real_session = boto3.Session
+ clients = []
+
+ def create_session(*args, **kwargs):
+ session = real_session(
+ *args, aws_access_key_id="testing",
aws_secret_access_key="testing", **kwargs
+ )
+ real_client = session.client
+
+ def create_client(*args, **kwargs):
+ client = real_client(*args, **kwargs)
+ clients.append(client)
+ stubber = Stubber(client)
+ stubber.add_response(
+ "get_web_identity_token",
+ {
+ "WebIdentityToken": "aws-jwt",
+ "Expiration": datetime(2024, 1, 1, tzinfo=UTC),
+ },
+ {
+ "Audience": ["snowflakecomputing.com"],
+ "SigningAlgorithm": "ES384",
+ "DurationSeconds": 3600,
+ },
+ )
+ stubber.activate()
+ return client
+
+ session.client = create_client
+ return session
+
+ with mock.patch("boto3.Session", autospec=True,
side_effect=create_session):
+ assert _fetch_aws_identity_token() == "aws-jwt"
+ assert clients[0].meta.region_name == expected
+ assert clients[0].meta.endpoint_url ==
f"https://sts.{expected}.amazonaws.com"
+ if aws_region or default_region:
+ metadata.assert_not_called()
+ else:
+ metadata.return_value.retrieve_region.assert_called_once_with()
+
+ @mock.patch("botocore.utils.InstanceMetadataRegionFetcher", autospec=True)
+ @mock.patch("boto3.Session", autospec=True)
+ def test_aws_missing_region_raises(self, session, metadata, monkeypatch):
+ monkeypatch.delenv("AWS_REGION", raising=False)
+ monkeypatch.delenv("AWS_DEFAULT_REGION", raising=False)
+ metadata.return_value.retrieve_region.return_value = None
+ with pytest.raises(ValueError, match="AWS_REGION or
AWS_DEFAULT_REGION"):
+ _fetch_aws_identity_token()
+ session.assert_not_called()
+
+ @mock.patch("boto3.Session", autospec=True)
+ def test_aws_old_boto3_raises(self, session, monkeypatch):
+ monkeypatch.setenv("AWS_REGION", "eu-west-1")
+ session.return_value.client.return_value = mock.Mock(spec=[])
+ with pytest.raises(AirflowOptionalProviderFeatureException,
match="sts:GetWebIdentityToken"):
+ _fetch_aws_identity_token()
+
+ @pytest.mark.parametrize(
+ ("fetch", "module", "installation"),
+ [
+ pytest.param(_fetch_aws_identity_token, "boto3", "Install boto3.",
id="aws"),
+ pytest.param(
+ _fetch_azure_identity_token,
+ "azure.identity",
+ "Install apache-airflow-providers-snowflake[microsoft.azure].",
+ id="azure",
+ ),
+ pytest.param(
+ _fetch_gcp_identity_token, "google.auth.compute_engine",
"Install google-auth.", id="gcp"
+ ),
+ ],
+ )
+ def test_missing_cloud_library_raises(self, fetch, module, installation):
+ with mock.patch.dict(sys.modules, {module: None}):
+ with pytest.raises(AirflowOptionalProviderFeatureException) as exc:
+ fetch()
+ assert installation in str(exc.value)
+
+ @pytest.mark.parametrize(("workload", "file_exists"), [(False, False),
(True, False), (True, True)])
+ @pytest.mark.parametrize("fails", [False, True])
+ @mock.patch("azure.identity.ManagedIdentityCredential", autospec=True)
+ @mock.patch("azure.identity.WorkloadIdentityCredential", autospec=True)
+ def test_azure_credential_selection_and_cleanup(
+ self, aks, managed, workload, file_exists, fails, monkeypatch, tmp_path
+ ):
+ from azure.core.credentials import AccessToken
+
+ token_file = tmp_path / "azure-token"
+ if file_exists:
+ token_file.write_text("projected-token")
+ for name in ("AZURE_CLIENT_ID", "AZURE_TENANT_ID",
"AZURE_FEDERATED_TOKEN_FILE"):
+ if workload:
+ monkeypatch.setenv(
+ name, str(token_file) if name ==
"AZURE_FEDERATED_TOKEN_FILE" else "configured"
+ )
+ else:
+ monkeypatch.delenv(name, raising=False)
+ monkeypatch.setenv("MANAGED_IDENTITY_CLIENT_ID", "mi-client-id")
+ selected, unused = (aks, managed) if workload else (managed, aks)
+ selected.return_value.get_token.return_value =
AccessToken("azure-token", 3600)
+ if fails:
+ selected.return_value.get_token.side_effect =
RuntimeError("unavailable")
+ with pytest.raises(RuntimeError, match="unavailable"):
+ _fetch_azure_identity_token()
+ else:
+ assert _fetch_azure_identity_token() == "azure-token"
+ if workload:
+ selected.assert_called_once_with()
+ else:
+ selected.assert_called_once_with(client_id="mi-client-id")
+ unused.assert_not_called()
+ selected.return_value.get_token.assert_called_once_with(
+ "api://fd3f753b-eed3-462c-b6a7-a4b5bb650aad/.default"
+ )
+ selected.return_value.close.assert_called_once_with()
+
+ @pytest.fixture
+ def google_auth(self):
+ pytest.importorskip("google.auth")
+
+ @pytest.mark.usefixtures("google_auth")
+ @mock.patch("google.auth.compute_engine.IDTokenCredentials", autospec=True)
+ @mock.patch("google.auth.transport.requests.Request", autospec=True)
+ def test_gcp_uses_metadata_identity_even_with_application_credentials(
+ self, request, credential, monkeypatch
+ ):
+ monkeypatch.setenv("GOOGLE_APPLICATION_CREDENTIALS",
"/does/not/exist.json")
+ credential.return_value.token = "gcp-id-token"
+ assert _fetch_gcp_identity_token() == "gcp-id-token"
+ credential.assert_called_once_with(
+ request.return_value, "snowflakecomputing.com",
use_metadata_identity_endpoint=True
+ )
+
credential.return_value.refresh.assert_called_once_with(request.return_value)
+
+
class TestSnowflakeRestTokenRepr:
def test_repr_does_not_leak_the_token(self):
"""The default dataclass repr would print the bearer token; that must
not happen, since