dwsmith1983 opened a new issue, #5542:
URL: https://github.com/apache/datafusion-comet/issues/5542
### Describe the bug
Native Azure store construction (azure.rs create_store) seeds the builder
with MicrosoftAzureBuilder::from_env() and then layers the translated
fs.azure.* config on top. Same-key collisions resolve correctly (Hadoop wins),
but object_store's build() selects the credential by a fixed chain across
different keys — Bearer > AccessKey > WorkloadIdentity > ClientSecret > SAS >
CLI > IMDS — so ambient env vars that land higher in the chain than the
configured mechanism win. Hadoop's ABFS driver reads no environment variables
at all (verified: zero getenv references across all classes of hadoop-azure
3.3.4 and 3.4.1), so the two engines can resolve different identities for the
same table.
This is the flip side of the #4747 fix, which added the env pickup to make
workload identity work.
### Steps to reproduce
No Azure account needed; the credential the builder selects was extracted
directly from the store built by the exact create_store path, with controlled
env + Hadoop conf:
1. env AZURE_STORAGE_TOKEN=ambient-env-bearer-token + Hadoop
fs.azure.account.key.<acct> set → winning credential:
BearerToken("ambient-env-bearer-token"). The configured account key is ignored.
2. env AZURE_STORAGE_ACCOUNT_KEY=env-injected-key + Hadoop OAuth
client-credentials configured → winning credential:
AccessKey("env-injected-key"). The configured service principal is ignored.
3. env AZURE_FEDERATED_TOKEN_FILE=<webhook token file> (what the AKS
Workload Identity webhook injects into every annotated pod) + Hadoop OAuth
client id/tenant/secret configured → winning credential: a hybrid
WorkloadIdentityOAuth using Hadoop's client-id and tenant with the webhook's
token file; the explicitly configured client secret is silently dropped.
Controls: same-key collisions correctly resolve to the Hadoop value, and
env-only workload identity works as #4747 intended. Also latent:
AZURE_STORAGE_USE_EMULATOR, AZURE_ALLOW_HTTP, and AZURE_PROXY_URL parse from
env and reroute transport.
On a real cluster this surfaces as a 403 on tables the stock reader handles
fine or, when the ambient identity also has access, a read under a credential
the job never configured.
### Expected behavior
Explicit fs.azure.* auth configuration should fully determine the native
store's identity, exactly as it does for Hadoop's own reader. Ambient env
should only be consulted when Hadoop config specifies no auth mechanism
(preserving the #4747 workload-identity case).
### Additional context
Fix directions: when the translated Hadoop config carries an auth mechanism,
clear/skip conflicting env-derived keys before build(); or gate admission like
the S3/GCS credential-divergence checks in #5365. The Delta contrib is
currently unaffected (it declines all abfss:// scans, see #4993); the exposed
path is plain parquet on abfss.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
