Chinmay1220 commented on code in PR #68344:
URL: https://github.com/apache/airflow/pull/68344#discussion_r4608463074
##########
providers/microsoft/azure/src/airflow/providers/microsoft/azure/fs/adls.py:
##########
@@ -47,12 +68,20 @@ def get_fs(conn_id: str | None, storage_options: dict[str,
Any] | None = None) -
if connection_string:
return AzureBlobFileSystem(connection_string=connection_string)
- options: dict[str, Any] = {
- "account_url": parse_blob_account_url(conn.host, conn.login),
- }
-
# mirror handling of custom field "client_secret_auth_config" from extras.
Ignore if missing as AzureBlobFileSystem can handle.
tenant_id = get_field(conn_id=conn_id, conn_type=conn_type, extras=extras,
field_name="tenant_id")
+ options: dict[str, Any] = {}
+ account_name = _bare_account_name_from_host(conn.host)
+ if conn_type == "adls" and account_name:
+ # An ADLS Active Directory connection uses ``login`` as the client_id,
so a bare host
+ # is the storage account name rather than part of an account URL.
+ options["account_name"] = account_name
+ elif conn_type != "adls" or conn.host or not tenant_id:
+ # WASB, full-URL, and custom-endpoint connections keep the historical
account_url path.
+ options["account_url"] = parse_blob_account_url(conn.host, conn.login)
+ # Otherwise (adls + no host + a tenant_id) neither account_name nor
account_url is set, so
+ # adlfs falls back to DefaultAzureCredential using the client_id/secret
assembled below.
Review Comment:
Fixed in f2237e3. The comment now says no account is set on this path, so it
has to come from `account_name` in Extra or `AZURE_STORAGE_ACCOUNT_NAME`, which
is adlfs's fallback. The no-host test case has a note that it pins the options
passed, not a configuration that connects on its own.
##########
providers/microsoft/azure/docs/filesystems/adls.rst:
##########
@@ -0,0 +1,77 @@
+ .. Licensed to the Apache Software Foundation (ASF) under one
+ or more contributor license agreements. See the NOTICE file
+ distributed with this work for additional information
+ regarding copyright ownership. The ASF licenses this file
+ to you under the Apache License, Version 2.0 (the
+ "License"); you may not use this file except in compliance
+ with the License. You may obtain a copy of the License at
+
+ .. http://www.apache.org/licenses/LICENSE-2.0
+
+ .. Unless required by applicable law or agreed to in writing,
+ software distributed under the License is distributed on an
+ "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ KIND, either express or implied. See the License for the
+ specific language governing permissions and limitations
+ under the License.
+
+Azure Data Lake Storage Gen2 Filesystem
+=======================================
+
+The Azure Data Lake Storage Gen2 filesystem provides access to ADLS Gen2
+through Airflow's ObjectStoragePath interface.
+
+Supported URL formats:
+
+* ``abfs://connection_id@container/path/to/file``
+* ``abfss://connection_id@container/path/to/file``
+* ``adl://connection_id@container/path/to/file``
+
+Connection Configuration
+------------------------
+
+Create an ADLS Gen2 connection in Airflow with the following parameters:
+
+* **Connection Type**: adls
+* **Host**: Storage account name, for example ``mystorageaccount``. For a
private endpoint or
+ custom domain, set the ``account_host`` field below rather than a full URL
here.
+* **Login**: Client ID for Active Directory authentication
+* **Password**: Client secret for Active Directory authentication, or account
key for shared key authentication
+
+The connection form provides additional configuration fields:
Review Comment:
Fixed in f2237e3. The list is now split: the form fields (`tenant_id`,
`connection_string`, plus `managed_identity_client_id` and
`workload_identity_tenant_id`, since `get_fs()` reads them), then
`account_name` and `account_host` as Extra keys. The Host bullet also says
`account_host` goes in Extra.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]