djouallah opened a new issue, #2311: URL: https://github.com/apache/iceberg-go/issues/2311
### Apache Iceberg version main (development), b7792e9e8799 ### Please describe the bug 🐞 `io/gocloud/azure` builds the container URL from the account name and `adls.endpoint` only. It drops the storage domain in the location's host. So when `adls.endpoint` is not set, every request goes to `<account>.blob.core.windows.net`, whatever the `abfss://` URI says. On Microsoft Fabric OneLake a table lives at `abfss://<workspace>@onelake.dfs.fabric.microsoft.com/<lakehouse>/Tables/<ns>/<table>`. Reading it sends `GET https://onelake.blob.core.windows.net/...`, which fails with 401. I expect the same for any account outside the public-cloud domain (`*.dfs.core.chinacloudapi.cn`, `*.dfs.core.usgovcloudapi.net`), but have only tested OneLake. From Go it works once the caller sets `adls.endpoint=blob.fabric.microsoft.com`. The `iceberg` CLI has no way to pass IO properties, though, so it can't read OneLake tables at all. **Reproduce** (REST catalog `https://onelake.table.fabric.microsoft.com/iceberg`, after `az login`): ``` iceberg --uri https://onelake.table.fabric.microsoft.com/iceberg --token "$TOKEN" --warehouse "<ws>/<lh>" describe table ns.t # works: catalog only iceberg ... files ns.t iceberg ... compact analyze ns.t iceberg ... clean-orphan-files --dry-run ns.t ``` The last three fail the same way: ``` could not open manifest file: open abfss://<ws>@onelake.dfs.fabric.microsoft.com/<lh>/Tables/ns/t/metadata/snap-...avro: blob (key "<lh>/Tables/ns/t/metadata/snap-...avro") (code=Unknown): GET https://onelake.blob.core.windows.net/<ws>/<lh>%2FTables%2Fns%2Ft%2Fmetadata%2Fsnap-...avro RESPONSE 401: Server failed to authenticate the request. ERROR CODE: InvalidAuthenticationInfo <AuthenticationErrorDetail>Issuer validation failed. Issuer did not match.</AuthenticationErrorDetail> ``` **Where.** In `createAzureBucketWithOptions` (`io/gocloud/azure/azure.go`), `endpoint := props[io.ADLSEndpoint]` goes straight into `createContainerURL(location.accountName, protocol, endpoint, ...)`, so an empty endpoint means the azureblob default domain. `newAdlsLocation` does parse the host (`acct.Host`), but uses it only as the SAS-token key. **Possible fix.** When `adls.endpoint` is not set, take the storage domain from the URI host after the account label, with `dfs.` mapped to `blob.`: `onelake.dfs.fabric.microsoft.com` gives `blob.fabric.microsoft.com`, and `acct.dfs.core.windows.net` still gives `blob.core.windows.net`. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
