KalyanKadiyala commented on issue #13760: URL: https://github.com/apache/iceberg/issues/13760#issuecomment-3181078663
Hi @murphycrosby , Sorry for the delay in response. Few things that cross my mind here - - ADLS g2 (ABFS driver) uses OAuth to authenticate, where the appropriate token is fetched using the configured token provider. IMHO - the configured token provider isn't your problem here. - Question to validate - "is the configured Service Principal Id has permissions to the path", which isn't the scenario here based on the returned HTTP-403 code. For reference - https://docs.databricks.com/_extras/notebooks/source/adls-gen2-service-principal.html. - As you observe from the log-sequence/stack traces, Iceberg code paths or the DSv2 table lookup aren't the root cause here. They are suffering due to 403 error which prevented successful discovery. To isolate the issue - - I'm assuming you are running this job with required FS configurations (i.e., valid allowed Application Id). From the following log entry, it appears the effective identity isn't able to traverse the paths (needs list permission at the container entry, and to each nested folder there-in). > 25/08/08 16:21:52 WARN DeltaTableUtils: _**Access error while exploring path hierarchy**_ for a delta log.original path=abfss://[email protected]/lakehouse/domain/thing/blah, path with error=abfss://[email protected]/. - Based on your initial detail - where you had successful listing of directory paths, will be good to validate the effective identity i.e., SPN or end user id that's consumed to run directory listings. - Review grant permissions to the SPN to traverse the paths. Identity used by dbutils is most likely the end user identity with permissions to the paths listed. AFAIK, it doesn't use SPN. Considering this not being a Iceberg specific issue, IMHO reach out to the support channels on ADB. Best, Kalyan -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
