felipecrv opened a new pull request, #39361:
URL: https://github.com/apache/arrow/pull/39361
### Rationale for this change
The `FileSystem` implementation based on Azure Blob Storage should implement
directory operations according to filesystem semantics. When Hierarchical
Namespace (HNS) is enabled, we can rely on Azure Data Lake Storage Gen 2 APIs
implementing the filesystem semantics for us, but when all we have is the Blobs
API, we should emulate it.
### What changes are included in this PR?
- Skip fewer tests
- Re-implement `GetFileInfo` using `ListBlobsByHierarchy` instead of
`ListBlobs`
- Re-implement `CreateDir` with an upfront HNS support check instead of
falling back to Blobs API after an error
- Add comprehensive tests to `CreateDir`
- Add `HasSubmitBatchBug` to check if a test inside any scenario is
affected by a certain Azurite issue
- Implement `DeleteDir` to work properly on flat namespace storage accounts
(non-HNS accounts)
-
### Are these changes tested?
Yes. By existing and new tests added by this PR itself.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
