GitHub user keen85 added a comment to the discussion: Azure Storage Gen2 (hierarchical namespaces) - use DFS endpoint to improve performance
@tustvold. thanks for the explanation. Probably you are right for this scenario. Still, when having a container where there are 10 million files, scattered across 100 thousand folders, listing the content of a particular folder will be much faster using DFS endpoint, right? When using Blob endpoint, Storage service would scans the entire container index to find blobs matching the desired folder prefix. When using DFS endpoint, the Storage service can access the folder’s child entries directly - no search involved here. GitHub link: https://github.com/apache/arrow-rs-object-store/discussions/481#discussioncomment-14592011 ---- This is an automatically sent email for [email protected]. To unsubscribe, please send an email to: [email protected]
