Thomas Marquardt created HADOOP-15547:
-----------------------------------------
Summary: WASB: listStatus performance
Key: HADOOP-15547
URL: https://issues.apache.org/jira/browse/HADOOP-15547
Project: Hadoop Common
Issue Type: Bug
Components: fs/azure
Affects Versions: 3.0.2, 2.9.1
Reporter: Thomas Marquardt
Assignee: Thomas Marquardt
The WASB implementation of Filesystem.listStatus is very slow due to O(n!)
algorithm to remove duplicates and uses too much memory due to the extra
conversion from BlobListItem to FileMetadata to FileStatus. It takes over 30
minutes to list 700,000 files.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
