Lari Sinisalo created JCLOUDS-1488:
--------------------------------------
Summary: Filesystem list call with prefix is slow in large
containers
Key: JCLOUDS-1488
URL: https://issues.apache.org/jira/browse/JCLOUDS-1488
Project: jclouds
Issue Type: Bug
Components: jclouds-blobstore
Affects Versions: 2.1.1
Environment: Java version: java version "1.8.0_131"
Operating system: Fedora 27 x86_64
Reporter: Lari Sinisalo
When the filesystem blobstore is used, running the following code takes very
long if there are a lot of files in the container:
{code}
ListContainerOptions options = new ListContainerOptions();
options.prefix("test-container-subdirectory/");
Set<? extends StorageMetadata> results =
blobStore.list("test-container",options);
{code}
See the attached Java source file for the full code.
On my system, running the attached Java code takes over 10 seconds to list a
single file if there are 500,000 files in the container outside that prefix.
Output from the attached code:
{code}
Number of blobs listed: 1
First listed blob: test-container-subdirectory/file-to-list
Time it took to list the blobs: 13256 ms
{code}
A more general version of this problem was reported previously in JCLOUDS-1371.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
