Github user jinossy commented on the pull request:
https://github.com/apache/tajo/pull/1024#issuecomment-222400539
Guys,
I found the improved performance reason. If there is not set the delimiter,
the listObjects return a list of summary information about the objects. it
reduce the requests to aws.
please see the below comments
```
http://docs.aws.amazon.com/AWSJavaSDK/latest/javadoc/com/amazonaws/services/s3/model/ListObjectsRequest.html
Contains options to return a list of summary information about the objects
in the specified bucket. Depending on the request parameters, additional
information is returned, such as common prefixes if a delimiter was specified.
List results are always returned in lexicographic (alphabetical) order.
Buckets can contain a virtually unlimited number of keys, and the complete
results of a list query can be extremely large. To manage large result sets,
Amazon S3 uses pagination to split them into multiple responses. Always check
the ObjectListing.isTruncated() method to see if the returned listing is
complete, or if callers need to make additional calls to get more results.
Alternatively, use the AmazonS3Client.listNextBatchOfObjects(ObjectListing)
method as an easy way to get the next page of object listings.
Calling setDelimiter(String) sets the delimiter, allowing groups of keys
that share the delimiter-terminated prefix to be included in the returned
listing. This allows applications to organize and browse their keys
hierarchically, similar to how a file system organizes files into directories.
These common prefixes can be retrieved through the
ObjectListing.getCommonPrefixes() method.
For example, consider a bucket that contains the following keys:
"foo/bar/baz"
"foo/bar/bash"
"foo/bar/bang"
"foo/boo"
If calling listObjects with a prefix value of "foo/" and a delimiter value
of "/" on this bucket, an ObjectListing is returned that contains one key
("foo/boo") and one entry in the common prefixes list ("foo/bar/"). To see
deeper into the virtual hierarchy, make another call to listObjects setting the
prefix parameter to any interesting common prefix to list the individual keys
under that prefix.
The total number of keys in a bucket doesn't substantially affect list
performance, nor does the presence or absence of additional request parameters.
```
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
