[
https://issues.apache.org/jira/browse/HADOOP-16228?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16810714#comment-16810714
]
Steve Loughran commented on HADOOP-16228:
-----------------------------------------
Going to say this is a bug in the store then. You are going to have to stay
with the v1 listing
Against a third party store the diff between v2 and v1 matter less. For s3, v2
offers: less data => less overhead in the comms with S3, and the ability for
lists to return < 1000 records -which is used to stop problems when a listing
has to skip over a very number of version markers before returning real data.
Unless the store does versioning, that issue isn't going to surface.
> Throwing OOM exception for ListStatus (v2) when using S3A
> ----------------------------------------------------------
>
> Key: HADOOP-16228
> URL: https://issues.apache.org/jira/browse/HADOOP-16228
> Project: Hadoop Common
> Issue Type: Bug
> Components: fs/s3
> Affects Versions: 3.1.0
> Reporter: Ranith Sardar
> Priority: Major
>
> {code:java}
> @InterfaceStability.Unstable
> public static final String LIST_VERSION = "fs.s3a.list.version";
> @InterfaceStability.Unstable
> public static final int DEFAULT_LIST_VERSION = 2;
> {code}
> If the files in the bucket are more than 1k, it will throw OOM error for V2
> version.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
