[
https://issues.apache.org/jira/browse/HADOOP-17105?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Steve Loughran resolved HADOOP-17105.
-------------------------------------
Fix Version/s: 3.3.1
Resolution: Fixed
patch merged to 3.3 and trunk branches. Thanks!
> S3AFS globStatus attempts to resolve symlinks
> ---------------------------------------------
>
> Key: HADOOP-17105
> URL: https://issues.apache.org/jira/browse/HADOOP-17105
> Project: Hadoop Common
> Issue Type: Bug
> Components: fs/s3
> Reporter: Jimmy Zuber
> Assignee: Jimmy Zuber
> Priority: Minor
> Fix For: 3.3.1
>
>
> The S3AFileSystem implementation of the globStatus API has a setting
> configured to resolve symlinks. Under certain circumstances, this will cause
> additional file existence checks to be performed in order to determine if a
> FileStatus signifies a symlink. As symlinks are not supported in
> S3AFileSystem, these calls are unnecessary.
> Code snapshot (permalink):
> [https://github.com/apache/hadoop/blob/2a67e2b1a0e3a5f91056f5b977ef9c4c07ba6718/hadoop-tools/hadoop-aws/src/main/java/org/apache/hadoop/fs/s3a/S3AFileSystem.java#L4002]
> Causes additional getFileStatus call here (permalink):
> [https://github.com/apache/hadoop/blob/1921e94292f0820985a0cfbf8922a2a1a67fe921/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/Globber.java#L308]
> Current code snippet:
> {code:java}
> /**
> * Override superclass so as to disable symlink resolution and so avoid
> * some calls to the FS which may have problems when the store is being
> * inconsistent.
> * {@inheritDoc}
> */
> @Override
> public FileStatus[] globStatus(
> final Path pathPattern,
> final PathFilter filter)
> throws IOException {
> entryPoint(INVOCATION_GLOB_STATUS);
> return Globber.createGlobber(this)
> .withPathPattern(pathPattern)
> .withPathFiltern(filter)
> .withResolveSymlinks(true)
> .build()
> .glob();
> }
> {code}
>
> The fix should be pretty simple, just flip "withResolveSymlinks" to false.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-dev-h...@hadoop.apache.org
