sidseth commented on a change in pull request #1601: HADOOP-16635. S3A
innerGetFileStatus scans for directories-only still does a HEAD.
URL: https://github.com/apache/hadoop/pull/1601#discussion_r333749988
##########
File path:
hadoop-tools/hadoop-aws/src/main/java/org/apache/hadoop/fs/s3a/S3AFileSystem.java
##########
@@ -2730,39 +2730,41 @@ S3AFileStatus innerGetFileStatus(final Path f,
* @throws FileNotFoundException when the path does not exist
* @throws IOException on other problems.
*/
+ @VisibleForTesting
@Retries.RetryTranslated
- private S3AFileStatus s3GetFileStatus(final Path path,
- String key,
+ S3AFileStatus s3GetFileStatus(final Path path,
+ final String key,
final Set<StatusProbeEnum> probes,
final Set<Path> tombstones) throws IOException {
- if (!key.isEmpty() && probes.contains(StatusProbeEnum.Head)) {
- try {
- ObjectMetadata meta = getObjectMetadata(key);
-
- if (objectRepresentsDirectory(key, meta.getContentLength())) {
- LOG.debug("Found exact file: fake directory");
- return new S3AFileStatus(Tristate.TRUE, path, username);
- } else {
- LOG.debug("Found exact file: normal file");
+ if (!key.isEmpty()) {
+ if (probes.contains(StatusProbeEnum.Head) && !key.endsWith("/")) {
Review comment:
Not sure if this is ready to review.
With this change - if someone asks for a "HEAD" only, on a URL which happens
to have a trailing "/" nothing will be looked up.
IF HAED + DIRMARKER - then yes, at least 1 request will go out.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
