[GitHub] [hadoop] steveloughran commented on issue #1838: HADOOP-16711 Add way to skip verifyBuckets check in S3A fs init()

[GitBox]

steveloughran commented on issue #1838: HADOOP-16711 Add way to skip 
verifyBuckets check in S3A fs init()
URL: https://github.com/apache/hadoop/pull/1838#issuecomment-585351633
 
 
   Also once I fix that by adding a trailing /, the getFileStatus("/") fails to 
raise an FNFE, which is because S3Guard is enabled for all buckets on my test 
setup, *and s3guard DDB will create a stub FS Status on a root entry*. 
   
   ```
   java.lang.AssertionError: Expected a java.io.FileNotFoundException to be 
thrown, but got the result: : 
S3AFileStatus{path=s3a://random-bucket-11df1b68-2535-4a9b-9fd5-c6a4d5a6c192/; 
isDirectory=true; modification_time=0; access_time=0; owner=stevel; 
group=stevel; permission=rwxrwxrwx; isSymlink=false; hasAcl=false; 
isEncrypted=true; isErasureCoded=false} isEmptyDirectory=UNKNOWN eTag=null 
versionId=null
   
        at 
org.apache.hadoop.test.LambdaTestUtils.intercept(LambdaTestUtils.java:499)
        at 
org.apache.hadoop.test.LambdaTestUtils.intercept(LambdaTestUtils.java:384)
        at 
org.apache.hadoop.test.LambdaTestUtils.intercept(LambdaTestUtils.java:453)
        at 
org.apache.hadoop.fs.s3a.ITestS3ABucketExistence.testNoBucketProbing(ITestS3ABucketExistence.java:59)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at 
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
        at 
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:498)
        at 
org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:50)
        at 
org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
        at 
org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:47)
        at 
org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
        at 
org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26)
        at 
org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27)
        at org.junit.rules.TestWatcher$1.evaluate(TestWatcher.java:55)
        at 
org.junit.internal.runners.statements.FailOnTimeout$CallableStatement.call(FailOnTimeout.java:298)
        at 
org.junit.internal.runners.statements.FailOnTimeout$CallableStatement.call(FailOnTimeout.java:292)
        at java.util.concurrent.FutureTask.run$$$capture(FutureTask.java:266)
        at java.util.concurrent.FutureTask.run(FutureTask.java)
        at java.lang.Thread.run(Thread.java:748)
   ```
   
   We could consider shortcutting some of the getFileStatus queries against / 
in S3A FS itself -it's always a dir after all

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org