steveloughran commented on issue #952: HADOOP-16729 out of band deletes URL: https://github.com/apache/hadoop/pull/952#issuecomment-501752524 I have just pushed up a PR with changes. If I didn't need this in so that I could base my own PR atop it, I'd be seriously considering say "Use java 8 time over millis, as it guarantees that there won't be any bits of the code which assumes it is seconds" latter is awful about assessing the value of all enumerated moves & countermoves. In this instance, ### S3Guard.addAncestors() that walk up the tree uses an isDeleted() checl. Should that include TTL probes Note: I'm not going to do that now, because I've pushed that work further into DDB itself (needed to deal with scale issues); if changes are needed then they should be based off that patch. Please review that code and suggest the next action ### Imports keep that ordering of imports what we expect ``` java.* --- javax.* --- non-org.apache --- org.apache --- static. * ``` ` and in each one, in order. This is critical to help merge conflict. If a class already has inconsistent ordering, don't worry -but don't make it worse. Always check the imports in reviews for this reason. ### Tests I got a failure in a test run in teardown of `ITestDynamoDBMetadataStore`; happens if the FS didn't get created. Root cause shows up on some of the other test cases: `java.io.FileNotFoundException: DynamoDB table 's3guard-stevel-testing' is being deleted in region eu-west-1 at org.apache.hadoop.fs.s3a.s3guard.DynamoDBMetadataStore.initTable(DynamoDBMetadataStore.java:1293)` ``` at java.util.concurrent.FutureTask.run(FutureTask.java:266) at java.lang.Thread.run(Thread.java:748) [ERROR] testDeleteSubtreeHostPath(org.apache.hadoop.fs.s3a.s3guard.ITestDynamoDBMetadataStore) Time elapsed: 0.206 s <<< ERROR! java.lang.NullPointerException at org.apache.hadoop.fs.s3a.s3guard.MetadataStoreTestBase.strToPath(MetadataStoreTestBase.java:1035) at org.apache.hadoop.fs.s3a.s3guard.ITestDynamoDBMetadataStore.tearDown(ITestDynamoDBMetadataStore.java:219) at sun.reflect.GeneratedMethodAccessor24.invoke(Unknown Source) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:50) at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12) at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:47) at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:33) at org.junit.internal.runners.statements.FailOnTimeout$CallableStatement.call(FailOnTimeout.java:298) at org.junit.internal.runners.statements.FailOnTimeout$CallableStatement.call(FailOnTimeout.java:292) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at java.lang.Thread.run(Thread.java:748) ``` It's not caused by this PR and addressed in my rename PR, which doesn't do that cleanup unless fs != null. Also (I believe) unrelated, the Magic Committer ITest is playing up, and as the logs of the AM don't seem to be saved, can't quite debug. ``` [ERROR] testMRJob(org.apache.hadoop.fs.s3a.commit.magic.ITestMagicCommitMRJob) Time elapsed: 458.528 s <<< FAILURE! java.lang.AssertionError: No cleanup: unexpectedly found s3a://hwdev-steve-ireland-new/fork-0004/test/testMRJob/__magic as S3AFileStatus{path=s3a://hwdev-steve-ireland-new/fork-0004/test/testMRJob/__magic; isDirectory=true; modification_time=0; access_time=0; owner=stevel; group=stevel; permission=rwxrwxrwx; isSymlink=false; hasAcl=false; isEncrypted=true; isErasureCoded=false} isEmptyDirectory=UNKNOWN eTag=null versionId=null at org.junit.Assert.fail(Assert.java:88) at org.apache.hadoop.fs.contract.ContractTestUtils.assertPathDoesNotExist(ContractTestUtils.java:977) at org.apache.hadoop.fs.contract.AbstractFSContractTestBase.assertPathDoesNotExist(AbstractFSContractTestBase.java:305) at org.apache.hadoop.fs.s3a.commit.magic.ITestMagicCommitMRJob.customPostExecutionValidation(ITestMagicCommitMRJob.java:96) at org.apache.hadoop.fs.s3a.commit.AbstractITCommitMRJob.testMRJob(AbstractITCommitMRJob.java:162) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:50) at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12) at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:47) at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17) at org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26) at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27) at org.junit.rules.ExternalResource$1.evaluate(ExternalResource.java:48) at org.junit.rules.TestWatcher$1.evaluate(TestWatcher.java:55) at org.junit.internal.runners.statements.FailOnTimeout$CallableStatement.call(FailOnTimeout.java:298) at org.junit.internal.runners.statements.FailOnTimeout$CallableStatement.call(FailOnTimeout.java:292) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at java.lang.Thread.run(Thread.java:748) ``` Could this been related? Well, iff the path was still in S3 and the tombstone hadn't expired, yes. But I doubt that. Something I will worry about myself. I'll add some more assertions.... +that v1 listing again. ## Next Steps Gabor - pull this test down and do the scale test runs with all the options (auth, nonauth, local) and see how well it goes. If we dont' see problems or we believe they are transient unrelated issues, then I'll vote on this later on today
---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected] With regards, Apache Git Services --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
