steveloughran commented on pull request #2149: URL: https://github.com/apache/hadoop/pull/2149#issuecomment-672283597
latest patch tweaks markertool, but also adds pathcapabilities probes to the s3a store so you can see if an instance is (a) markeraware and (b) whether markers are being kept or deleted on a given path. Look at the docs for details. ``` s bin/hadoop jar $CLOUDSTORE pathcapability fs.s3a.capability.directory.marker.keep s3a://stevel-london/tables Probing s3a://stevel-london/tables for capability fs.s3a.capability.directory.marker.keep 2020-08-11 22:15:15,501 [main] INFO s3a.S3AFileSystem (S3Guard.java:logS3GuardDisabled(1152)) - S3Guard is disabled on this bucket: stevel-london 2020-08-11 22:15:15,506 [main] INFO impl.DirectoryPolicyImpl (DirectoryPolicyImpl.java:getDirectoryPolicy(143)) - Directory markers will be kept on authoritative paths Using filesystem s3a://stevel-london Path s3a://stevel-london/tables has capability fs.s3a.capability.directory.marker.keep ``` branch-3.2 doesn't support Path capabilities, but it does do it for stream capabilities; I'll extend cloudstore to have a `streamcapabilities` command too ---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
