steveloughran opened a new pull request #1845: HADOOP-16711 Add way to skip verifyBuckets check in S3A fs init() URL: https://github.com/apache/hadoop/pull/1845 see #1838 Adds a new exception UnknownStoreException to indicate "there's no store there" * raised in verify bucket existence checks * and when translating AWS exceptions into IOEs * The S3A retry policy fails fast on this * And s3GetFileStatus recognises the same failure and raises it Except when the metastore shortcircuits S3 IO, this means all operations against a nonexistent store will fail with a unique exception. ITestS3ABucketExistence is extended to * disable metastore (getFileStatus(/) was returning a value) * always create new instances * invoke all the operations which catch and swallow FNFEs (exists, isFile, isDir, delete) also: disable the probe for landsat-pds so that we get more test coverage of the option. Tested: S3 ireland w/ DDB, and the probe set to 0 by default, everywhere (No obvious speedup...)
---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected] With regards, Apache Git Services --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
