steveloughran commented on issue #1229: HADOOP-16490. Improve S3Guard handling of FNFEs in copy URL: https://github.com/apache/hadoop/pull/1229#issuecomment-520156284 Thinking about the new flag passed in to s3GetFileStatus to skip the object probe # it should be an enumset of probes: * object: check object * marker: look for directory marker * list: make a LIST call 1. when we create a file, we only want (marker, list) 2. for mkdir we would only do (object, list) and if those found nothing -create the marker, without ever looking for the marker. This will avoid creating a 404 on that probe. we can also optimise some directory ops by skipping the list and doing the real list if the (object, marker) calls failed
---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected] With regards, Apache Git Services --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
