liuml07 commented on a change in pull request #1861: HADOOP-13230. Optionally
retain directory markers
URL: https://github.com/apache/hadoop/pull/1861#discussion_r410645571
##########
File path:
hadoop-tools/hadoop-aws/src/main/java/org/apache/hadoop/fs/s3a/S3AFileSystem.java
##########
@@ -1265,18 +1282,30 @@ public WriteOperationHelper getWriteOperationHelper() {
* is not a directory.
*/
@Override
- public FSDataOutputStream createNonRecursive(Path path,
+ public FSDataOutputStream createNonRecursive(Path p,
FsPermission permission,
EnumSet<CreateFlag> flags,
int bufferSize,
short replication,
long blockSize,
Progressable progress) throws IOException {
entryPoint(INVOCATION_CREATE_NON_RECURSIVE);
+ final Path path = makeQualified(p);
Path parent = path.getParent();
- if (parent != null) {
- // expect this to raise an exception if there is no parent
- if (!getFileStatus(parent).isDirectory()) {
+ // expect this to raise an exception if there is no parent dir
+ if (parent != null && !parent.isRoot()) {
+ S3AFileStatus status;
+ try {
+ // optimize for the directory existing: Call list first
+ status = innerGetFileStatus(parent, false,
+ StatusProbeEnum.DIRECTORIES);
+ } catch (FileNotFoundException e) {
+ // no dir, fall back to looking for a file
+ // (failure condition if true)
+ status = innerGetFileStatus(parent, false,
+ StatusProbeEnum.HEAD_ONLY);
Review comment:
I know they are the same, but is `StatusProbeEnum.FILE` a bit better here
since the status is for heading a file.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
