BukrosSzabolcs commented on a change in pull request #4066:
URL: https://github.com/apache/hbase/pull/4066#discussion_r801400439
##########
File path:
hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/SecureBulkLoadManager.java
##########
@@ -390,11 +400,16 @@ public String prepareBulkLoad(final byte[] family, final
String srcPath, boolean
LOG.debug("Moving " + p + " to " + stageP);
FileStatus origFileStatus = fs.getFileStatus(p);
origPermissions.put(srcPath, origFileStatus.getPermission());
+ origlSources.put(stageP.toString(), srcPath);
Review comment:
I do not think concurrency is an issue. `SecureBulkLoadListener` keeps
track of the moved files and a separate listener is created for each region in
each bulkLoad. So even with parallel bulkLoads they can not touch each other's
files. Using the same source folder can be an issue but it always was an issue.
I'm not sure I understand your comment about "always pass a custom staging
dir". The staging dir by default is different for each bulkLoad process. I
break this by introducing the "custom staging dir" which always points to the
live data folder as a workaround to skip moving hfiles to an actual staging dir
without loosing the existing error handling. We can't change it and decrease
the number of moves at the same time.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
