cnauroth commented on code in PR #7651:
URL: https://github.com/apache/hadoop/pull/7651#discussion_r2067072005
##########
hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/LocalDirAllocator.java:
##########
@@ -393,6 +396,8 @@ int getCurrentDirectoryIndex() {
*/
public Path getLocalPathForWrite(String pathStr, long size,
Configuration conf, boolean checkWrite) throws IOException {
+ LOG.debug("searchng for directory for file at {}, size = {};
checkWrite={}",
Review Comment:
Minor typo: "searching".
##########
hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/LocalDirAllocator.java:
##########
@@ -406,27 +411,40 @@ public Path getLocalPathForWrite(String pathStr, long
size,
pathStr = pathStr.substring(1);
}
Path returnPath = null;
-
- if(size == SIZE_UNKNOWN) { //do roulette selection: pick dir with
probability
- //proportional to available size
- long[] availableOnDisk = new long[ctx.dirDF.length];
- long totalAvailable = 0;
-
- //build the "roulette wheel"
- for(int i =0; i < ctx.dirDF.length; ++i) {
- final DF target = ctx.dirDF[i];
- // attempt to recreate the dir so that getAvailable() is valid
- // if it fails, getAvailable() will return 0, so the dir will
- // be declared unavailable.
- // return value is logged at debug to keep spotbugs quiet.
- final boolean b = new File(target.getDirPath()).mkdirs();
- LOG.debug("mkdirs of {}={}", target, b);
- availableOnDisk[i] = target.getAvailable();
- totalAvailable += availableOnDisk[i];
+
+ final int dirCount = ctx.dirDF.length;
+ long[] availableOnDisk = new long[dirCount];
+ long totalAvailable = 0;
+
+ StringBuilder pathNames = new StringBuilder();
+
+ //build the "roulette wheel"
+ for (int i =0; i < dirCount; ++i) {
+ final DF target = ctx.dirDF[i];
+ // attempt to recreate the dir so that getAvailable() is valid
+ // if it fails, getAvailable() will return 0, so the dir will
+ // be declared unavailable.
+ // return value is logged at debug to keep spotbugs quiet.
+ final String name = target.getDirPath();
+ pathNames.append(" ").append(name);
+ final File dirPath = new File(name);
+ if (!dirPath.exists()) {
+ LOG.debug("creating buffer dir {}}", name);
+ dirPath.mkdirs();
Review Comment:
Do you want to debug-log the boolean return value? It should almost always
be `true` now that you added the `exists()` check, but it can still be `false`
for edge cases like permissions violations.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
