Github user StefanRRichter commented on a diff in the pull request:
https://github.com/apache/flink/pull/5176#discussion_r160155461
--- Diff:
flink-runtime/src/main/java/org/apache/flink/runtime/blob/BlobUtils.java ---
@@ -127,21 +132,28 @@ private static BlobStoreService
createFileSystemBlobStore(Configuration configur
}
/**
- * Creates a local storage directory for a blob service under the given
parent directory.
+ * Creates a local storage directory for a blob service under the
configuration parameter given
+ * by {@link BlobServerOptions#STORAGE_DIRECTORY}. If this is
<tt>null</tt> or empty, we will
+ * fall back to Flink's temp directories (given by
+ * {@link org.apache.flink.configuration.CoreOptions#TMP_DIRS}) and
choose one among them at
+ * random.
*
- * @param basePath
- * base path, i.e. parent directory, of the storage
directory to use (if <tt>null</tt> or
- * empty, the path in <tt>java.io.tmpdir</tt> will be used)
+ * @param config
+ * Flink configuration
*
* @return a new local storage directory
*
* @throws IOException
* thrown if the local file storage cannot be created or
is not usable
*/
- static File initLocalStorageDirectory(String basePath) throws
IOException {
+ static File initLocalStorageDirectory(Configuration config) throws
IOException {
+
+ String basePath =
config.getString(BlobServerOptions.STORAGE_DIRECTORY);
+
File baseDir;
if (StringUtils.isNullOrWhitespaceOnly(basePath)) {
- baseDir = new
File(System.getProperty("java.io.tmpdir"));
+ final String[] tmpDirPaths =
TaskManagerServicesConfiguration.parseTempDirectories(config);
+ baseDir = new
File(tmpDirPaths[rnd.nextInt(tmpDirPaths.length)]);
--- End diff --
I wonder if this is the best strategy and if we should only return a single
directory, which might represent only a fraction of the available space.
However, I would assume in most cases user only specify a single directory
anyways. So what to do about this, is it feasible that the blob cache can deal
with all the offered directories?
---
