hequn8128 commented on a change in pull request #10126: [FLINK-14590][python]
Unify the working directory of Java process and Python process when submitting
python jobs via "flink run -py"
URL: https://github.com/apache/flink/pull/10126#discussion_r353705790
##########
File path:
flink-python/src/main/java/org/apache/flink/client/python/PythonDriverEnvUtils.java
##########
@@ -85,28 +84,24 @@ public void run() {
* Prepares PythonEnvironment to start python process.
*
* @param pythonLibFiles The dependent Python files.
+ * @param tmpDir The temporary directory which files will be copied to.
* @return PythonEnvironment the Python environment which will be
executed in Python process.
*/
- public static PythonEnvironment preparePythonEnvironment(List<Path>
pythonLibFiles)
- throws IOException, InterruptedException {
+ public static PythonEnvironment preparePythonEnvironment(
+ List<Path> pythonLibFiles,
+ String tmpDir) throws IOException,
InterruptedException {
PythonEnvironment env = new PythonEnvironment();
- // 1. setup temporary local directory for the user files
- String tmpDir = System.getProperty("java.io.tmpdir") +
- File.separator + "pyflink" + File.separator +
UUID.randomUUID();
+ tmpDir = new File(tmpDir).getAbsolutePath();
+ // 1. setup temporary local directory for the user files
Path tmpDirPath = new Path(tmpDir);
FileSystem fs = tmpDirPath.getFileSystem();
- if (fs.exists(tmpDirPath)) {
- fs.delete(tmpDirPath, true);
- }
fs.mkdirs(tmpDirPath);
- env.workingDirectory = tmpDirPath.toString();
-
- StringBuilder pythonPathEnv = new StringBuilder();
+ env.storageDirectory = tmpDir;
- pythonPathEnv.append(env.workingDirectory);
+ List<String> pythonPathList = new ArrayList<>();
Review comment:
Move the comment "// 2. append the internal lib files to PYTHONPATH." before
this line?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
With regards,
Apache Git Services
