azagrebin commented on a change in pull request #8759: [FLINK-12868][yarn] Fix
yarn cluster can not be deployed if plugins dir does not exist
URL: https://github.com/apache/flink/pull/8759#discussion_r294351611
##########
File path:
flink-yarn/src/main/java/org/apache/flink/yarn/AbstractYarnClusterDescriptor.java
##########
@@ -1520,25 +1520,27 @@ protected void
addEnvironmentFoldersToShipFiles(Collection<File> effectiveShipFi
// (for other files users explicitly set the ship files)
String libDir = System.getenv().get(ENV_FLINK_LIB_DIR);
if (libDir != null) {
- addEnvFolderToShipFiles(effectiveShipFiles, libDir,
ENV_FLINK_LIB_DIR);
+ File directoryFile = new File(libDir);
+ if (directoryFile.isDirectory()) {
+ effectiveShipFiles.add(directoryFile);
+ } else {
+ throw new YarnDeploymentException("The
environment variable '" + ENV_FLINK_LIB_DIR +
+ "' is set to '" + libDir + "' but the
directory doesn't exist.");
+ }
} else if (this.shipFiles.isEmpty()) {
LOG.warn("Environment variable '{}' not set and ship
files have not been provided manually. " +
"Not shipping any library files.",
ENV_FLINK_LIB_DIR);
}
String pluginsDir = System.getenv().get(ENV_FLINK_PLUGINS_DIR);
if (pluginsDir != null) {
- addEnvFolderToShipFiles(effectiveShipFiles, pluginsDir,
ENV_FLINK_PLUGINS_DIR);
- }
- }
-
- private void addEnvFolderToShipFiles(Collection<File>
effectiveShipFiles, String directory, String environmentVariableName) {
- File directoryFile = new File(directory);
- if (directoryFile.isDirectory()) {
- effectiveShipFiles.add(directoryFile);
- } else {
- throw new YarnDeploymentException("The environment
variable '" + environmentVariableName +
- "' is set to '" + directory + "' but the
directory doesn't exist.");
+ File directoryFile = new File(pluginsDir);
Review comment:
nit: I would move `ENV_FLINK_LIB_DIR `/ `ENV_FLINK_PLUGINS_DIR ` cases to
separate methods even if deduplication is under question
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
With regards,
Apache Git Services
