kl0u commented on a change in pull request #13024:
URL: https://github.com/apache/flink/pull/13024#discussion_r473046293
##########
File path:
flink-clients/src/main/java/org/apache/flink/client/cli/CliFrontend.java
##########
@@ -219,34 +219,49 @@ protected void run(String[] args) throws Exception {
final ProgramOptions programOptions =
ProgramOptions.create(commandLine);
- final PackagedProgram program =
- getPackagedProgram(programOptions);
+ final List<URL> jobJars =
getJobJarAndDependencies(programOptions);
- final List<URL> jobJars = program.getJobJarAndDependencies();
final Configuration effectiveConfiguration =
getEffectiveConfiguration(
activeCommandLine, commandLine, programOptions,
jobJars);
LOG.debug("Effective executor configuration: {}",
effectiveConfiguration);
+ final PackagedProgram program =
getPackagedProgram(programOptions, effectiveConfiguration);
+
try {
executeProgram(effectiveConfiguration, program);
} finally {
program.deleteExtractedLibraries();
}
}
- private PackagedProgram getPackagedProgram(ProgramOptions
programOptions) throws ProgramInvocationException, CliArgsException {
+ /**
+ * Get all provided libraries needed to run the program from the
ProgramOptions.
+ */
+ public List<URL> getJobJarAndDependencies(ProgramOptions
programOptions) throws FileNotFoundException, ProgramInvocationException {
Review comment:
Here this can be package-private and not public. Also we should annotate
it as `@VisibleForTesting`.
##########
File path:
flink-clients/src/main/java/org/apache/flink/client/cli/CliFrontend.java
##########
@@ -219,34 +219,49 @@ protected void run(String[] args) throws Exception {
final ProgramOptions programOptions =
ProgramOptions.create(commandLine);
- final PackagedProgram program =
- getPackagedProgram(programOptions);
+ final List<URL> jobJars =
getJobJarAndDependencies(programOptions);
- final List<URL> jobJars = program.getJobJarAndDependencies();
final Configuration effectiveConfiguration =
getEffectiveConfiguration(
activeCommandLine, commandLine, programOptions,
jobJars);
LOG.debug("Effective executor configuration: {}",
effectiveConfiguration);
+ final PackagedProgram program =
getPackagedProgram(programOptions, effectiveConfiguration);
+
try {
executeProgram(effectiveConfiguration, program);
} finally {
program.deleteExtractedLibraries();
}
}
- private PackagedProgram getPackagedProgram(ProgramOptions
programOptions) throws ProgramInvocationException, CliArgsException {
+ /**
+ * Get all provided libraries needed to run the program from the
ProgramOptions.
+ */
+ public List<URL> getJobJarAndDependencies(ProgramOptions
programOptions) throws FileNotFoundException, ProgramInvocationException {
+ String entryPointClass =
programOptions.getEntryPointClassName();
+ String jarFilePath = programOptions.getJarFilePath();
+
+ File jarFile = jarFilePath != null ? getJarFile(jarFilePath) :
null;
+
+ return PackagedProgram
+ .getJobJarAndDependencies(jarFile != null ? jarFile :
null, entryPointClass);
Review comment:
This can become `return
PackagedProgram.getJobJarAndDependencies(jarFile, entryPointClass);`
##########
File path:
flink-clients/src/main/java/org/apache/flink/client/cli/CliFrontend.java
##########
@@ -219,34 +219,49 @@ protected void run(String[] args) throws Exception {
final ProgramOptions programOptions =
ProgramOptions.create(commandLine);
- final PackagedProgram program =
- getPackagedProgram(programOptions);
+ final List<URL> jobJars =
getJobJarAndDependencies(programOptions);
- final List<URL> jobJars = program.getJobJarAndDependencies();
final Configuration effectiveConfiguration =
getEffectiveConfiguration(
activeCommandLine, commandLine, programOptions,
jobJars);
LOG.debug("Effective executor configuration: {}",
effectiveConfiguration);
+ final PackagedProgram program =
getPackagedProgram(programOptions, effectiveConfiguration);
+
try {
executeProgram(effectiveConfiguration, program);
} finally {
program.deleteExtractedLibraries();
}
}
- private PackagedProgram getPackagedProgram(ProgramOptions
programOptions) throws ProgramInvocationException, CliArgsException {
+ /**
+ * Get all provided libraries needed to run the program from the
ProgramOptions.
+ */
+ public List<URL> getJobJarAndDependencies(ProgramOptions
programOptions) throws FileNotFoundException, ProgramInvocationException {
+ String entryPointClass =
programOptions.getEntryPointClassName();
+ String jarFilePath = programOptions.getJarFilePath();
+
+ File jarFile = jarFilePath != null ? getJarFile(jarFilePath) :
null;
+
+ return PackagedProgram
+ .getJobJarAndDependencies(jarFile != null ? jarFile :
null, entryPointClass);
+ }
+
+ private PackagedProgram getPackagedProgram(
+ ProgramOptions programOptions,
+ Configuration effectiveConfiguration) throws
ProgramInvocationException, CliArgsException {
PackagedProgram program;
try {
LOG.info("Building program from JAR file");
- program = buildProgram(programOptions);
+ program = buildProgram(programOptions,
effectiveConfiguration);
} catch (FileNotFoundException e) {
throw new CliArgsException("Could not build the program
from JAR file: " + e.getMessage(), e);
}
return program;
}
- private <T> Configuration getEffectiveConfiguration(
+ public <T> Configuration getEffectiveConfiguration(
Review comment:
Here this can be package-private and not public. Also we should annotate
it as @VisibleForTesting.
##########
File path:
flink-clients/src/main/java/org/apache/flink/client/program/PackagedProgram.java
##########
@@ -220,12 +220,47 @@ public ClassLoader getUserCodeClassLoader() {
* Returns all provided libraries needed to run the program.
*/
public List<URL> getJobJarAndDependencies() {
- List<URL> libs = new
ArrayList<URL>(this.extractedTempLibraries.size() + 1);
+ return getJobJarAndDependencies(jarFile,
extractedTempLibraries, isPython);
+ }
+
+ /**
+ * Returns all provided libraries needed to run the program.
+ */
+ public static List<URL> getJobJarAndDependencies(File jarFile,
@Nullable String entryPointClassName) throws ProgramInvocationException {
Review comment:
In this class you add this method only, so the rest of the changes can
be reverted, right. Essentially the `public List<URL>
getJobJarAndDependencies()` can stay as is currently on the master and you can
remove the `List<URL> getJobJarAndDependencies(URL jarFileUrl, List<File>
extractedTempLibraries, boolean isPython)`.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
