ibzib commented on a change in pull request #13835:
URL: https://github.com/apache/beam/pull/13835#discussion_r567007332
##########
File path:
sdks/java/extensions/sql/src/main/java/org/apache/beam/sdk/extensions/sql/impl/JavaUdfLoader.java
##########
@@ -111,20 +115,39 @@ private File downloadFile(String inputPath, String
mimeType) throws IOException
}
}
+ private File getLocalJar(String inputJarPath) throws IOException {
+ if (!jarCache.containsKey(inputJarPath)) {
+ jarCache.put(inputJarPath, downloadFile(inputJarPath,
"application/java-archive"));
+ }
+ return jarCache.get(inputJarPath);
+ }
+
private ClassLoader createClassLoader(String inputJarPath) throws
IOException {
- File tmpJar = downloadFile(inputJarPath, "application/java-archive");
+ File tmpJar = getLocalJar(inputJarPath);
return new URLClassLoader(new URL[] {tmpJar.toURI().toURL()});
}
+ public ClassLoader createClassLoader(List<String> inputJarPaths) throws
IOException {
+ List<File> localJars = new ArrayList<>();
+ for (String inputJar : inputJarPaths) {
+ localJars.add(getLocalJar(inputJar));
Review comment:
Yep, done. (I originally tried to do this the functional way using
streams, but Java doesn't really let you throw checked exceptions from lambdas.
😢 )
##########
File path:
sdks/java/extensions/sql/src/main/java/org/apache/beam/sdk/extensions/sql/impl/rel/BeamCalcRel.java
##########
@@ -217,10 +226,14 @@ public Calc copy(RelTraitSet traitSet, RelNode input,
RexProgram program) {
Types.lookupMethod(DoFn.ProcessContext.class, "output",
Object.class),
output))));
- CalcFn calcFn = new CalcFn(builder.toBlock().toString(), outputSchema);
+ CalcFn calcFn = new CalcFn(builder.toBlock().toString(), outputSchema,
getJarPaths(program));
// validate generated code
- calcFn.compile();
+ try {
+ calcFn.compile();
+ } catch (IOException e) {
Review comment:
done
##########
File path:
sdks/java/extensions/sql/src/main/java/org/apache/beam/sdk/extensions/sql/impl/rel/BeamCalcRel.java
##########
@@ -274,6 +298,25 @@ public void processElement(ProcessContext c) {
}
}
+ private static List<String> getJarPaths(RexProgram program) {
+ List<String> jarPaths = new ArrayList<>();
Review comment:
done
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
