[
https://issues.apache.org/jira/browse/BEAM-6558?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Work on BEAM-6558 started by Kenneth Knowles.
---------------------------------------------
> Beam SQL transitive dependencies appear incomplete / broken
> -----------------------------------------------------------
>
> Key: BEAM-6558
> URL: https://issues.apache.org/jira/browse/BEAM-6558
> Project: Beam
> Issue Type: Bug
> Components: dsl-sql
> Affects Versions: 2.9.0
> Reporter: Kenneth Knowles
> Assignee: Kenneth Knowles
> Priority: Blocker
> Fix For: 2.10.0
>
>
> Reporting on the dev list:
> ----
> Hi, all,
> Newbie here trying to figure out how to use published
> beam-sdks-java-extensions-sql-2.9.0 in my own project.
> I tried to create a gradle project to use BeamSQL sdk libraries. Here is the
> build.gradle I have:
> {code:java}
> plugins {
> id 'java'
> }
> group 'com.mycompany.myproject'
> version '1.0-SNAPSHOT'
> sourceCompatibility = 1.8
> repositories {
> mavenCentral()
> }
> apply plugin: 'java'
> sourceSets.main.java.srcDirs = [
> 'src/main/java'
> ]
> dependencies {
> compile 'org.apache.beam:beam-sdks-java-core:2.9.0'
> compile 'org.apache.beam:beam-sdks-java-extensions-sql:2.9.0'
> compile 'com.google.code.findbugs:jsr305:3.0.2'
> runtime 'org.apache.beam:beam-runners-direct-java:2.9.0'
> testCompile group: 'junit', name: 'junit', version: '4.12'
> }
> // Run basic SQL example
> task runBasicExample(type: JavaExec) {
> description = "Run basic SQL example"
> main = "com.mycompany.myproject.streamsql.examples.BeamSQLExample"
> classpath = sourceSets.main.runtimeClasspath
> args = ["--runner=DirectRunner"]
> println classpath.getAsPath()
> println args
> }
> {code}
> The example BeamSQLExample is just copied from
> [https://github.com/apache/beam/blob/master/sdks/java/extensions/sql/src/main/java/org/apache/beam/sdk/extensions/sql/example/BeamSqlExample.java].
> I was able to compile the example in JDK8. However, when I tried to run it, I
> hit the following exception:
> {code:java}
> Exception in thread "main" java.util.ServiceConfigurationError:
> org.apache.beam.sdk.extensions.sql.impl.udf.BeamBuiltinFunctionProvider:
> Provider org.apache.beam.sdk.extensions.sql.impl.udf.BuiltinStringFunctions
> could not be instantiated
> at java.util.ServiceLoader.fail(ServiceLoader.java:232)
> at java.util.ServiceLoader.access$100(ServiceLoader.java:185)
> at
> java.util.ServiceLoader$LazyIterator.nextService(ServiceLoader.java:384)
> at java.util.ServiceLoader$LazyIterator.next(ServiceLoader.java:404)
> at java.util.ServiceLoader$1.next(ServiceLoader.java:480)
> at
> org.apache.beam.sdk.extensions.sql.impl.BeamSqlEnv.loadBeamBuiltinFunctions(BeamSqlEnv.java:128)
> at
> org.apache.beam.sdk.extensions.sql.SqlTransform.expand(SqlTransform.java:94)
> at
> org.apache.beam.sdk.extensions.sql.SqlTransform.expand(SqlTransform.java:76)
> at org.apache.beam.sdk.Pipeline.applyInternal(Pipeline.java:537)
> at org.apache.beam.sdk.Pipeline.applyTransform(Pipeline.java:471)
> at org.apache.beam.sdk.values.PCollection.apply(PCollection.java:357)
> at
> com.mycompany.myprject.streamsql.examples.BeamSQLExample.main(BeamSQLExample.java:72)
> Caused by: java.lang.NoClassDefFoundError:
> org/apache/commons/codec/DecoderException
> at java.lang.Class.getDeclaredConstructors0(Native Method)
> at java.lang.Class.privateGetDeclaredConstructors(Class.java:2671)
> at java.lang.Class.getConstructor0(Class.java:3075)
> at java.lang.Class.newInstance(Class.java:412)
> at
> java.util.ServiceLoader$LazyIterator.nextService(ServiceLoader.java:380)
> ... 9 more
> Caused by: java.lang.ClassNotFoundException:
> org.apache.commons.codec.DecoderException
> at java.net.URLClassLoader.findClass(URLClassLoader.java:382)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
> at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:349)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
> ... 14 more
> {code}
> When I traced into the code, it turns out that the failure occurs when the
> classloader tries to get the default constructor w/o any parameters for
> BuiltinStringFunctions.class. I double checked my local gradle cache and
> confirmed that the jar is there:
> {code:java}
> SJCMAC91THJHD4:beamsql-demo ypan$ ls -l
> ~/.gradle/caches/modules-2/files-2.1/org.apache.beam/beam-sdks-java-extensions-sql/2.9.0/67e7675519859ff332619c4c6ea5d26a505dbd50/beam-sdks-java-extensions-sql-2.9.0.jar
>
> -rw-r--r-- 1 ypan 192360288 12761025 Jan 29 18:02
> /Users/ypan/.gradle/caches/modules-2/files-2.1/org.apache.beam/beam-sdks-java-extensions-sql/2.9.0/67e7675519859ff332619c4c6ea5d26a505dbd50/beam-sdks-java-extensions-sql-2.9.0.jar
> {code}
> I also have tried to compile the sql sdk libraries in Beam's source repo and
> just copying over the generated class files over to my runtime classpath.
> Apparently, that did not work well since there are many shadowed libraries
> now need explicit declaring dependencies. I tried to search for an example of
> maven or gradle dependency declaration that users will use to create their
> own application using the SQL SDKs but was not lucky enough to find one.
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
