Wow, thanks for the great report. Your configuration looks good to me. I filed https://issues.apache.org/jira/browse/BEAM-6558 to figure this out.
Kenn On Wed, Jan 30, 2019 at 7:01 PM Yi Pan <[email protected]> wrote: > Hi, all, > > Newbie here trying to figure out how to use published > beam-sdks-java-extensions-sql-2.9.0 in my own project. > > I tried to create a gradle project to use BeamSQL sdk libraries. Here is > the build.gradle I have: > {code} > plugins { > id 'java' > } > > group 'com.mycompany.myproject' > version '1.0-SNAPSHOT' > > sourceCompatibility = 1.8 > > repositories { > mavenCentral() > } > > apply plugin: 'java' > > sourceSets.main.java.srcDirs = [ > 'src/main/java' > ] > > dependencies { > compile 'org.apache.beam:beam-sdks-java-core:2.9.0' > compile 'org.apache.beam:beam-sdks-java-extensions-sql:2.9.0' > compile 'com.google.code.findbugs:jsr305:3.0.2' > runtime 'org.apache.beam:beam-runners-direct-java:2.9.0' > testCompile group: 'junit', name: 'junit', version: '4.12' > } > > // Run basic SQL example > task runBasicExample(type: JavaExec) { > description = "Run basic SQL example" > main = "com.mycompany.myproject.streamsql.examples.BeamSQLExample" > classpath = sourceSets.main.runtimeClasspath > args = ["--runner=DirectRunner"] > println classpath.getAsPath() > println args > } > {code} > > The example BeamSQLExample is just copied from > https://github.com/apache/beam/blob/master/sdks/java/extensions/sql/src/main/java/org/apache/beam/sdk/extensions/sql/example/BeamSqlExample.java > . > > I was able to compile the example in JDK8. However, when I tried to run > it, I hit the following exception: > {code} > Exception in thread "main" java.util.ServiceConfigurationError: > org.apache.beam.sdk.extensions.sql.impl.udf.BeamBuiltinFunctionProvider: > Provider org.apache.beam.sdk.extensions.sql.impl.udf.BuiltinStringFunctions > could not be instantiated > at java.util.ServiceLoader.fail(ServiceLoader.java:232) > at java.util.ServiceLoader.access$100(ServiceLoader.java:185) > at java.util.ServiceLoader$LazyIterator.nextService(ServiceLoader.java:384) > at java.util.ServiceLoader$LazyIterator.next(ServiceLoader.java:404) > at java.util.ServiceLoader$1.next(ServiceLoader.java:480) > at > org.apache.beam.sdk.extensions.sql.impl.BeamSqlEnv.loadBeamBuiltinFunctions(BeamSqlEnv.java:128) > at > org.apache.beam.sdk.extensions.sql.SqlTransform.expand(SqlTransform.java:94) > at > org.apache.beam.sdk.extensions.sql.SqlTransform.expand(SqlTransform.java:76) > at org.apache.beam.sdk.Pipeline.applyInternal(Pipeline.java:537) > at org.apache.beam.sdk.Pipeline.applyTransform(Pipeline.java:471) > at org.apache.beam.sdk.values.PCollection.apply(PCollection.java:357) > at > com.mycompany.myprject.streamsql.examples.BeamSQLExample.main(BeamSQLExample.java:72) > Caused by: java.lang.NoClassDefFoundError: > org/apache/commons/codec/DecoderException > at java.lang.Class.getDeclaredConstructors0(Native Method) > at java.lang.Class.privateGetDeclaredConstructors(Class.java:2671) > at java.lang.Class.getConstructor0(Class.java:3075) > at java.lang.Class.newInstance(Class.java:412) > at java.util.ServiceLoader$LazyIterator.nextService(ServiceLoader.java:380) > ... 9 more > Caused by: java.lang.ClassNotFoundException: > org.apache.commons.codec.DecoderException > at java.net.URLClassLoader.findClass(URLClassLoader.java:382) > at java.lang.ClassLoader.loadClass(ClassLoader.java:424) > at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:349) > at java.lang.ClassLoader.loadClass(ClassLoader.java:357) > ... 14 more > {code} > > When I traced into the code, it turns out that the failure occurs when the > classloader tries to get the default constructor w/o any parameters for > BuiltinStringFunctions.class. I double checked my local gradle cache and > confirmed that the jar is there: > {code} > SJCMAC91THJHD4:beamsql-demo ypan$ ls -l > ~/.gradle/caches/modules-2/files-2.1/org.apache.beam/beam-sdks-java-extensions-sql/2.9.0/67e7675519859ff332619c4c6ea5d26a505dbd50/beam-sdks-java-extensions-sql-2.9.0.jar > -rw-r--r-- 1 ypan 192360288 12761025 Jan 29 18:02 > /Users/ypan/.gradle/caches/modules-2/files-2.1/org.apache.beam/beam-sdks-java-extensions-sql/2.9.0/67e7675519859ff332619c4c6ea5d26a505dbd50/beam-sdks-java-extensions-sql-2.9.0.jar > {code} > > I also have tried to compile the sql sdk libraries in Beam's source repo > and just copying over the generated class files over to my runtime > classpath. Apparently, that did not work well since there are many shadowed > libraries now need explicit declaring dependencies. I tried to search for > an example of maven or gradle dependency declaration that users will use to > create their own application using the SQL SDKs but was not lucky enough to > find one. > > Given the amount of debugging and searching effort I have put it there, I > think that it is reasonable to ask the question to the community. Can > someone shed some light on what went wrong in my build project? > > Thanks! > > -Yi > >
