You might be in jar hell. If even adding the jar to 3rd party jars is causing an exception, then clearly you should not be putting the jar in Drill's classpath without taking corrective action.
My guess is that you would need to build an uber-jar containing the UDF and *all* its dependencies. The dependencies will have to be shaded to avoid name conflicts. You then put the uber-jar in 3rd party jars. This will be helpful : [1]. Also, see the examples [2], [3] HTH. [1] http://maven.apache.org/plugins/maven-shade-plugin/index.html [2] http://maven.apache.org/plugins/maven-shade-plugin/examples/includes-excludes.html [3] http://maven.apache.org/plugins/maven-shade-plugin/examples/class-relocation.html On Fri, Sep 14, 2018 at 6:00 AM Charles Givre (JIRA) <[email protected]> wrote: > Charles Givre created DRILL-6743: > ------------------------------------ > > Summary: H2O AI Library Causes Errors > Key: DRILL-6743 > URL: https://issues.apache.org/jira/browse/DRILL-6743 > Project: Apache Drill > Issue Type: Bug > Components: Functions - Drill > Affects Versions: 1.14.0, 1.15.0 > Environment: Mac OSX High Sierra > Reporter: Charles Givre > > > I've been working on a UDF to use a POJO generated by H2O Ai. The basic > idea is that a person could train a machine learning model in H2O, generate > a POJO and then use Drill and the POJO to make predictions from data in > Drill. This depends on importing the h2o-genmodel.jar ([ > https://mvnrepository.com/artifact/ai.h2o/h2o-genmodel)]. However when > you even put that JAR file into Drill's site directory or into the > /jars/3rdParty you get the following errors: > > I've been speaking with [~paul-rogers] about this, and he thinks it has to > do with a shaded module, but does anyone have any idea how to fix this? > > > > {{2018-09-13 23:38:54,163 ERROR > [2464d2b3-7a4a-4449-d08e-61283bce78e0:frag:0:0] > record.AbstractUnaryRecordBatch: Failure during query}} > {{org.apache.drill.exec.exception.SchemaChangeException: Failure while > attempting to load generated class}} > {{ > at > org.apache.drill.exec.physical.impl.project.ProjectRecordBatch.setupNewSchemaFromInput(ProjectRecordBatch.java:572)}} > {{ at > org.apache.drill.exec.physical.impl.project.ProjectRecordBatch.setupNewSchema(ProjectRecordBatch.java:582)}} > {{ at > org.apache.drill.exec.record.AbstractUnaryRecordBatch.innerNext(AbstractUnaryRecordBatch.java:101)}} > {{ > at > org.apache.drill.exec.physical.impl.project.ProjectRecordBatch.innerNext(ProjectRecordBatch.java:142)}} > {{ at > org.apache.drill.exec.record.AbstractRecordBatch.next(AbstractRecordBatch.java:172)}} > {{ at > org.apache.drill.exec.physical.impl.BaseRootExec.next(BaseRootExec.java:103)}} > {{ at > org.apache.drill.exec.physical.impl.ScreenCreator$ScreenRoot.innerNext(ScreenCreator.java:83)}} > {{ at > org.apache.drill.exec.physical.impl.BaseRootExec.next(BaseRootExec.java:93)}} > {{ at org.apache.drill.exec.work > .fragment.FragmentExecutor$1.run(FragmentExecutor.java:293)}} > {{ at org.apache.drill.exec.work > .fragment.FragmentExecutor$1.run(FragmentExecutor.java:280)}} > {{ at java.security.AccessController.doPrivileged(Native Method)}} > {{ at javax.security.auth.Subject.doAs(Subject.java:422)}} > {{ at > org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1657)}} > {{ at org.apache.drill.exec.work > .fragment.FragmentExecutor.run(FragmentExecutor.java:280)}} > {{ at > org.apache.drill.common.SelfCleaningRunnable.run(SelfCleaningRunnable.java:38)}} > {{ at > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)}} > {{ at > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)}} > {{ at java.lang.Thread.run(Thread.java:745)}} > {{Caused by: > org.apache.drill.exec.exception.ClassTransformationException: > org.apache.drill.shaded.guava.com.google.common.util.concurrent.UncheckedExecutionException: > java.lang.IllegalArgumentException}} > {{ at > org.apache.drill.exec.compile.CodeCompiler.createInstances(CodeCompiler.java:197)}} > {{ at > org.apache.drill.exec.compile.CodeCompiler.createInstance(CodeCompiler.java:163)}} > {{ at > org.apache.drill.exec.ops.BaseFragmentContext.getImplementationClass(BaseFragmentContext.java:56)}} > {{ > at > org.apache.drill.exec.physical.impl.project.ProjectRecordBatch.setupNewSchemaFromInput(ProjectRecordBatch.java:569)}} > {{ ... 17 more}} > {{Caused by: > org.apache.drill.shaded.guava.com.google.common.util.concurrent.UncheckedExecutionException: > java.lang.IllegalArgumentException}} > {{ at > org.apache.drill.shaded.guava.com.google.common.cache.LocalCache$Segment.get(LocalCache.java:2218)}} > {{ at > org.apache.drill.shaded.guava.com.google.common.cache.LocalCache.get(LocalCache.java:4147)}} > {{ at > org.apache.drill.shaded.guava.com.google.common.cache.LocalCache.getOrLoad(LocalCache.java:4151)}} > {{ at > org.apache.drill.shaded.guava.com.google.common.cache.LocalCache$LocalLoadingCache.get(LocalCache.java:5140)}} > {{ at > org.apache.drill.exec.compile.CodeCompiler.createInstances(CodeCompiler.java:186)}} > {{ ... 20 more}} > {{Caused by: java.lang.IllegalArgumentException}} > {{ at org.objectweb.asm.ClassReader.<init>(Unknown Source)}} > {{ at org.objectweb.asm.ClassReader.<init>(Unknown Source)}} > {{ at > org.apache.drill.exec.compile.AsmUtil.classFromBytes(AsmUtil.java:93)}} > {{ at > org.apache.drill.exec.compile.AsmUtil.isClassBytesOk(AsmUtil.java:80)}} > {{ at > org.apache.drill.exec.compile.MergeAdapter.getMergedClass(MergeAdapter.java:206)}} > {{ at > org.apache.drill.exec.compile.ClassTransformer.getImplementationClass(ClassTransformer.java:289)}} > {{ at > org.apache.drill.exec.compile.ClassTransformer.getImplementationClass(ClassTransformer.java:228)}} > {{ at > org.apache.drill.exec.compile.CodeCompiler$CodeGenCompiler.compile(CodeCompiler.java:79)}} > {{ at > org.apache.drill.exec.compile.CodeCompiler.makeClass(CodeCompiler.java:229)}} > {{ at > org.apache.drill.exec.compile.CodeCompiler.access$300(CodeCompiler.java:41)}} > {{ at > org.apache.drill.exec.compile.CodeCompiler$Loader.load(CodeCompiler.java:212)}} > {{ at > org.apache.drill.exec.compile.CodeCompiler$Loader.load(CodeCompiler.java:209)}} > {{ > at > org.apache.drill.shaded.guava.com.google.common.cache.LocalCache$LoadingValueReference.loadFuture(LocalCache.java:3708)}} > {{ at > org.apache.drill.shaded.guava.com.google.common.cache.LocalCache$Segment.loadSync(LocalCache.java:2416)}} > {{ at > org.apache.drill.shaded.guava.com.google.common.cache.LocalCache$Segment.lockedGetOrLoad(LocalCache.java:2299)}} > {{ at > org.apache.drill.shaded.guava.com.google.common.cache.LocalCache$Segment.get(LocalCache.java:2212)}} > {{ ... 24 more}} > {{2018-09-13 23:38:54,170 ERROR > [2464d2b3-7a4a-4449-d08e-61283bce78e0:frag:0:0] fragment.FragmentExecutor: > SYSTEM ERROR: IllegalArgumentException}} > > {{Fragment 0:0}} > > {{[Error Id: 90ee4178-17b7-4566-af59-85b99fb64d22 on > charles-mbp.fios-router.home:31010]}} > {{org.apache.drill.common.exceptions.UserException: SYSTEM ERROR: > IllegalArgumentException}} > > {{Fragment 0:0}} > > {{[Error Id: 90ee4178-17b7-4566-af59-85b99fb64d22 on > charles-mbp.fios-router.home:31010]}} > {{ at > org.apache.drill.common.exceptions.UserException$Builder.build(UserException.java:633)}} > {{ at org.apache.drill.exec.work > .fragment.FragmentExecutor.sendFinalState(FragmentExecutor.java:360)}} > {{ at org.apache.drill.exec.work > .fragment.FragmentExecutor.cleanup(FragmentExecutor.java:215)}} > {{ at org.apache.drill.exec.work > .fragment.FragmentExecutor.run(FragmentExecutor.java:326)}} > {{ at > org.apache.drill.common.SelfCleaningRunnable.run(SelfCleaningRunnable.java:38)}} > {{ at > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)}} > {{ at > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)}} > {{ at java.lang.Thread.run(Thread.java:745)}} > {{Caused by: org.apache.drill.exec.exception.SchemaChangeException: > Failure while attempting to load generated class}} > {{ > at > org.apache.drill.exec.physical.impl.project.ProjectRecordBatch.setupNewSchemaFromInput(ProjectRecordBatch.java:572)}} > {{ at > org.apache.drill.exec.physical.impl.project.ProjectRecordBatch.setupNewSchema(ProjectRecordBatch.java:582)}} > {{ at > org.apache.drill.exec.record.AbstractUnaryRecordBatch.innerNext(AbstractUnaryRecordBatch.java:101)}} > {{ at > org.apache.drill.exec.physical.impl.project.ProjectRecordBatch.innerNext(ProjectRecordBatch.java:142)}} > {{ at > org.apache.drill.exec.record.AbstractRecordBatch.next(AbstractRecordBatch.java:172)}} > {{ at > org.apache.drill.exec.physical.impl.BaseRootExec.next(BaseRootExec.java:103)}} > {{ at > org.apache.drill.exec.physical.impl.ScreenCreator$ScreenRoot.innerNext(ScreenCreator.java:83)}} > {{ at > org.apache.drill.exec.physical.impl.BaseRootExec.next(BaseRootExec.java:93)}} > {{ at org.apache.drill.exec.work > .fragment.FragmentExecutor$1.run(FragmentExecutor.java:293)}} > {{ at org.apache.drill.exec.work > .fragment.FragmentExecutor$1.run(FragmentExecutor.java:280)}} > {{ at java.security.AccessController.doPrivileged(Native Method)}} > {{ at javax.security.auth.Subject.doAs(Subject.java:422)}} > {{ at > org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1657)}} > {{ at org.apache.drill.exec.work > .fragment.FragmentExecutor.run(FragmentExecutor.java:280)}} > {{ ... 4 more}} > {{Caused by: > org.apache.drill.exec.exception.ClassTransformationException: > org.apache.drill.shaded.guava.com.google.common.util.concurrent.UncheckedExecutionException: > java.lang.IllegalArgumentException}} > {{ at > org.apache.drill.exec.compile.CodeCompiler.createInstances(CodeCompiler.java:197)}} > {{ at > org.apache.drill.exec.compile.CodeCompiler.createInstance(CodeCompiler.java:163)}} > {{ at > org.apache.drill.exec.ops.BaseFragmentContext.getImplementationClass(BaseFragmentContext.java:56)}} > {{ > at > org.apache.drill.exec.physical.impl.project.ProjectRecordBatch.setupNewSchemaFromInput(ProjectRecordBatch.java:569)}} > {{ ... 17 more}} > {{Caused by: > org.apache.drill.shaded.guava.com.google.common.util.concurrent.UncheckedExecutionException: > java.lang.IllegalArgumentException}} > {{ at > org.apache.drill.shaded.guava.com.google.common.cache.LocalCache$Segment.get(LocalCache.java:2218)}} > {{ at > org.apache.drill.shaded.guava.com.google.common.cache.LocalCache.get(LocalCache.java:4147)}} > {{ at > org.apache.drill.shaded.guava.com.google.common.cache.LocalCache.getOrLoad(LocalCache.java:4151)}} > {{ at > org.apache.drill.shaded.guava.com.google.common.cache.LocalCache$LocalLoadingCache.get(LocalCache.java:5140)}} > {{ at > org.apache.drill.exec.compile.CodeCompiler.createInstances(CodeCompiler.java:186)}} > {{ ... 20 more}} > {{Caused by: java.lang.IllegalArgumentException}} > {{ at org.objectweb.asm.ClassReader.<init>(Unknown Source)}} > {{ at org.objectweb.asm.ClassReader.<init>(Unknown Source)}} > {{ at > org.apache.drill.exec.compile.AsmUtil.classFromBytes(AsmUtil.java:93)}} > {{ at > org.apache.drill.exec.compile.AsmUtil.isClassBytesOk(AsmUtil.java:80)}} > {{ at > org.apache.drill.exec.compile.MergeAdapter.getMergedClass(MergeAdapter.java:206)}} > {{ at > org.apache.drill.exec.compile.ClassTransformer.getImplementationClass(ClassTransformer.java:289)}} > {{ at > org.apache.drill.exec.compile.ClassTransformer.getImplementationClass(ClassTransformer.java:228)}} > {{ at > org.apache.drill.exec.compile.CodeCompiler$CodeGenCompiler.compile(CodeCompiler.java:79)}} > {{ at > org.apache.drill.exec.compile.CodeCompiler.makeClass(CodeCompiler.java:229)}} > {{ at > org.apache.drill.exec.compile.CodeCompiler.access$300(CodeCompiler.java:41)}} > {{ at > org.apache.drill.exec.compile.CodeCompiler$Loader.load(CodeCompiler.java:212)}} > {{ at > org.apache.drill.exec.compile.CodeCompiler$Loader.load(CodeCompiler.java:209)}} > {{ > at > org.apache.drill.shaded.guava.com.google.common.cache.LocalCache$LoadingValueReference.loadFuture(LocalCache.java:3708)}} > {{ at > org.apache.drill.shaded.guava.com.google.common.cache.LocalCache$Segment.loadSync(LocalCache.java:2416)}} > {{ at > org.apache.drill.shaded.guava.com.google.common.cache.LocalCache$Segment.lockedGetOrLoad(LocalCache.java:2299)}} > {{ at > org.apache.drill.shaded.guava.com.google.common.cache.LocalCache$Segment.get(LocalCache.java:2212)}} > {{ ... 24 more}} > {{Error: SYSTEM ERROR: IllegalArgumentException}} > > {{Fragment 0:0}} > > > > -- > This message was sent by Atlassian JIRA > (v7.6.3#76005) >
