Thanks for the help everyone. It turns out that the latest version of the JAR seems to have fixed the issue. I closed the JIRA. — C
> On Sep 17, 2018, at 18:45, Parth Chandra <[email protected]> wrote: > > You might be in jar hell. > > If even adding the jar to 3rd party jars is causing an exception, then > clearly you should not be putting the jar in Drill's classpath without > taking corrective action. > > My guess is that you would need to build an uber-jar containing the UDF and > *all* its dependencies. The dependencies will have to be shaded to avoid > name conflicts. You then put the uber-jar in 3rd party jars. > > This will be helpful : [1]. Also, see the examples [2], [3] > > HTH. > > > [1] http://maven.apache.org/plugins/maven-shade-plugin/index.html > [2] > http://maven.apache.org/plugins/maven-shade-plugin/examples/includes-excludes.html > [3] > http://maven.apache.org/plugins/maven-shade-plugin/examples/class-relocation.html > > > > > > On Fri, Sep 14, 2018 at 6:00 AM Charles Givre (JIRA) <[email protected]> > wrote: > >> Charles Givre created DRILL-6743: >> ------------------------------------ >> >> Summary: H2O AI Library Causes Errors >> Key: DRILL-6743 >> URL: https://issues.apache.org/jira/browse/DRILL-6743 >> Project: Apache Drill >> Issue Type: Bug >> Components: Functions - Drill >> Affects Versions: 1.14.0, 1.15.0 >> Environment: Mac OSX High Sierra >> Reporter: Charles Givre >> >> >> I've been working on a UDF to use a POJO generated by H2O Ai. The basic >> idea is that a person could train a machine learning model in H2O, generate >> a POJO and then use Drill and the POJO to make predictions from data in >> Drill. This depends on importing the h2o-genmodel.jar ([ >> https://mvnrepository.com/artifact/ai.h2o/h2o-genmodel)]. However when >> you even put that JAR file into Drill's site directory or into the >> /jars/3rdParty you get the following errors: >> >> I've been speaking with [~paul-rogers] about this, and he thinks it has to >> do with a shaded module, but does anyone have any idea how to fix this? >> >> >> >> {{2018-09-13 23:38:54,163 ERROR >> [2464d2b3-7a4a-4449-d08e-61283bce78e0:frag:0:0] >> record.AbstractUnaryRecordBatch: Failure during query}} >> {{org.apache.drill.exec.exception.SchemaChangeException: Failure while >> attempting to load generated class}} >> {{ >> at >> org.apache.drill.exec.physical.impl.project.ProjectRecordBatch.setupNewSchemaFromInput(ProjectRecordBatch.java:572)}} >> {{ at >> org.apache.drill.exec.physical.impl.project.ProjectRecordBatch.setupNewSchema(ProjectRecordBatch.java:582)}} >> {{ at >> org.apache.drill.exec.record.AbstractUnaryRecordBatch.innerNext(AbstractUnaryRecordBatch.java:101)}} >> {{ >> at >> org.apache.drill.exec.physical.impl.project.ProjectRecordBatch.innerNext(ProjectRecordBatch.java:142)}} >> {{ at >> org.apache.drill.exec.record.AbstractRecordBatch.next(AbstractRecordBatch.java:172)}} >> {{ at >> org.apache.drill.exec.physical.impl.BaseRootExec.next(BaseRootExec.java:103)}} >> {{ at >> org.apache.drill.exec.physical.impl.ScreenCreator$ScreenRoot.innerNext(ScreenCreator.java:83)}} >> {{ at >> org.apache.drill.exec.physical.impl.BaseRootExec.next(BaseRootExec.java:93)}} >> {{ at org.apache.drill.exec.work >> .fragment.FragmentExecutor$1.run(FragmentExecutor.java:293)}} >> {{ at org.apache.drill.exec.work >> .fragment.FragmentExecutor$1.run(FragmentExecutor.java:280)}} >> {{ at java.security.AccessController.doPrivileged(Native Method)}} >> {{ at javax.security.auth.Subject.doAs(Subject.java:422)}} >> {{ at >> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1657)}} >> {{ at org.apache.drill.exec.work >> .fragment.FragmentExecutor.run(FragmentExecutor.java:280)}} >> {{ at >> org.apache.drill.common.SelfCleaningRunnable.run(SelfCleaningRunnable.java:38)}} >> {{ at >> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)}} >> {{ at >> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)}} >> {{ at java.lang.Thread.run(Thread.java:745)}} >> {{Caused by: >> org.apache.drill.exec.exception.ClassTransformationException: >> org.apache.drill.shaded.guava.com.google.common.util.concurrent.UncheckedExecutionException: >> java.lang.IllegalArgumentException}} >> {{ at >> org.apache.drill.exec.compile.CodeCompiler.createInstances(CodeCompiler.java:197)}} >> {{ at >> org.apache.drill.exec.compile.CodeCompiler.createInstance(CodeCompiler.java:163)}} >> {{ at >> org.apache.drill.exec.ops.BaseFragmentContext.getImplementationClass(BaseFragmentContext.java:56)}} >> {{ >> at >> org.apache.drill.exec.physical.impl.project.ProjectRecordBatch.setupNewSchemaFromInput(ProjectRecordBatch.java:569)}} >> {{ ... 17 more}} >> {{Caused by: >> org.apache.drill.shaded.guava.com.google.common.util.concurrent.UncheckedExecutionException: >> java.lang.IllegalArgumentException}} >> {{ at >> org.apache.drill.shaded.guava.com.google.common.cache.LocalCache$Segment.get(LocalCache.java:2218)}} >> {{ at >> org.apache.drill.shaded.guava.com.google.common.cache.LocalCache.get(LocalCache.java:4147)}} >> {{ at >> org.apache.drill.shaded.guava.com.google.common.cache.LocalCache.getOrLoad(LocalCache.java:4151)}} >> {{ at >> org.apache.drill.shaded.guava.com.google.common.cache.LocalCache$LocalLoadingCache.get(LocalCache.java:5140)}} >> {{ at >> org.apache.drill.exec.compile.CodeCompiler.createInstances(CodeCompiler.java:186)}} >> {{ ... 20 more}} >> {{Caused by: java.lang.IllegalArgumentException}} >> {{ at org.objectweb.asm.ClassReader.<init>(Unknown Source)}} >> {{ at org.objectweb.asm.ClassReader.<init>(Unknown Source)}} >> {{ at >> org.apache.drill.exec.compile.AsmUtil.classFromBytes(AsmUtil.java:93)}} >> {{ at >> org.apache.drill.exec.compile.AsmUtil.isClassBytesOk(AsmUtil.java:80)}} >> {{ at >> org.apache.drill.exec.compile.MergeAdapter.getMergedClass(MergeAdapter.java:206)}} >> {{ at >> org.apache.drill.exec.compile.ClassTransformer.getImplementationClass(ClassTransformer.java:289)}} >> {{ at >> org.apache.drill.exec.compile.ClassTransformer.getImplementationClass(ClassTransformer.java:228)}} >> {{ at >> org.apache.drill.exec.compile.CodeCompiler$CodeGenCompiler.compile(CodeCompiler.java:79)}} >> {{ at >> org.apache.drill.exec.compile.CodeCompiler.makeClass(CodeCompiler.java:229)}} >> {{ at >> org.apache.drill.exec.compile.CodeCompiler.access$300(CodeCompiler.java:41)}} >> {{ at >> org.apache.drill.exec.compile.CodeCompiler$Loader.load(CodeCompiler.java:212)}} >> {{ at >> org.apache.drill.exec.compile.CodeCompiler$Loader.load(CodeCompiler.java:209)}} >> {{ >> at >> org.apache.drill.shaded.guava.com.google.common.cache.LocalCache$LoadingValueReference.loadFuture(LocalCache.java:3708)}} >> {{ at >> org.apache.drill.shaded.guava.com.google.common.cache.LocalCache$Segment.loadSync(LocalCache.java:2416)}} >> {{ at >> org.apache.drill.shaded.guava.com.google.common.cache.LocalCache$Segment.lockedGetOrLoad(LocalCache.java:2299)}} >> {{ at >> org.apache.drill.shaded.guava.com.google.common.cache.LocalCache$Segment.get(LocalCache.java:2212)}} >> {{ ... 24 more}} >> {{2018-09-13 23:38:54,170 ERROR >> [2464d2b3-7a4a-4449-d08e-61283bce78e0:frag:0:0] fragment.FragmentExecutor: >> SYSTEM ERROR: IllegalArgumentException}} >> >> {{Fragment 0:0}} >> >> {{[Error Id: 90ee4178-17b7-4566-af59-85b99fb64d22 on >> charles-mbp.fios-router.home:31010]}} >> {{org.apache.drill.common.exceptions.UserException: SYSTEM ERROR: >> IllegalArgumentException}} >> >> {{Fragment 0:0}} >> >> {{[Error Id: 90ee4178-17b7-4566-af59-85b99fb64d22 on >> charles-mbp.fios-router.home:31010]}} >> {{ at >> org.apache.drill.common.exceptions.UserException$Builder.build(UserException.java:633)}} >> {{ at org.apache.drill.exec.work >> .fragment.FragmentExecutor.sendFinalState(FragmentExecutor.java:360)}} >> {{ at org.apache.drill.exec.work >> .fragment.FragmentExecutor.cleanup(FragmentExecutor.java:215)}} >> {{ at org.apache.drill.exec.work >> .fragment.FragmentExecutor.run(FragmentExecutor.java:326)}} >> {{ at >> org.apache.drill.common.SelfCleaningRunnable.run(SelfCleaningRunnable.java:38)}} >> {{ at >> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)}} >> {{ at >> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)}} >> {{ at java.lang.Thread.run(Thread.java:745)}} >> {{Caused by: org.apache.drill.exec.exception.SchemaChangeException: >> Failure while attempting to load generated class}} >> {{ >> at >> org.apache.drill.exec.physical.impl.project.ProjectRecordBatch.setupNewSchemaFromInput(ProjectRecordBatch.java:572)}} >> {{ at >> org.apache.drill.exec.physical.impl.project.ProjectRecordBatch.setupNewSchema(ProjectRecordBatch.java:582)}} >> {{ at >> org.apache.drill.exec.record.AbstractUnaryRecordBatch.innerNext(AbstractUnaryRecordBatch.java:101)}} >> {{ at >> org.apache.drill.exec.physical.impl.project.ProjectRecordBatch.innerNext(ProjectRecordBatch.java:142)}} >> {{ at >> org.apache.drill.exec.record.AbstractRecordBatch.next(AbstractRecordBatch.java:172)}} >> {{ at >> org.apache.drill.exec.physical.impl.BaseRootExec.next(BaseRootExec.java:103)}} >> {{ at >> org.apache.drill.exec.physical.impl.ScreenCreator$ScreenRoot.innerNext(ScreenCreator.java:83)}} >> {{ at >> org.apache.drill.exec.physical.impl.BaseRootExec.next(BaseRootExec.java:93)}} >> {{ at org.apache.drill.exec.work >> .fragment.FragmentExecutor$1.run(FragmentExecutor.java:293)}} >> {{ at org.apache.drill.exec.work >> .fragment.FragmentExecutor$1.run(FragmentExecutor.java:280)}} >> {{ at java.security.AccessController.doPrivileged(Native Method)}} >> {{ at javax.security.auth.Subject.doAs(Subject.java:422)}} >> {{ at >> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1657)}} >> {{ at org.apache.drill.exec.work >> .fragment.FragmentExecutor.run(FragmentExecutor.java:280)}} >> {{ ... 4 more}} >> {{Caused by: >> org.apache.drill.exec.exception.ClassTransformationException: >> org.apache.drill.shaded.guava.com.google.common.util.concurrent.UncheckedExecutionException: >> java.lang.IllegalArgumentException}} >> {{ at >> org.apache.drill.exec.compile.CodeCompiler.createInstances(CodeCompiler.java:197)}} >> {{ at >> org.apache.drill.exec.compile.CodeCompiler.createInstance(CodeCompiler.java:163)}} >> {{ at >> org.apache.drill.exec.ops.BaseFragmentContext.getImplementationClass(BaseFragmentContext.java:56)}} >> {{ >> at >> org.apache.drill.exec.physical.impl.project.ProjectRecordBatch.setupNewSchemaFromInput(ProjectRecordBatch.java:569)}} >> {{ ... 17 more}} >> {{Caused by: >> org.apache.drill.shaded.guava.com.google.common.util.concurrent.UncheckedExecutionException: >> java.lang.IllegalArgumentException}} >> {{ at >> org.apache.drill.shaded.guava.com.google.common.cache.LocalCache$Segment.get(LocalCache.java:2218)}} >> {{ at >> org.apache.drill.shaded.guava.com.google.common.cache.LocalCache.get(LocalCache.java:4147)}} >> {{ at >> org.apache.drill.shaded.guava.com.google.common.cache.LocalCache.getOrLoad(LocalCache.java:4151)}} >> {{ at >> org.apache.drill.shaded.guava.com.google.common.cache.LocalCache$LocalLoadingCache.get(LocalCache.java:5140)}} >> {{ at >> org.apache.drill.exec.compile.CodeCompiler.createInstances(CodeCompiler.java:186)}} >> {{ ... 20 more}} >> {{Caused by: java.lang.IllegalArgumentException}} >> {{ at org.objectweb.asm.ClassReader.<init>(Unknown Source)}} >> {{ at org.objectweb.asm.ClassReader.<init>(Unknown Source)}} >> {{ at >> org.apache.drill.exec.compile.AsmUtil.classFromBytes(AsmUtil.java:93)}} >> {{ at >> org.apache.drill.exec.compile.AsmUtil.isClassBytesOk(AsmUtil.java:80)}} >> {{ at >> org.apache.drill.exec.compile.MergeAdapter.getMergedClass(MergeAdapter.java:206)}} >> {{ at >> org.apache.drill.exec.compile.ClassTransformer.getImplementationClass(ClassTransformer.java:289)}} >> {{ at >> org.apache.drill.exec.compile.ClassTransformer.getImplementationClass(ClassTransformer.java:228)}} >> {{ at >> org.apache.drill.exec.compile.CodeCompiler$CodeGenCompiler.compile(CodeCompiler.java:79)}} >> {{ at >> org.apache.drill.exec.compile.CodeCompiler.makeClass(CodeCompiler.java:229)}} >> {{ at >> org.apache.drill.exec.compile.CodeCompiler.access$300(CodeCompiler.java:41)}} >> {{ at >> org.apache.drill.exec.compile.CodeCompiler$Loader.load(CodeCompiler.java:212)}} >> {{ at >> org.apache.drill.exec.compile.CodeCompiler$Loader.load(CodeCompiler.java:209)}} >> {{ >> at >> org.apache.drill.shaded.guava.com.google.common.cache.LocalCache$LoadingValueReference.loadFuture(LocalCache.java:3708)}} >> {{ at >> org.apache.drill.shaded.guava.com.google.common.cache.LocalCache$Segment.loadSync(LocalCache.java:2416)}} >> {{ at >> org.apache.drill.shaded.guava.com.google.common.cache.LocalCache$Segment.lockedGetOrLoad(LocalCache.java:2299)}} >> {{ at >> org.apache.drill.shaded.guava.com.google.common.cache.LocalCache$Segment.get(LocalCache.java:2212)}} >> {{ ... 24 more}} >> {{Error: SYSTEM ERROR: IllegalArgumentException}} >> >> {{Fragment 0:0}} >> >> >> >> -- >> This message was sent by Atlassian JIRA >> (v7.6.3#76005) >>
