Thank you Ted for the super trade. I really appreciate it. But then as of now I don't have any UDF . I was just trying test simple UDF on git. Issue is resolved it is working for me now. I downloaded older version of Maven. I was using latest version of Maven , guess that was the issue.
Regards, Alok Tanna On Fri, Jun 26, 2015 at 2:44 AM, Ted Dunning <[email protected]> wrote: > Alok, > > Can you put your UDF in a git repo that I can access? > > I am happy to build you a modified version in exchange for you helping to > document the somewhat obscure constraints on how a UDF needs to be written. > > Sound like a good trade? > > n > > On Thu, Jun 25, 2015 at 3:21 PM, Alok Tanna <[email protected]> > wrote: > > > I got the full JDK version now I am getting below errors while building > the > > JAR file > > > > [root@ip-172-31-34-166 simple-drill-functions]# > > /home/ec2-user/apache-maven-3.3. > > 3/bin/mvn package > > [INFO] Scanning for projects... > > [INFO] > > [INFO] > > ------------------------------------------------------------------------ > > [INFO] Building simple-drill-function 1.0-SNAPSHOT > > [INFO] > > ------------------------------------------------------------------------ > > [INFO] > > [INFO] --- maven-resources-plugin:2.6:resources (default-resources) @ > > simple-dri ll-function --- > > [WARNING] Using platform encoding (UTF-8 actually) to copy filtered > > resources, i .e. build is platform > > dependent! > > [INFO] Copying 1 resource > > [INFO] > > [INFO] --- maven-compiler-plugin:2.3.2:compile (default-compile) @ > > simple-drill- function --- > > [WARNING] File encoding has not been set, using platform encoding UTF-8, > > i.e. bu ild is platform dependent! > > [INFO] Compiling 4 source files to > > /home/ec2-user/simple-drill-functions/target/ > > classes > > [INFO] ------------------------------------------------------------- > > [ERROR] COMPILATION ERROR : > > [INFO] ------------------------------------------------------------- > > [ERROR] > > /home/ec2-user/simple-drill-functions/src/main/java/com/mapr/drill/Simpl > > eAdd.java:[3,33] error: package > > org.apache.drill.exec.expr does not exist > > [ERROR] > > /home/ec2-user/simple-drill-functions/src/main/java/com/mapr/drill/Simpl > > eAdd.java:[4,45] error: package > > org.apache.drill.exec.expr.annotations does not > > exist > > [ERROR] > > /home/ec2-user/simple-drill-functions/src/main/java/com/mapr/drill/Simpl > > eAdd.java:[5,62] error: package > > org.apache.drill.exec.expr.annotations.FunctionT > > emplate does not exist > > [ERROR] > > /home/ec2-user/simple-drill-functions/src/main/java/com/mapr/drill/Simpl > > eAdd.java:[6,62] error: package > > org.apache.drill.exec.expr.annotations.FunctionT > > emplate does not exist > > [ERROR] > > /home/ec2-user/simple-drill-functions/src/main/java/com/mapr/drill/Simpl > > eAdd.java:[7,45] error: package > > org.apache.drill.exec.expr.annotations does not > > exist > > [ERROR] > > /home/ec2-user/simple-drill-functions/src/main/java/com/mapr/drill/Simpl > > eAdd.java:[8,45] error: package > > org.apache.drill.exec.expr.annotations does not > > exist > > [ERROR] > > /home/ec2-user/simple-drill-functions/src/main/java/com/mapr/drill/Simpl > > eAdd.java:[9,41] error: package > > org.apache.drill.exec.expr.holders does not exis > > t > > [ERROR] > > /home/ec2-user/simple-drill-functions/src/main/java/com/mapr/drill/Simpl > > eAdd.java:[10,41] error: package > > org.apache.drill.exec.expr.holders does not exi > > st > > [ERROR] > > /home/ec2-user/simple-drill-functions/src/main/java/com/mapr/drill/Simpl > > eAdd.java:[22,45] error: cannot find > > symbol > > [ERROR] class SimpleAdd > > > > > /home/ec2-user/simple-drill-functions/src/main/java/com/mapr/drill/SimpleAdd.jav > > a:[21,5] error: cannot find symbol > > [ERROR] class SimpleAdd > > > > > /home/ec2-user/simple-drill-functions/src/main/java/com/mapr/drill/SimpleAdd.jav > > a:[24,8] error: cannot find symbol > > [ERROR] class IntIntAdd > > > > > /home/ec2-user/simple-drill-functions/src/main/java/com/mapr/drill/SimpleAdd.jav > > a:[26,8] error: cannot find symbol > > [ERROR] class IntIntAdd > > > > > /home/ec2-user/simple-drill-functions/src/main/java/com/mapr/drill/SimpleAdd.jav > > a:[28,8] error: cannot find symbol > > [ERROR] class IntIntAdd > > > > > /home/ec2-user/simple-drill-functions/src/main/java/com/mapr/drill/Zip.java:[3,3 > > 3] error: package > > org.apache.drill.exec.expr does not exist > > [ERROR] > > /home/ec2-user/simple-drill-functions/src/main/java/com/mapr/drill/Zip.j > > ava:[4,45] error: package > > org.apache.drill.exec.expr.annotations does not exist > > [ERROR] > > /home/ec2-user/simple-drill-functions/src/main/java/com/mapr/drill/Zip.j > > ava:[5,45] error: package > > org.apache.drill.exec.expr.annotations does not exist > > [ERROR] > > /home/ec2-user/simple-drill-functions/src/main/java/com/mapr/drill/Zip.j > > ava:[6,45] error: package > > org.apache.drill.exec.expr.annotations does not exist > > [ERROR] > > /home/ec2-user/simple-drill-functions/src/main/java/com/mapr/drill/Zip.j > > ava:[7,41] error: package > > org.apache.drill.exec.expr.holders does not exist > > [ERROR] > > /home/ec2-user/simple-drill-functions/src/main/java/com/mapr/drill/Zip.j > > ava:[8,41] error: package > > org.apache.drill.exec.expr.holders does not exist > > [ERROR] > > /home/ec2-user/simple-drill-functions/src/main/java/com/mapr/drill/Zip.j > > ava:[9,50] error: package > > org.apache.drill.exec.vector.complex.writer does not e > > xist > > [ERROR] > > /home/ec2-user/simple-drill-functions/src/main/java/com/mapr/drill/Zip.j > > ava:[64,43] error: cannot find symbol > > [ERROR] class Zip > > > > > /home/ec2-user/simple-drill-functions/src/main/java/com/mapr/drill/Zip.java:[63, > > 5] error: cannot find symbol > > [ERROR] class Zip > > > > > /home/ec2-user/simple-drill-functions/src/main/java/com/mapr/drill/Zip.java:[66, > > 8] error: cannot find symbol > > [ERROR] class IntZip2 > > > > > /home/ec2-user/simple-drill-functions/src/main/java/com/mapr/drill/Zip.java:[68, > > 8] error: cannot find symbol > > [ERROR] class IntZip2 > > > > > /home/ec2-user/simple-drill-functions/src/main/java/com/mapr/drill/Zip.java:[71, > > 18] error: package BaseWriter does > not > > exist > > [ERROR] > > /home/ec2-user/simple-drill-functions/src/main/java/com/mapr/drill/Zip.j > > ava:[82,43] error: cannot find symbol > > [ERROR] class Zip > > > > > /home/ec2-user/simple-drill-functions/src/main/java/com/mapr/drill/Zip.java:[81, > > 5] error: cannot find symbol > > [ERROR] class Zip > > > > > /home/ec2-user/simple-drill-functions/src/main/java/com/mapr/drill/Zip.java:[84, > > 8] error: cannot find symbol > > [ERROR] class IntZip3 > > > > > /home/ec2-user/simple-drill-functions/src/main/java/com/mapr/drill/Zip.java:[86, > > 8] error: cannot find symbol > > [ERROR] class IntZip3 > > > > > /home/ec2-user/simple-drill-functions/src/main/java/com/mapr/drill/Zip.java:[88, > > 8] error: cannot find symbol > > [ERROR] class IntZip3 > > > > > /home/ec2-user/simple-drill-functions/src/main/java/com/mapr/drill/Zip.java:[91, > > 18] error: package BaseWriter does > not > > exist > > [ERROR] > > /home/ec2-user/simple-drill-functions/src/main/java/com/mapr/drill/Zip.j > > ava:[102,43] error: cannot find symbol > > [ERROR] class Zip > > > > > /home/ec2-user/simple-drill-functions/src/main/java/com/mapr/drill/Zip.java:[101 > > ,5] error: cannot find symbol > > [ERROR] class Zip > > > > > /home/ec2-user/simple-drill-functions/src/main/java/com/mapr/drill/Zip.java:[104 > > ,8] error: cannot find symbol > > [ERROR] class IntZip4 > > > > > /home/ec2-user/simple-drill-functions/src/main/java/com/mapr/drill/Zip.java:[106 > > ,8] error: cannot find symbol > > [ERROR] class IntZip4 > > > > > /home/ec2-user/simple-drill-functions/src/main/java/com/mapr/drill/Zip.java:[108 > > ,8] error: cannot find symbol > > [ERROR] class IntZip4 > > > > > /home/ec2-user/simple-drill-functions/src/main/java/com/mapr/drill/Zip.java:[110 > > ,8] error: cannot find symbol > > [ERROR] class IntZip4 > > > > > /home/ec2-user/simple-drill-functions/src/main/java/com/mapr/drill/Zip.java:[113 > > ,18] error: package BaseWriter does > > not exist > > [ERROR] > > /home/ec2-user/simple-drill-functions/src/main/java/com/mapr/drill/Zip.j > > ava:[124,45] error: cannot find symbol > > [ERROR] class Zip > > > > > /home/ec2-user/simple-drill-functions/src/main/java/com/mapr/drill/Zip.java:[123 > > ,5] error: cannot find symbol > > [ERROR] class Zip > > > > > /home/ec2-user/simple-drill-functions/src/main/java/com/mapr/drill/Zip.java:[126 > > ,8] error: cannot find symbol > > [ERROR] class FloatZip2 > > > > > /home/ec2-user/simple-drill-functions/src/main/java/com/mapr/drill/Zip.java:[128 > > ,8] error: cannot find symbol > > [ERROR] class FloatZip2 > > > > > /home/ec2-user/simple-drill-functions/src/main/java/com/mapr/drill/Zip.java:[131 > > ,18] error: package BaseWriter does > > not exist > > [ERROR] > > /home/ec2-user/simple-drill-functions/src/main/java/com/mapr/drill/Zip.j > > ava:[147,45] error: cannot find symbol > > [ERROR] class Zip > > > > > /home/ec2-user/simple-drill-functions/src/main/java/com/mapr/drill/Zip.java:[146 > > ,5] error: cannot find symbol > > [ERROR] class Zip > > > > > /home/ec2-user/simple-drill-functions/src/main/java/com/mapr/drill/Zip.java:[149 > > ,8] error: cannot find symbol > > [ERROR] class FloatZip3 > > > > > /home/ec2-user/simple-drill-functions/src/main/java/com/mapr/drill/Zip.java:[151 > > ,8] error: cannot find symbol > > [ERROR] class FloatZip3 > > > > > /home/ec2-user/simple-drill-functions/src/main/java/com/mapr/drill/Zip.java:[153 > > ,8] error: cannot find symbol > > [ERROR] class FloatZip3 > > > > > /home/ec2-user/simple-drill-functions/src/main/java/com/mapr/drill/Zip.java:[156 > > ,18] error: package BaseWriter does > > not exist > > [ERROR] > > /home/ec2-user/simple-drill-functions/src/main/java/com/mapr/drill/Zip.j > > ava:[172,45] error: cannot find symbol > > [ERROR] class Zip > > > > > /home/ec2-user/simple-drill-functions/src/main/java/com/mapr/drill/Zip.java:[171 > > ,5] error: cannot find symbol > > [ERROR] class Zip > > > > > /home/ec2-user/simple-drill-functions/src/main/java/com/mapr/drill/Zip.java:[174 > > ,8] error: cannot find symbol > > [ERROR] class FloatZip4 > > > > > /home/ec2-user/simple-drill-functions/src/main/java/com/mapr/drill/Zip.java:[176 > > ,8] error: cannot find symbol > > [ERROR] class FloatZip4 > > > > > /home/ec2-user/simple-drill-functions/src/main/java/com/mapr/drill/Zip.java:[178 > > ,8] error: cannot find symbol > > [ERROR] class FloatZip4 > > > > > /home/ec2-user/simple-drill-functions/src/main/java/com/mapr/drill/Zip.java:[180 > > ,8] error: cannot find symbol > > [ERROR] class FloatZip4 > > > > > /home/ec2-user/simple-drill-functions/src/main/java/com/mapr/drill/Zip.java:[183 > > ,18] error: package BaseWriter does > > not exist > > [ERROR] > > /home/ec2-user/simple-drill-functions/src/main/java/com/mapr/drill/ListS > > um.java:[3,33] error: package > > org.apache.drill.exec.expr does not exist > > [ERROR] > > /home/ec2-user/simple-drill-functions/src/main/java/com/mapr/drill/ListS > > um.java:[4,45] error: package > > org.apache.drill.exec.expr.annotations does not ex > > ist > > [ERROR] > > /home/ec2-user/simple-drill-functions/src/main/java/com/mapr/drill/ListS > > um.java:[5,45] error: package > > org.apache.drill.exec.expr.annotations does not ex > > ist > > [ERROR] > > /home/ec2-user/simple-drill-functions/src/main/java/com/mapr/drill/ListS > > um.java:[6,45] error: package > > org.apache.drill.exec.expr.annotations does not ex > > ist > > [ERROR] > > /home/ec2-user/simple-drill-functions/src/main/java/com/mapr/drill/ListS > > um.java:[7,41] error: package > > org.apache.drill.exec.expr.holders does not exist > > [ERROR] > > /home/ec2-user/simple-drill-functions/src/main/java/com/mapr/drill/ListS > > um.java:[8,41] error: package > > org.apache.drill.exec.expr.holders does not exist > > [ERROR] > > /home/ec2-user/simple-drill-functions/src/main/java/com/mapr/drill/ListS > > um.java:[9,35] error: package > > org.apache.drill.exec.vector does not exist > > [ERROR] > > /home/ec2-user/simple-drill-functions/src/main/java/com/mapr/drill/ListS > > um.java:[27,39] error: cannot find > > symbol > > [ERROR] class ListSum > > > > > /home/ec2-user/simple-drill-functions/src/main/java/com/mapr/drill/ListSum.java: > > [26,5] error: cannot find symbol > > [ERROR] class ListSum > > > > > /home/ec2-user/simple-drill-functions/src/main/java/com/mapr/drill/ListSum.java: > > [29,8] error: cannot find symbol > > > > > > ---------------------------------------------------- > > *Alok Tanna | **Business Analyst* > > > > > > c 215-410-8236 | e [email protected] | w eXert Digital > > <http://www.exertdigital.com/> > > https://www.facebook.com/exertdigital > > https://www.linkedin.com/company/exert-digital > > > > On Thu, Jun 25, 2015 at 2:24 AM, Ted Dunning <[email protected]> > > wrote: > > > > > As Vince says, get a full JDK, version 7 or above. > > > > > > On Wed, Jun 24, 2015 at 2:17 PM, Vince Gonzalez < > > [email protected]> > > > wrote: > > > > > > > Try using the JDK. > > > > > > > > Also, here's a POM: > > > > https://github.com/vicenteg/DrillJDBCExample/blob/master/pom.xml > > > > > > > > On Wed, Jun 24, 2015 at 2:10 PM, Alok Tanna <[email protected] > > > > > > wrote: > > > > > > > > > I was able to install the Drill dependence locally but now I am > > getting > > > > > complication error > > > > > > > > > > INFO] Changes detected - recompiling the module! > > > > > [WARNING] File encoding has not been set, using platform encoding > > > UTF-8, > > > > > i.e. build is platform dependent! > > > > > [INFO] Compiling 4 source files to > > > > > /home/ec2-user/simple-drill-functions/target/classes > > > > > [INFO] > ------------------------------------------------------------- > > > > > [ERROR] COMPILATION ERROR : > > > > > [INFO] > ------------------------------------------------------------- > > > > > [ERROR] No compiler is provided in this environment. Perhaps you > are > > > > > running on a JRE rather than a JDK? > > > > > [INFO] 1 error > > > > > [INFO] > ------------------------------------------------------------- > > > > > [INFO] > > > > > > > > > ------------------------------------------------------------------------ > > > > > [INFO] BUILD FAILURE > > > > > [INFO] > > > > > > > > > ------------------------------------------------------------------------ > > > > > [INFO] Total time: 1.484 s > > > > > [INFO] Finished at: 2015-06-24T18:04:05+00:00 > > > > > [INFO] Final Memory: 7M/17M > > > > > [INFO] > > > > > > > > > ------------------------------------------------------------------------ > > > > > [ERROR] Failed to execute goal > > > > > org.apache.maven.plugins:maven-compiler-plugin:3.0:compile > > > > > (default-compile) on project simple-drill-function: Compilation > > failure > > > > > [ERROR] No compiler is provided in this environment. Perhaps you > are > > > > > running on a JRE rather than a JDK? > > > > > [ERROR] -> [Help 1] > > > > > [ERROR] > > > > > [ERROR] To see the full stack trace of the errors, re-run Maven > with > > > the > > > > -e > > > > > switch. > > > > > [ERROR] Re-run Maven using the -X switch to enable full debug > > logging. > > > > > > > > > > > > > > > Thanks, > > > > > Alok Tanna > > > > > > > > > > ---------- Forwarded message ---------- > > > > > From: Alok Tanna <[email protected]> > > > > > Date: Wed, Jun 24, 2015 at 1:16 PM > > > > > Subject: Re: Custom Functions > > > > > To: [email protected] > > > > > > > > > > > > > > > Vince, If possible can you share your POM.xml file . > > > > > > > > > > > > > > > > > > > > > > > > > On Wed, Jun 24, 2015 at 1:16 PM, Alok Tanna < > [email protected] > > > > > > > > wrote: > > > > > > > > > > > Vince, If possible can you share your POM.xml file . > > > > > > > > > > > > > > > > > > On Wed, Jun 24, 2015 at 12:47 PM, Vince Gonzalez < > > > > > [email protected] > > > > > > > wrote: > > > > > > > > > > > >> Alok, > > > > > >> > > > > > >> I had to install the Drill dependencies locally since they're > not > > in > > > > > maven > > > > > >> central (DRILL-284 < > > https://issues.apache.org/jira/browse/DRILL-284 > > > > >). > > > > > >> > > > > > >> I did the following with drill-1.0.0: > > > > > >> > > > > > >> mvn install:install-file > > > > > >> > > > -Dfile=/opt/apache-drill-1.0.0/jars/drill-java-exec-1.0.0-rebuffed.jar > > > > > >> -DgroupId=org.apache.drill.exec -DartifactId=drill-java-exec > > > > > >> -Dversion=1.0.0 -Dpackaging=jar -DgeneratePom=true > > > > > >> mvn install:install-file > > > > > >> > > -Dfile=/opt/apache-drill-1.0.0/jars/drill-common-1.0.0-rebuffed.jar > > > > > >> -DgroupId=org.apache.drill -DartifactId=drill-common > > -Dversion=1.0.0 > > > > > >> -Dpackaging=jar -DgeneratePom=true > > > > > >> > > > > > >> > > > > > >> Worked for me. > > > > > >> > > > > > >> > > > > > >> On Wed, Jun 24, 2015 at 12:35 PM, Alok Tanna < > > > [email protected] > > > > > > > > > > >> wrote: > > > > > >> > > > > > >> > Thank you Ted for the below information. I would be perfectly > > fine > > > > to > > > > > >> use > > > > > >> > the Java API. > > > > > >> > I tried running the Simple Drill function . > > > > > >> > https://github.com/mapr-demos/simple-drill-functions > > > > > >> > > > > > > >> > But then when I am trying to run the package , I am getting > > below > > > > > error > > > > > >> . > > > > > >> > I am also attaching the detailed error file . > > > > > >> > > > > > > >> > [WARNING] The POM for > > > > org.apache.drill.exec:drill-java-exec:jar:1.0.0 > > > > > is > > > > > >> > missing > > > > > >> > , no dependency information available > > > > > >> > > > > > > >> > [ERROR] Failed to execute goal on project > simple-drill-function: > > > > Could > > > > > >> not > > > > > >> > resol > > > > > >> > ve dependencies for project > > > > > >> > com.mapr:simple-drill-function:jar:1.0-SNAPSHOT: Cou > > > > > >> > ld not find artifact > > > org.apache.drill.exec:drill-java-exec:jar:1.0.0 > > > > > in > > > > > >> > central > > > > > >> > (https://repo.maven.apache.org/maven2) -> [Help 1] > > > > > >> > > > > > > >> > > > > > > >> > Thanks, > > > > > >> > Alok Tanna > > > > > >> > eXertDigital > > > > > >> > > > > > > >> > > > > > > >> > On Wed, Jun 24, 2015 at 12:42 AM, Ted Dunning < > > > > [email protected]> > > > > > >> > wrote: > > > > > >> > > > > > > >> >> Yes and no. > > > > > >> >> > > > > > >> >> It would be pretty easy to build a Drill function that calls > > out > > > to > > > > > >> Jython > > > > > >> >> code. It should be relatively easy to pass a Python function > > > name > > > > in > > > > > >> as > > > > > >> >> one of the arguments as well. > > > > > >> >> > > > > > >> >> The issues with this approach are: > > > > > >> >> > > > > > >> >> 1) kinda ugly because you aren't calling your code directly > > > > > >> >> > > > > > >> >> 2) Jython isn't quite Python > > > > > >> >> > > > > > >> >> I have no idea how to do it specifically, but it is plausible > > > that > > > > > you > > > > > >> >> could build something that automatically generates the > > interlude > > > > > >> function > > > > > >> >> for you in Java that calls your Python code. Python > annotations > > > > might > > > > > >> be > > > > > >> >> the cat's pajamas for this. Or not. Hard to say. > > > > > >> >> > > > > > >> >> The Java API that Drill provides is not a traditional API in > > the > > > > > normal > > > > > >> >> sense of the word because Drill actually snarfs the source > code > > > and > > > > > >> >> inlines > > > > > >> >> it into Drill-generated code. It wouldn't understand your > > Python > > > > > code > > > > > >> at > > > > > >> >> all. > > > > > >> >> > > > > > >> >> It might be possible to teach Drill how to understand how to > > > > > transform > > > > > >> >> Python UDF's directly into something that Drill could use, > but > > > that > > > > > >> would > > > > > >> >> definitely be a bit ambitious. > > > > > >> >> > > > > > >> >> How utterly clean an interface do you need? > > > > > >> >> > > > > > >> >> > > > > > >> >> > > > > > >> >> > > > > > >> >> > > > > > >> >> > > > > > >> >> On Tue, Jun 23, 2015 at 10:10 PM, Alok Tanna < > > > > > [email protected]> > > > > > >> >> wrote: > > > > > >> >> > > > > > >> >> > Can we create Custom Functions using Python in Drill ? > > > > > >> >> > > > > > > >> >> > I do understand Drill provides a high performance Java API > > with > > > > > >> >> interfaces > > > > > >> >> > so that we can implement to develop simple and aggregate > > custom > > > > > >> >> functions. > > > > > >> >> > > > > > > >> >> > Thanks, > > > > > >> >> > Alok Tanna > > > > > >> >> > > > > > > >> >> > > > > > >> > > > > > > >> > > > > > > >> > > > > > > > > > > > > > > > > > > > > > > > > > > >
