Hi, This is working now.
This, unless someone corrects me, is the code needed to get a string parameter for a UDF: - input2.buffer.toString(input2.start, input2.end-input2.start,java.nio.charset.Charset.defaultCharset()) Regards, -Stefan (not the happiest camper) On Mon, Jul 20, 2015 at 2:07 PM, Stefán Baxter <[email protected]> wrote: > Hi, > > After going through the log this is clear what is happening (once the > Drill picked up the UDF a bit earlier this morning). > > I'm calling the VarCharHolder.toString() to get the text value for the > parameter and that is throwing this exception. > > Two observations: > > 1. Calling a deprecated function, even though that its not optimal, > usually does not cause such drastic results. > 2. As far as I can see there is no easy way to get the string value of > a property without carving out a piece of the buffer. > > What am I missing here? > > Regards, > -Stefan > > > On Mon, Jul 20, 2015 at 12:58 PM, Jacques Nadeau <[email protected]> > wrote: > >> Can you enable verbose errors at the session level? It may reveal more >> about what is failing. >> On Jul 20, 2015 5:32 AM, "Stefán Baxter" <[email protected]> >> wrote: >> >> > Hi Jim, >> > >> > I have made those changes and I'm wondering if you can runs this be >> using >> > the two .jar files that the "mvn package" places in the target >> directory? >> > >> > I have tried to have Drill pick those up but the error now is: >> > >> > Error: SYSTEM ERROR: UnsupportedOperationException >> > Fragment 0:0 >> > [Error Id: da589dd4-4cfd-4659-8b93-219074ab8c72 on localhost:31010] >> > (state=,code=0) >> > >> > It seems to indicate that it's picking up the functions but that they >> can >> > not be run. >> > >> > Regards, >> > - Stefán >> > >> > On Mon, Jul 20, 2015 at 1:25 AM, Jim Bates <[email protected]> wrote: >> > >> > > I pulled out your udf class and threw it into my package. It worked >> for >> > me >> > > with a few modifications. >> > > >> > > You can not have imported classes or method references in your eval or >> > > setup methods as the code will get pulled out and executed somewhere >> else >> > > and it won't be able to find it. With that in mind, Period will need >> to >> > be >> > > org.joda.time.Period, DateTime will need to be org.joda.time.DateTime >> and >> > > roundTimeStamp will need to be >> > > com.activitystream.drill.udfs.ASUserDefinedFunctions.roundTimeStamp. >> > > >> > > >> > > >> > > On Sun, Jul 19, 2015 at 7:02 PM, Stefán Baxter < >> > [email protected]> >> > > wrote: >> > > >> > > > Hi, >> > > > >> > > > The project can be found here: >> > > > https://github.com/acmeguy/asdrill >> > > > >> > > > Thank you, >> > > > -Stefán >> > > > >> > > > On Sun, Jul 19, 2015 at 11:57 PM, Stefán Baxter < >> > > [email protected] >> >> > > > > >> > > > wrote: >> > > > >> > > > > Hi, >> > > > > >> > > > > I'm more than happy to share the little that is there (I will >> publish >> > > it >> > > > > on github and send link tomorrow). >> > > > > >> > > > > I ended up copying my UDF (singl-file.java) into the >> > > > simple-drill-function >> > > > > project where it got picked up. >> > > > > >> > > > > Then I discovered a whole new set of dependencies/limitations >> > > > > >> > > > > - The UDF are recompiled - any imports are invalid or at least >> > > > > overwritten >> > > > > - import org.joda.time.Period; (means that Period class is not >> > > > > resolved on runtime) >> > > > > - Error: SYSTEM ERROR: CompileException: Line 71, Column 26: >> > Cannot >> > > > > determine simple type name "Period" >> > > > > >> > > > > - Calling any "outside" functions, like I was calling a static >> > > > > function of the new class (same file), leads to an errors as >> they >> > > are >> > > > not >> > > > > not resolved >> > > > > - Error: SYSTEM ERROR: CompileException: Line 70, Column 35: A >> > > method >> > > > > named "roundTimeStamp" is not declared in any enclosing class >> nor >> > > any >> > > > > supertype, nor through a static import >> > > > > >> > > > > Perhaps this was mentioned in the documentation but this is, at >> the >> > > very >> > > > > least, not straight forward and super-inviting. >> > > > > >> > > > > Thank you for your assistance, we will keep trying :) >> > > > > >> > > > > Regards, >> > > > > -Stefan >> > > > > >> > > > > >> > > > > On Sun, Jul 19, 2015 at 11:34 PM, Tugdual Grall < >> [email protected]> >> > > > wrote: >> > > > > >> > > > >> Hi Stefan, >> > > > >> >> > > > >> Do you think you can share your complete project ? >> > > > >> >> > > > >> This will help to debug it for you. >> > > > >> >> > > > >> T >> > > > >> >> > > > >> On Sunday, July 19, 2015, Stefán Baxter < >> [email protected]> >> > > > >> wrote: >> > > > >> >> > > > >> > Hi Ted, >> > > > >> > >> > > > >> > I fetched this, built it and deployed it without problems. >> > > > >> > I can not see any real difference other than this deploys two >> .jar >> > > (I >> > > > >> tried >> > > > >> > that as well earlier). >> > > > >> > >> > > > >> > I'm still trying to figure out why Drill is not picking up my >> UDFs >> > > > >> > >> > > > >> > Regards, >> > > > >> > -Stefán >> > > > >> > >> > > > >> > On Sun, Jul 19, 2015 at 10:45 PM, Ted Dunning < >> > > [email protected] >> > > > >> > <javascript:;>> wrote: >> > > > >> > >> > > > >> > > Stefan, >> > > > >> > > >> > > > >> > > Have you seen this github project: >> > > > >> > > >> > > > >> > > https://github.com/mapr-demos/simple-drill-functions >> > > > >> > > >> > > > >> > > ? >> > > > >> > > >> > > > >> > > >> > > > >> > > On Sun, Jul 19, 2015 at 2:14 PM, Stefán Baxter < >> > > > >> > [email protected] <javascript:;>> >> > > > >> > > wrote: >> > > > >> > > >> > > > >> > > > Hi Jim, >> > > > >> > > > >> > > > >> > > > I'm still not able to make this work. Do you have a sample >> > .jar >> > > > file >> > > > >> > > with a >> > > > >> > > > small example that you are running? >> > > > >> > > > >> > > > >> > > > Regards, >> > > > >> > > > -Stefan >> > > > >> > > > >> > > > >> > > > On Sun, Jul 19, 2015 at 6:46 PM, Ted Dunning < >> > > > [email protected] >> > > > >> > <javascript:;>> >> > > > >> > > > wrote: >> > > > >> > > > >> > > > >> > > > > >> > > > >> > > > > Sounds like a fine example, not because of sophistication >> > but >> > > > >> because >> > > > >> > > it >> > > > >> > > > > deals with dates. >> > > > >> > > > > >> > > > >> > > > > Check the drill logs. It is likely that drill is grumpy >> > about >> > > > >> > > something >> > > > >> > > > > in your udf or packaging. >> > > > >> > > > > >> > > > >> > > > > Also, feel free to snitch the pom from the simple >> examples >> > in >> > > > >> order >> > > > >> > to >> > > > >> > > > get >> > > > >> > > > > the pieces assembled and packaged correctly. >> > > > >> > > > > >> > > > >> > > > > Sent from my iPhone >> > > > >> > > > > >> > > > >> > > > > > On Jul 19, 2015, at 11:25, Stefán Baxter < >> > > > >> > [email protected] <javascript:;>> >> > > > >> > > > > wrote: >> > > > >> > > > > > >> > > > >> > > > > > Hi Jim, >> > > > >> > > > > > >> > > > >> > > > > > My UDF is currently so simple that I'm not sure you >> need >> > it >> > > > (or >> > > > >> > want >> > > > >> > > > it). >> > > > >> > > > > > >> > > > >> > > > > > It basically just rounds a timestamp value with ISO >> 8601 >> > > > periods >> > > > >> > > > > > asRoundTimestamp(timestampvalue,'PT10M'). >> > > > >> > > > > > >> > > > >> > > > > > I would be more than happy to contribute to your >> project >> > > > rather >> > > > >> > than >> > > > >> > > > > > building our own :). >> > > > >> > > > > > >> > > > >> > > > > > Is the repo public? >> > > > >> > > > > > >> > > > >> > > > > > Regards, >> > > > >> > > > > > -Stefan >> > > > >> > > > > > >> > > > >> > > > > > >> > > > >> > > > > > >> > > > >> > > > > >> On Sun, Jul 19, 2015 at 6:18 PM, Jim Bates < >> > > > >> [email protected] >> > > > >> > <javascript:;>> >> > > > >> > > > wrote: >> > > > >> > > > > >> >> > > > >> > > > > >> Maven will typically create a jar for class and a jar >> for >> > > > >> source >> > > > >> > > when >> > > > >> > > > > told >> > > > >> > > > > >> to do so. I just include the source files in the same >> jar >> > > as >> > > > >> the >> > > > >> > > class >> > > > >> > > > > >> files. There is a github example drill udf project we >> are >> > > > >> working >> > > > >> > on >> > > > >> > > > to >> > > > >> > > > > >> include several examples to simplify the learning >> curve. >> > If >> > > > >> your >> > > > >> > > > > >> interested... I'd love to have you add your udf. >> > > > >> > > > > >> On Jul 19, 2015 12:59 PM, "Stefán Baxter" < >> > > > >> > > [email protected] <javascript:;>> >> > > > >> > > > > >> wrote: >> > > > >> > > > > >> >> > > > >> > > > > >>> Hi again, >> > > > >> > > > > >>> >> > > > >> > > > > >>> Going over the documentation once more I came across >> > this: >> > > > >> > > > > >>> >> > > > >> > > > > >>> - Add the sources and classes JAR files to Drill’s >> > > > >> classpath. >> > > > >> > > > > >>> >> > > > >> > > > > >>> I'm only including a standard .jar (with classes no >> > > sources) >> > > > >> and >> > > > >> > I >> > > > >> > > > > wonder >> > > > >> > > > > >>> why this dependency is listed. >> > > > >> > > > > >>> >> > > > >> > > > > >>> Anyways, that it the next step. >> > > > >> > > > > >>> >> > > > >> > > > > >>> A sample .jar that works with the drill-config >> > counterpart >> > > > >> would >> > > > >> > > be a >> > > > >> > > > > >>> welcome addition to the documentation. >> > > > >> > > > > >>> >> > > > >> > > > > >>> Regards, >> > > > >> > > > > >>> -Stefan >> > > > >> > > > > >>> >> > > > >> > > > > >>> >> > > > >> > > > > >>> On Sun, Jul 19, 2015 at 5:53 PM, Stefán Baxter < >> > > > >> > > > > >> [email protected] <javascript:;>> >> > > > >> > > > > >>> wrote: >> > > > >> > > > > >>> >> > > > >> > > > > >>>> Hi Jim, >> > > > >> > > > > >>>> >> > > > >> > > > > >>>> Now I have added the file to the jar (both root and >> > > > resources >> > > > >> > > > folder) >> > > > >> > > > > >> but >> > > > >> > > > > >>>> that does not seem to change anything. >> > > > >> > > > > >>>> >> > > > >> > > > > >>>> Any additional ideas? >> > > > >> > > > > >>>> >> > > > >> > > > > >>>> Regards, >> > > > >> > > > > >>>> -Stefan >> > > > >> > > > > >>>> >> > > > >> > > > > >>>> On Sun, Jul 19, 2015 at 5:40 PM, Stefán Baxter < >> > > > >> > > > > >>> [email protected] <javascript:;>> >> > > > >> > > > > >>>> wrote: >> > > > >> > > > > >>>> >> > > > >> > > > > >>>>> NO! >> > > > >> > > > > >>>>> >> > > > >> > > > > >>>>> Thank you , will do that right now :) >> > > > >> > > > > >>>>> >> > > > >> > > > > >>>>> On Sun, Jul 19, 2015 at 5:38 PM, Jim Bates < >> > > > >> > [email protected] <javascript:;>> >> > > > >> > > > > >> wrote: >> > > > >> > > > > >>>>> >> > > > >> > > > > >>>>>> Did you include a file drill-module.conf in your >> jar >> > > > along >> > > > >> > with >> > > > >> > > > > >> source >> > > > >> > > > > >>>>>> files? >> > > > >> > > > > >>>>>> On Jul 19, 2015 12:20 PM, "Stefán Baxter" < >> > > > >> > > > > [email protected] <javascript:;> >> >> > > > >> >> > > > >> > > > > >>> >> > > > >> > > > > >>>>>> wrote: >> > > > >> > > > > >>>>>> >> > > > >> > > > > >>>>>>> Hi, >> > > > >> > > > > >>>>>>> >> > > > >> > > > > >>>>>>> I'm trying to deploy a UDF that I have written >> > > according >> > > > >> to >> > > > >> > the >> > > > >> > > > > >>>>>>> documentation. >> > > > >> > > > > >>>>>>> >> > > > >> > > > > >>>>>>> I have also: >> > > > >> > > > > >>>>>>> >> > > > >> > > > > >>>>>>> 1. Copied the jar file >> > > > >> > > > > >>>>>>> to jars/3rdparty >> > > > >> > > > > >>>>>>> >> > > > >> > > > > >>>>>>> 2. Changed the config >> "conf/drill-override.conf" >> > to >> > > > >> > include: >> > > > >> > > > > >>>>>>> drill.logical.function.package += >> > > > >> > > > > >> >> > > > >> > > >> > > > >> ["org.apache.drill.exec.expr.fn.impl","com.activitystream.drill.udfs"] >> > > > >> > > > > >>>>>>> >> > > > >> > > > > >>>>>>> 3. Rerun the Drillbit >> > > > >> > > > > >>>>>>> bin/drillbit.sh start >> > > > >> > > > > >>>>>>> >> > > > >> > > > > >>>>>>> The log shows no sign of the the UDF being loaded >> > but >> > > > if I >> > > > >> > use >> > > > >> > > it >> > > > >> > > > > >> in >> > > > >> > > > > >>> a >> > > > >> > > > > >>>>>> SQL >> > > > >> > > > > >>>>>>> statement then this error is thrown: >> > > > >> > > > > >>>>>>> >> > > > >> > > > > >>>>>>> Error: PARSE ERROR: From line 1, column 184 to >> line >> > 1, >> > > > >> column >> > > > >> > > > 219: >> > > > >> > > > > >> No >> > > > >> > > > > >>>>>> match >> > > > >> > > > > >>>>>>> found for function signature >> asRoundTimeStamp(<ANY>, >> > > > >> > > <CHARACTER>) >> > > > >> > > > > >>>>>>> >> > > > >> > > > > >>>>>>> >> > > > >> > > > > >>>>>>> The relevant part of the UDF source: >> > > > >> > > > > >>>>>>> >> > > > >> > > > > >>>>>>> package com.activitystream.drill.udfs; >> > > > >> > > > > >>>>>>> >> > > > >> > > > > >>>>>>> public class ASUserDefinedFunctions { >> > > > >> > > > > >>>>>>> >> > > > >> > > > > >>>>>>> @FunctionTemplate(name = "asRoundTimeStamp", >> > scope >> > > = >> > > > >> > > > > >>>>>>> FunctionTemplate.FunctionScope.SIMPLE, nulls = >> > > > >> > > > > >>>>>>> FunctionTemplate.NullHandling.NULL_IF_NULL) >> > > > >> > > > > >>>>>>> >> > > > >> > > > > >>>>>>> public static class RoundTimeStamp implements >> > > > >> > > DrillSimpleFunc >> > > > >> > > > > >>>>>>> >> > > > >> > > > > >>>>>>> >> > > > >> > > > > >>>>>>> Can anyone please assist? >> > > > >> > > > > >>>>>>> >> > > > >> > > > > >>>>>>> Regards, >> > > > >> > > > > >>>>>>> -Stefan >> > > > >> > > > > >> >> > > > >> > > > > >> > > > >> > > > >> > > > >> > > >> > > > >> > >> > > > >> >> > > > > >> > > > > >> > > > >> > > >> > >> > >
