Re: need a hive generic udf which also works on spark sql

2017-01-17 Thread Sirisha Cheruvu
Thank You Deepak On Jan 18, 2017 9:02 AM, "Deepak Sharma" wrote: > Did you tried this with spark-shell? > Please try this. > $spark-shell --jars /home/cloudera/Downloads/genudnvl2.jar > > On the spark shell: > val hc = new org.apache.spark.sql.hive.HiveContext(sc) ; >

Re: need a hive generic udf which also works on spark sql

2017-01-17 Thread Sirisha Cheruvu
Brilliant... It Worked .. i should give --jars instead of add jar statement On Jan 18, 2017 9:02 AM, "Deepak Sharma" wrote: > Did you tried this with spark-shell? > Please try this. > $spark-shell --jars /home/cloudera/Downloads/genudnvl2.jar > > On the spark shell: >

Re: need a hive generic udf which also works on spark sql

2017-01-17 Thread Deepak Sharma
Did you tried this with spark-shell? Please try this. $spark-shell --jars /home/cloudera/Downloads/genudnvl2.jar On the spark shell: val hc = new org.apache.spark.sql.hive.HiveContext(sc) ; hc.sql("create temporary function nexr_nvl2 as ' com.nexr.platform.hive.udf.GenericUDFNVL2'");

Re: need a hive generic udf which also works on spark sql

2017-01-17 Thread Sirisha Cheruvu
This error org.apache.spark.sql.AnalysisException: No handler for Hive udf class com.nexr.platform.hive.udf.GenericUDFNVL2 because: com.nexr.platform.hive.udf.GenericUDFNVL2.; line 1 pos 26 at org.apache.spark.sql.hive.HiveFunctionRegistry$$anonfun$ lookupFunction$2.apply(hiveUDFs.scala:105) at

Re: need a hive generic udf which also works on spark sql

2017-01-17 Thread Deepak Sharma
On the sqlcontext or hivesqlcontext , you can register the function as udf below: *hiveSqlContext.udf.register("func_name",func(_:String))* Thanks Deepak On Wed, Jan 18, 2017 at 8:45 AM, Sirisha Cheruvu wrote: > Hey > > Can yu send me the source code of hive java udf which

Re: need a hive generic udf which also works on spark sql

2017-01-17 Thread Takeshi Yamamuro
Hi, AFAIK, you could use Hive GenericUDF stuffs in spark without much effort. If you'd like to check test suites about that, you'd better to visit HiveUDFSuite. https://github.com/apache/spark/blob/master/sql/hive/src/test/scala/org/apache/spark/sql/hive/execution/HiveUDFSuite.scala I also have

need a hive generic udf which also works on spark sql

2017-01-17 Thread Sirisha Cheruvu
Hi Anybody has a test and tried generic udf with object inspector implementaion which sucessfully ran on both hive and spark-sql please share me the git hub link or source code file Thanks in advance Sirisha