Brilliant... It Worked ..
i should give --jars
instead of add jar statement
On Jan 18, 2017 9:02 AM, "Deepak Sharma" <deepakmc...@gmail.com> wrote:
> Did you tried this with spark-shell?
> Please try this.
> $spark-shell --jars /home/cloudera/Downloads/genudnvl2.jar
>
> On the spark shell:
> val hc = new org.apache.spark.sql.hive.HiveContext(sc) ;
> hc.sql("create temporary function nexr_nvl2 as '
> com.nexr.platform.hive.udf.GenericUDFNVL2'");
> hc.sql("select nexr_nvl2(name,let,ret) from testtab5").show;
>
> This should work.
>
> So basically in your spark program , you can specify the jars while
> submitting the job using spark-submit.
>
> There is no need to have add jars statement in the spark program itself.
>
> Thanks
> Deepak
>
>
> On Wed, Jan 18, 2017 at 8:58 AM, Sirisha Cheruvu <siri8...@gmail.com>
> wrote:
>
>> This error
>> org.apache.spark.sql.AnalysisException: No handler for Hive udf class
>> com.nexr.platform.hive.udf.GenericUDFNVL2 because:
>> com.nexr.platform.hive.udf.GenericUDFNVL2.; line 1 pos 26
>> at org.apache.spark.sql.hive.HiveFunctionRegistry$$anonfun$look
>> upFunction$2.apply(hiveUDFs.scala:105)
>> at org.apache.spark.sql.hive.HiveFunctionRegistry$$anonfun$look
>> upFunction$2.apply(hiveUDFs.scala:64)
>> at scala.util.Try.getOrElse(Try.scala:77)
>>
>>
>>
>>
>>
>> My script is this
>>
>>
>> import org.apache.spark.sql.hive.HiveContext
>> val hc = new org.apache.spark.sql.hive.HiveContext(sc) ;
>> hc.sql("add jar /home/cloudera/Downloads/genudnvl2.jar");
>> hc.sql("create temporary function nexr_nvl2 as '
>> com.nexr.platform.hive.udf.GenericUDFNVL2'");
>> hc.sql("select nexr_nvl2(name,let,ret) from testtab5").show;
>> System.exit(0);
>>
>>
>> On Jan 17, 2017 2:01 PM, "Sirisha Cheruvu" <siri8...@gmail.com> wrote:
>>
>>> Hi
>>>
>>> Anybody has a test and tried generic udf with object inspector
>>> implementaion which sucessfully ran on both hive and spark-sql
>>>
>>> please share me the git hub link or source code file
>>>
>>> Thanks in advance
>>> Sirisha
>>>
>>
>
>
> --
> Thanks
> Deepak
> www.bigdatabig.com
> www.keosha.net
>
