Re: Best way to use Spark UDFs via Hive (Spark Thrift Server)

[Deenar Toraskar]

You can do the following. Start the spark-shell. Register the UDFs in the
shell using sqlContext, then start the Thrift Server using startWithContext
from the spark shell:
https://github.com/apache/spark/blob/master/sql/hive-thriftserver/src/main/scala/org/apache/spark/sql/hive/thriftserver/HiveThriftServer2.scala#L56


Regards
Deenar

On 23 October 2015 at 01:15, Dave Moyers  wrote:

> Hi,
>
> We have several udf's written in Scala that we use within jobs submitted
> into Spark. They work perfectly with the sqlContext after being registered.
> We also allow access to saved tables via the Hive Thrift server bundled
> with Spark. However, we would like to allow Hive connections to use the
> udf's in their queries against the saved tables. Is there a way to register
> udf's such that they can be used within both a Spark job and in a Hive
> connection?
>
> Thanks!
> Dave
>
> Sent from my iPad
> -
> To unsubscribe, e-mail: user-unsubscr...@spark.apache.org
> For additional commands, e-mail: user-h...@spark.apache.org
>
>

Best way to use Spark UDFs via Hive (Spark Thrift Server)

[Dave Moyers]

Hi,

We have several udf's written in Scala that we use within jobs submitted into 
Spark. They work perfectly with the sqlContext after being registered. We also 
allow access to saved tables via the Hive Thrift server bundled with Spark. 
However, we would like to allow Hive connections to use the udf's in their 
queries against the saved tables. Is there a way to register udf's such that 
they can be used within both a Spark job and in a Hive connection?

Thanks!
Dave

Sent from my iPad
-
To unsubscribe, e-mail: user-unsubscr...@spark.apache.org
For additional commands, e-mail: user-h...@spark.apache.org