Re: Is it okay to run Hive Java UDFS in Spark-sql. Anybody's still doing it?

[Jörn Franke]

There are many performance aspects here which may not only related to the UDF 
itself, but on configuration of platform, data etc.

You seem to have a performance problem with your UDFs. Maybe you can elaborate 
on 
1) what data you process (format, etc)
2) what you try to Analyse
3) how you implemented your udfs. Maybe the implementation is not optimal and 
then simply moving it from hive to spark does not give you any benefits. Bad 
code is still bad code in SparkSql


> On 2 Feb 2017, at 09:33, Alex  wrote:
> 
> Hi Team,
> 
> Do you really think if we make Hive Java UDF's to run on spark-sql  it will 
> make performance difference???  IS anybody here actually doing it.. 
> converting Hive UDF's to run on Spark-sql..
> 
> What would be your approach if asked to make Hive Java UDFS project run on 
> spark-sql
> 
> Would yu run the same java UDF using Spark-sql
>  
> or 
> 
> You would recode all java UDF to scala UDF and then run?
> 
> 
> Regards,
> Alex

-
To unsubscribe e-mail: user-unsubscr...@spark.apache.org

Is it okay to run Hive Java UDFS in Spark-sql. Anybody's still doing it?

[Alex]

Hi Team,

Do you really think if we make Hive Java UDF's to run on spark-sql  it will
make performance difference???  IS anybody here actually doing it..
converting Hive UDF's to run on Spark-sql..

What would be your approach if asked to make Hive Java UDFS project run on
spark-sql

Would yu run the same java UDF using Spark-sql

or

You would recode all java UDF to scala UDF and then run?


Regards,
Alex