[
https://issues.apache.org/jira/browse/SPARK-3947?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15013657#comment-15013657
]
Milad Bourhani commented on SPARK-3947:
---------------------------------------
I'm using vesion 1.5.0 because with 1.5.2 I have this problem:
{noformat}java.lang.NoSuchMethodError:
org.apache.spark.sql.execution.datasources.LogicalRelation.<init>(Lorg/apache/spark/sql/sources/BaseRelation;)V{noformat}
The problem I have is that the results are different depending on the Master
mode ({{local[2]}} in one case and {{spark://ubuntu:7077}} in the other).
{noformat}
// *** local[2] ***
// [[italy,emilia,87.5,87.5,175.000000000000000000,86.60254037844386],
// [italy,toscana,50.5,50.5,50.500000000000000000,50.5],
// [italy,puglia,70,70.0,70.000000000000000000,70.0]]
// *** spark://ubuntu:7077 ***
//
[[italy,emilia,87.5,2.54639494926E-313,175.000000000000000000,1.000000000064921],
//
[italy,toscana,50.5,5.09278989856E-313,50.500000000000000000,1.000000000028536],
//
[italy,puglia,70,5.09278989856E-313,70.000000000000000000,1.000000000030912]]
{noformat}
> Support Scala/Java UDAF
> -----------------------
>
> Key: SPARK-3947
> URL: https://issues.apache.org/jira/browse/SPARK-3947
> Project: Spark
> Issue Type: Sub-task
> Components: SQL
> Reporter: Pei-Lun Lee
> Assignee: Yin Huai
> Fix For: 1.5.0
>
> Attachments: spark-udaf.zip
>
>
> Right now only Hive UDAFs are supported. It would be nice to have UDAF
> similar to UDF through SQLContext.registerFunction.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
