[
https://issues.apache.org/jira/browse/SPARK-3947?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15013657#comment-15013657
]
Milad Bourhani edited comment on SPARK-3947 at 11/19/15 3:05 PM:
-----------------------------------------------------------------
I'm using vesion 1.5.0 because with 1.5.2 I have this problem:
{noformat}java.lang.NoSuchMethodError:
org.apache.spark.sql.execution.datasources.LogicalRelation.<init>(Lorg/apache/spark/sql/sources/BaseRelation;)V{noformat}
The problem I have is that the results are different depending on the Master
mode.
{{local[2]}} (correct):
{noformat}
[[italy,emilia,87.5,175.000000000000000000,86.60254037844386],
[italy,toscana,50.5,50.500000000000000000,50.5],
[italy,puglia,70,70.000000000000000000,70.0]]
{noformat}
{{spark://ubuntu:7077}} (wrong):
{noformat}
[[italy,emilia,87.5,175.000000000000000000,7.136378562382804E-157],
[italy,toscana,50.5,50.500000000000000000,5.09278989856E-313],
[italy,puglia,70,70.000000000000000000,5.09278989856E-313]]
{noformat}
was (Author: [email protected]):
I'm using vesion 1.5.0 because with 1.5.2 I have this problem:
{noformat}java.lang.NoSuchMethodError:
org.apache.spark.sql.execution.datasources.LogicalRelation.<init>(Lorg/apache/spark/sql/sources/BaseRelation;)V{noformat}
The problem I have is that the results are different depending on the Master
mode ({{local[2]}} in one case and {{spark://ubuntu:7077}} in the other).
Local (correct):
{noformat}
[[italy,emilia,87.5,175.000000000000000000,86.60254037844386],
[italy,toscana,50.5,50.500000000000000000,50.5],
[italy,puglia,70,70.000000000000000000,70.0]]
{noformat}
Remote (wrong):
{noformat}
[[italy,emilia,87.5,175.000000000000000000,7.136378562382804E-157],
[italy,toscana,50.5,50.500000000000000000,5.09278989856E-313],
[italy,puglia,70,70.000000000000000000,5.09278989856E-313]]
{noformat}
> Support Scala/Java UDAF
> -----------------------
>
> Key: SPARK-3947
> URL: https://issues.apache.org/jira/browse/SPARK-3947
> Project: Spark
> Issue Type: Sub-task
> Components: SQL
> Reporter: Pei-Lun Lee
> Assignee: Yin Huai
> Fix For: 1.5.0
>
> Attachments: spark-udaf.zip
>
>
> Right now only Hive UDAFs are supported. It would be nice to have UDAF
> similar to UDF through SQLContext.registerFunction.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
