[
https://issues.apache.org/jira/browse/SPARK-19102?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Hyukjin Kwon resolved SPARK-19102.
----------------------------------
Resolution: Incomplete
> Accuracy error of spark SQL results
> -----------------------------------
>
> Key: SPARK-19102
> URL: https://issues.apache.org/jira/browse/SPARK-19102
> Project: Spark
> Issue Type: Bug
> Components: SQL
> Affects Versions: 1.6.0, 1.6.1
> Environment: Spark 1.6.0, Hadoop 2.6.0,JDK 1.8,CentOS6.6
> Reporter: XiaodongCui
> Priority: Major
> Labels: bulk-closed
> Attachments: a.zip
>
>
> the problem is cube6's second column named sumprice is 10000 times bigger
> than the cube5's second column named sumprice,but they should be equal .the
> first sql is "SELECT areacode1, SUM(quantity*unitprice) AS sumprice FROM
> hd_salesflat GROUP BY areacode1" , the second sql is "SELECT areacode1,
> SUM(quantity*unitprice) AS sumprice, COUNT(DISTINCT transno) FROM
> hd_salesflat GROUP BY areacode1",the result of sumprice should be equal,but
> actually they are not ,that's the problem.
> code:
> ================================================================================
> DataFrame
> df1=sqlContext.read().parquet("hdfs://cdh01:8020/sandboxdata_A/test/a");
> df1.registerTempTable("hd_salesflat");
> DataFrame cube5 = sqlContext.sql("SELECT areacode1,
> SUM(quantity*unitprice) AS sumprice FROM hd_salesflat GROUP BY areacode1");
> DataFrame cube6 = sqlContext.sql("SELECT areacode1,
> SUM(quantity*unitprice) AS sumprice, COUNT(DISTINCT transno) FROM
> hd_salesflat GROUP BY areacode1");
> cube5.select("sumprice").show(50);
> cube6.select("sumprice").show(50);
> ================================================================================
> my data has only one row and four column in the attach file：
> transno | quantity | unitprice | areacode1
> 76317828| 1.0000 | 25.0000 | HDCN
> data schema：
> |-- areacode1: string (nullable = true)
> |-- quantity: decimal(20,4) (nullable = true)
> |-- unitprice: decimal(20,4) (nullable = true)
> |-- transno: string (nullable = true)
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
