[ https://issues.apache.org/jira/browse/SPARK-25413?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Dongjoon Hyun resolved SPARK-25413. ----------------------------------- Resolution: Invalid Thank you for reporting and making a PR, [~sandeep.katta2007]. According to the discussion on the JIRA issue and the given example is controlled by configuration, I'll close this as `Invalid` issue. {code} scala> sql("set spark.sql.decimalOperations.allowPrecisionLoss=false") scala> sql("select avg(salary)+10 from hiveBigDecimal").show(false) +----------------------------------------------------------------------------------------+ |(CAST(avg(salary) AS DECIMAL(32,14)) + CAST(CAST(10 AS DECIMAL(2,0)) AS DECIMAL(32,14)))| +----------------------------------------------------------------------------------------+ |37800224355780013.75982042536364 | +----------------------------------------------------------------------------------------+ {code} > [2.3.2.rc5 Blocker] Precision Value is going for toss when Avg is done > ---------------------------------------------------------------------- > > Key: SPARK-25413 > URL: https://issues.apache.org/jira/browse/SPARK-25413 > Project: Spark > Issue Type: Bug > Components: Spark Core > Affects Versions: 2.3.1 > Environment: Csv FIle content > > 1,23-07-2015,India,aaa1,phone197,ASD69643,1.23457E+16 > 2,24-07-2015,India,aaa2,phone756,ASD42892,1.23457E+16 > 3,25-07-2015,India,aaa3,phone1904,ASD37014,1.23457E+16 > 4,26-07-2015,India,aaa4,phone2435,ASD66902,1.23457E+16 > 5,27-07-2015,India,aaa5,phone2441,ASD90633,2.23457E+16 > 6,28-07-2015,India,aaa6,phone294,ASD59961,3.23457E+16 > 7,29-07-2015,India,aaa7,phone610,ASD14875,4.23457E+16 > 8,30-07-2015,India,aaa8,phone1848,ASD57308,5.23457E+16 > 9,18-07-2015,India,aaa9,phone706,ASD86717,6.23457E+16 > 10,19-07-2015,usa,aaa10,phone685,ASD30505,7.23457E+16 > 11,18-07-2015,china,aaa11,phone1554,ASD26101,8.23457E+16 > Reporter: sandeep katta > Priority: Blocker > Attachments: decimalBoundaryDataHive.csv > > > sql("create table if not exists hiveBigDecimal(ID Int, date Timestamp, > country String, name String, phonetype String, serialname String, salary > decimal(27, 10))row format delimited fields terminated by ','") > sql(s"LOAD DATA local inpath '$resourcesPath/decimalBoundaryDataHive.csv' > INTO table hiveBigDecimal") > sql("select avg(salary)+10 from hiveBigDecimal").show(false) > > Output with 2.3.1 > +----------------------------------------------------------------------------------------+ > |(CAST(avg(salary) AS DECIMAL(32,14)) + CAST(CAST(10 AS DECIMAL(2,0)) AS > DECIMAL(32,14)))| > +----------------------------------------------------------------------------------------+ > |37800224355780013.75982042536364| > +----------------------------------------------------------------------------------------+ > OutPut with 2.3.2_RC5 > |(CAST(avg(salary) AS DECIMAL(32,14)) + CAST(CAST(10 AS DECIMAL(2,0)) AS > DECIMAL(32,14)))| > +----------------------------------------------------------------------------------------+ > |37800224355780013.75982042536000 > | > +---------------------------------------------------------------------------------------- > *PS:If I revert SPARK-24957 then 2.3.1 and 2.3.2_rc5 output is same* > -- This message was sent by Atlassian JIRA (v7.6.3#76005) --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org