[jira] [Commented] (SPARK-26218) Throw exception on overflow for integers
[ https://issues.apache.org/jira/browse/SPARK-26218?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17242960#comment-17242960 ] Apache Spark commented on SPARK-26218: -- User 'luluorta' has created a pull request for this issue: https://github.com/apache/spark/pull/30585 > Throw exception on overflow for integers > > > Key: SPARK-26218 > URL: https://issues.apache.org/jira/browse/SPARK-26218 > Project: Spark > Issue Type: Sub-task > Components: SQL >Affects Versions: 3.0.0 >Reporter: Marco Gaido >Assignee: Marco Gaido >Priority: Major > Fix For: 3.0.0 > > > SPARK-24598 just updated the documentation in order to state that our > addition is a Java style one and not a SQL style. But in order to follow the > SQL standard we should instead throw an exception if an overflow occurs. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-26218) Throw exception on overflow for integers
[ https://issues.apache.org/jira/browse/SPARK-26218?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17242959#comment-17242959 ] Apache Spark commented on SPARK-26218: -- User 'luluorta' has created a pull request for this issue: https://github.com/apache/spark/pull/30585 > Throw exception on overflow for integers > > > Key: SPARK-26218 > URL: https://issues.apache.org/jira/browse/SPARK-26218 > Project: Spark > Issue Type: Sub-task > Components: SQL >Affects Versions: 3.0.0 >Reporter: Marco Gaido >Assignee: Marco Gaido >Priority: Major > Fix For: 3.0.0 > > > SPARK-24598 just updated the documentation in order to state that our > addition is a Java style one and not a SQL style. But in order to follow the > SQL standard we should instead throw an exception if an overflow occurs. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-26218) Throw exception on overflow for integers
[ https://issues.apache.org/jira/browse/SPARK-26218?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16816087#comment-16816087 ] Marco Gaido commented on SPARK-26218: - [~rxin] I see that. But the reason for this are: - the current behavior is against the SQL standard; - having incorrect results in some use cases is worse than having a job failure; - introducing a config which allows to choose which behavior to have allows users to decide which is the best option for their specific use cases. Moreover I can also see people maybe having the check for overflow turned on in dev environments, so that they can find issues and turned off in prod in order to avoid wasting hours as you mentioned. > Throw exception on overflow for integers > > > Key: SPARK-26218 > URL: https://issues.apache.org/jira/browse/SPARK-26218 > Project: Spark > Issue Type: Sub-task > Components: SQL >Affects Versions: 3.0.0 >Reporter: Marco Gaido >Priority: Major > > SPARK-24598 just updated the documentation in order to state that our > addition is a Java style one and not a SQL style. But in order to follow the > SQL standard we should instead throw an exception if an overflow occurs. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-26218) Throw exception on overflow for integers
[ https://issues.apache.org/jira/browse/SPARK-26218?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16814709#comment-16814709 ] Reynold Xin commented on SPARK-26218: - The no-exception is by design. Imagine you have an ETL job that runs for hours, and then it suddenly throws an exception because one row overflows ... > Throw exception on overflow for integers > > > Key: SPARK-26218 > URL: https://issues.apache.org/jira/browse/SPARK-26218 > Project: Spark > Issue Type: Sub-task > Components: SQL >Affects Versions: 3.0.0 >Reporter: Marco Gaido >Priority: Major > > SPARK-24598 just updated the documentation in order to state that our > addition is a Java style one and not a SQL style. But in order to follow the > SQL standard we should instead throw an exception if an overflow occurs. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-26218) Throw exception on overflow for integers
[ https://issues.apache.org/jira/browse/SPARK-26218?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16703266#comment-16703266 ] Apache Spark commented on SPARK-26218: -- User 'mgaido91' has created a pull request for this issue: https://github.com/apache/spark/pull/21599 > Throw exception on overflow for integers > > > Key: SPARK-26218 > URL: https://issues.apache.org/jira/browse/SPARK-26218 > Project: Spark > Issue Type: Sub-task > Components: SQL >Affects Versions: 3.0.0 >Reporter: Marco Gaido >Priority: Major > > SPARK-24598 just updated the documentation in order to state that our > addition is a Java style one and not a SQL style. But in order to follow the > SQL standard we should instead throw an exception if an overflow occurs. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-26218) Throw exception on overflow for integers
[ https://issues.apache.org/jira/browse/SPARK-26218?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16703270#comment-16703270 ] Apache Spark commented on SPARK-26218: -- User 'mgaido91' has created a pull request for this issue: https://github.com/apache/spark/pull/21599 > Throw exception on overflow for integers > > > Key: SPARK-26218 > URL: https://issues.apache.org/jira/browse/SPARK-26218 > Project: Spark > Issue Type: Sub-task > Components: SQL >Affects Versions: 3.0.0 >Reporter: Marco Gaido >Priority: Major > > SPARK-24598 just updated the documentation in order to state that our > addition is a Java style one and not a SQL style. But in order to follow the > SQL standard we should instead throw an exception if an overflow occurs. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
