Yuming Wang created SPARK-39173:
-----------------------------------
Summary: The error message is different if disable broadcast join
Key: SPARK-39173
URL: https://issues.apache.org/jira/browse/SPARK-39173
Project: Spark
Issue Type: Bug
Components: SQL
Affects Versions: 3.4.0
Reporter: Yuming Wang
How to reproduce this issue:
{code:scala}
Seq(-1, 1000000000L).foreach { broadcastThreshold =>
withSQLConf(
SQLConf.AUTO_BROADCASTJOIN_THRESHOLD.key -> broadcastThreshold.toString,
SQLConf.ANSI_ENABLED.key -> "true") {
val df = sql(
"""
|SELECT
| item.i_brand_id brand_id,
| avg(ss_ext_sales_price) avg_agg
|FROM store_sales, item
|WHERE store_sales.ss_item_sk = item.i_item_sk
|GROUP BY item.i_brand_id
""".stripMargin)
df.collect()
}
}
{code}
{noformat}
Error message: Job aborted due to stage failure: Task 0 in stage 10.0 failed 1
times, most recent failure: Lost task 0.0 in stage 10.0 (TID 9) (localhost
executor driver): org.apache.spark.SparkArithmeticException:
[CANNOT_CHANGE_DECIMAL_PRECISION]
Decimal(expanded,999999999999999999999999999999999.28175,38,5}) cannot be
represented as Decimal(38, 6). If necessary set "spark.sql.ansi.enabled" to
false to bypass this error.
Error message: Job aborted due to stage failure: Task 0 in stage 14.0 failed 1
times, most recent failure: Lost task 0.0 in stage 14.0 (TID 14) (localhost
executor driver): org.apache.spark.SparkArithmeticException:
[ARITHMETIC_OVERFLOW] Overflow in sum of decimals. If necessary set
spark.sql.ansi.enabled to false (except for ANSI interval type) to bypass this
error.
{noformat}
--
This message was sent by Atlassian Jira
(v8.20.7#820007)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
