Jan-Ole Sasse created SPARK-44420:
-------------------------------------
Summary: CheckOverflowInTableInsert does not handle cases where
the cast was moved by the optimizer
Key: SPARK-44420
URL: https://issues.apache.org/jira/browse/SPARK-44420
Project: Spark
Issue Type: Bug
Components: Optimizer
Affects Versions: 3.4.0
Reporter: Jan-Ole Sasse
CheckOverflowInTableInsert is supposed to rewrite error messages that come from
overflow exceptions on casts that are implicitly inserted for INSERTs to match
the output schema. This does not work in all cases, for example when the cast
is moved by the optimizer
{code:java}
CREATE TABLE source(value BIGINT) USING parquet
INSERT INTO source values (${Long.MaxValue.toString})
CREATE TABLE target(value INT) USING parquet
INSERT INTO target SELECT CASE WHEN value < 0 THEN 0 ELSE value END FROM
source{code}
In this case, the expression will no nothing because it only becomes effective
if the direct child is a cast ([matching
code|https://github.com/apache/spark/blob/master/sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/Cast.scala#L2186])
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
