Mihailo Aleksic created SPARK-53348:
---------------------------------------
Summary: Always persist ANSI value when creating a view or assume
it when querying if not stored
Key: SPARK-53348
URL: https://issues.apache.org/jira/browse/SPARK-53348
Project: Spark
Issue Type: Improvement
Components: SQL
Affects Versions: 4.0.0
Reporter: Mihailo Aleksic
Fix For: 4.1.0
I propose that we always store ANSI value when creating a view because
otherwise users can be affected by unwanted behavior. For example if user
creates a view on version that has ANSI = false by default he expects this not
to fail.
CREATE VIEW view AS SELECT CAST('abc' AS INT) AS a;
SELECT * FROM view.
But if user queries the view on the version which has ANSI = true by default,
above query is going to fail (because when we don't store the value, and we
store it only if explicitly set, we use the default one). Number of this and
similar use cases is huge, because ANSI impact area is huge and thus I propose
that we always store the value.
If the value is not stored, I propose that we use createVersion field to
determine whether the ANSI value should be true (Spark 4.0.0 and above) or
false (lower than Spark 4.0.0). If the createVersion field wasn't stored during
view creation, I propose that we assume that the ANSI = false because number of
those views is incomparable larger than the ones expecting ANSI = true
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
