[
https://issues.apache.org/jira/browse/SPARK-53348?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Mihailo Aleksic updated SPARK-53348:
------------------------------------
Description:
I propose that we always store ANSI value when creating a view because
otherwise users can be affected by unwanted behavior. For example if user
creates a view on version that has ANSI = false by default he expects this not
to fail.
CREATE VIEW view AS SELECT CAST('abc' AS INT) AS a;
SELECT * FROM view;
But if user queries the view on the version which has ANSI = true by default,
above query is going to fail (because when we don't store the value, and we
store it only if explicitly set, we use the default one). Number of this and
similar use cases is huge, because ANSI impact area is huge and thus I propose
that we always store the value.
If the value is not stored, I propose that we use createVersion field to
determine whether the ANSI value should be true (Spark 4.0.0 and above) or
false (lower than Spark 4.0.0). If the createVersion field wasn't stored during
view creation, I propose that we assume that the ANSI = false because number of
those views is incomparable larger than the ones expecting ANSI = true
was:
I propose that we always store ANSI value when creating a view because
otherwise users can be affected by unwanted behavior. For example if user
creates a view on version that has ANSI = false by default he expects this not
to fail.
CREATE VIEW view AS SELECT CAST('abc' AS INT) AS a;
SELECT * FROM view.
But if user queries the view on the version which has ANSI = true by default,
above query is going to fail (because when we don't store the value, and we
store it only if explicitly set, we use the default one). Number of this and
similar use cases is huge, because ANSI impact area is huge and thus I propose
that we always store the value.
If the value is not stored, I propose that we use createVersion field to
determine whether the ANSI value should be true (Spark 4.0.0 and above) or
false (lower than Spark 4.0.0). If the createVersion field wasn't stored during
view creation, I propose that we assume that the ANSI = false because number of
those views is incomparable larger than the ones expecting ANSI = true
> Always persist ANSI value when creating a view or assume it when querying if
> not stored
> ---------------------------------------------------------------------------------------
>
> Key: SPARK-53348
> URL: https://issues.apache.org/jira/browse/SPARK-53348
> Project: Spark
> Issue Type: Improvement
> Components: SQL
> Affects Versions: 4.0.0
> Reporter: Mihailo Aleksic
> Priority: Major
> Fix For: 4.1.0
>
>
> I propose that we always store ANSI value when creating a view because
> otherwise users can be affected by unwanted behavior. For example if user
> creates a view on version that has ANSI = false by default he expects this
> not to fail.
> CREATE VIEW view AS SELECT CAST('abc' AS INT) AS a;
> SELECT * FROM view;
> But if user queries the view on the version which has ANSI = true by default,
> above query is going to fail (because when we don't store the value, and we
> store it only if explicitly set, we use the default one). Number of this and
> similar use cases is huge, because ANSI impact area is huge and thus I
> propose that we always store the value.
> If the value is not stored, I propose that we use createVersion field to
> determine whether the ANSI value should be true (Spark 4.0.0 and above) or
> false (lower than Spark 4.0.0). If the createVersion field wasn't stored
> during view creation, I propose that we assume that the ANSI = false because
> number of those views is incomparable larger than the ones expecting ANSI =
> true
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
