[
https://issues.apache.org/jira/browse/SPARK-30893?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17041567#comment-17041567
]
Wenchen Fan commented on SPARK-30893:
-------------------------------------
My assumption is: after you create a `df`, calling `df.collect` multiple times
should always return the same result even if you change some configs.
If this is not what we expect, then we should only fix the data type changes
and leave the beavior ones.
cc [~hvanhovell] [~viirya] [~maropu] what's your opinion?
> Expressions should not change its data type/behavior after it's created
> -----------------------------------------------------------------------
>
> Key: SPARK-30893
> URL: https://issues.apache.org/jira/browse/SPARK-30893
> Project: Spark
> Issue Type: Bug
> Components: SQL
> Affects Versions: 3.0.0
> Reporter: Wenchen Fan
> Priority: Critical
>
> This is a problem because the configuration can change between different
> phases of planning, and this can silently break a query plan which can lead
> to crashes or data corruption.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
