cloud-fan commented on issue #25280: [SPARK-28548][SQL] explain() shows wrong result for persisted DataFrames after some operations URL: https://github.com/apache/spark/pull/25280#issuecomment-527012092 This is not a correctness issue. This is just an undocumented property "a dataframe can't change its physical plan once the physical plan is materialized" causing confusions. After so many days, I have a fresh idea now: when `df.cache` is called, reset the `cachedPlan`, `optimizedPlan` and physical plans. We can implement the lazy evaluation manually to support it instead of relying on Scals lazy val.
---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected] With regards, Apache Git Services --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
