Linhong Liu created SPARK-61570:
-----------------------------------
Summary: Cached query that calls a session-context function
(current_timezone / current_catalog / current_schema / current_user) is reused
under a different session context
Key: SPARK-61570
URL: https://issues.apache.org/jira/browse/SPARK-61570
Project: Spark
Issue Type: Bug
Components: SQL
Affects Versions: 5.0.0
Reporter: Linhong Liu
A cached query (CACHE TABLE, Dataset.cache()) that calls a session-context
function such as current_timezone(), current_catalog(),
current_schema()/current_database() or current_user() can be answered from a
cache entry created under a different session context, returning that context's
values. These functions have no fields, so they canonicalize identically in
every session and the normalized cache-key plan matches; they are bound
(ReplaceCurrentLike) only after the cache lookup.
Fix: record the session values the plan's session-context functions read when
it is cached, and reuse the entry only when the looking-up session's values
match. A new conf spark.sql.legacy.cache.ignoreSessionContextInPlanMatching
(default false) restores the previous behavior. Found via correctness analysis.
--
This message was sent by Atlassian Jira
(v10.3.26#10030026)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]