cloud-fan commented on code in PR #58317:
URL: https://github.com/apache/spark/pull/58317#discussion_r3992302185
##########
sql/core/src/main/scala/org/apache/spark/sql/execution/CacheManager.scala:
##########
@@ -348,6 +349,45 @@ class CacheManager extends Logging with
AdaptiveSparkPlanHelper {
recacheByCondition(spark, _.plan.exists(_.sameResult(normalized)))
}
+ /**
+ * Re-caches every entry whose plan contains a [[LogicalRelation]] for
`relation`.
+ * Unlike [[recacheByPlan]], this ignores CHAR/VARCHAR scan-mode identity so
a V1 write
+ * invalidates preserve-only, standard, and unbound cache entries for that
BaseRelation.
+ */
+ def recacheByV1Relation(spark: SparkSession, relation: BaseRelation): Unit =
{
+ recacheByCondition(spark, cd => cd.plan.exists {
+ case logical: LogicalRelation => logical.relation == relation
+ case _ => false
+ })
+ }
+
+ /**
+ * Re-caches every entry whose plan contains the given catalog-less
[[DataSourceV2Relation]].
+ * The scan mode is ignored only for this mutation-specific match so an
unbound write target
+ * invalidates preserve-native and standard cache entries without weakening
normal cache identity.
+ */
+ def recacheByV2Relation(spark: SparkSession, relation:
DataSourceV2Relation): Unit = {
+ val unboundRelation = relation.copy(charVarcharScanMode = None)
+ recacheByCondition(spark, cd => cd.plan.exists {
+ case cached: DataSourceV2Relation =>
+ cached.copy(charVarcharScanMode = None).sameResult(unboundRelation)
+ case _ => false
+ })
+ }
+
+ /**
+ * Looks up a cache entry for a V2 table mutation while ignoring only its
analyzed CHAR/VARCHAR
+ * scan mode. Normal cache substitution remains mode-sensitive.
+ */
+ def lookupCachedDataByV2Relation(relation: DataSourceV2Relation):
Option[CachedData] = {
+ val unboundRelation = relation.copy(charVarcharScanMode = None)
+ cachedData.find(_.plan.exists {
Review Comment:
**Non-blocking (P2):** This lookup is used to decide whether the table
itself must be recached after rename, but `plan.exists` also matches a cached
parent query that merely reads the table. If only `CACHE TABLE q AS SELECT ...
FROM t` exists, renaming `t` selects `q`, invalidates it, and leaves the
renamed base table cached even though it was never directly cached. Please
compare the complete cache key after clearing only the scan mode; descendant
matching should remain limited to write recaching.
**Recommended change:** Replace the singular descendant-based rename lookup
with a direct-root, mode-insensitive query that returns every matching
CachedData variant, and pass each variant's StorageLevel and
CharVarcharScanMode through rename restoration.
**Why this works:** Eliminate only harmless root aliases, require the
complete cached key plan to be the target DataSourceV2Relation after clearing
charVarcharScanMode on both sides, collect all matches, snapshot each
mode/storage pair, invalidate old-name dependents as today, then create and
cache one destination relation per prior direct variant.
**Scope:** sql/core/src/main/scala/org/apache/spark/sql/execution,
sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/v2,
sql/core/src/test/scala/org/apache/spark/sql/CachedTableSuite.scala
**Compatibility:** Mutation recache operations may still traverse
descendants to refresh queries after writes; only rename's direct table-cache
discovery becomes root-specific.
**Risks:** Over-normalizing aliases or options could merge unrelated table
or time-travel cache keys. Recaching several variants under one display name
must retain distinct semantic plan keys and individual storage levels.
**Constraints:** Ignore only charVarcharScanMode for this mutation-specific
discovery; normal cache substitution remains mode-sensitive. Do not include
dependent query caches or time-travel variants in the set restored as the
renamed base table. Continue invalidating caches that reference the old table
name after the snapshot is taken.
**Success:** Renaming when only a dependent query is cached does not cache
the destination table. Renaming a table with simultaneous PreserveNative and
SparkStandard caches restores both under the new identifier with their original
storage levels. A single directly cached table retains the current rename
behavior.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
