This is an automated email from the ASF dual-hosted git repository.
cloud-fan pushed a commit to branch branch-4.x
in repository https://gitbox.apache.org/repos/asf/spark.git
The following commit(s) were added to refs/heads/branch-4.x by this push:
new 40b9e057dd16 [SPARK-58604][SQL] Preserve viewDependencies when
rebuilding a View for ALTER VIEW
40b9e057dd16 is described below
commit 40b9e057dd1644822c56fe748ec6897d6ff2ee8f
Author: Szehon Ho <[email protected]>
AuthorDate: Thu Aug 6 16:05:12 2026 +0800
[SPARK-58604][SQL] Preserve viewDependencies when rebuilding a View for
ALTER VIEW
### What changes were proposed in this pull request?
`CatalogV2Util.viewInfoBuilderFrom` seeds a `View.Builder` from an existing
view so ALTER VIEW execs can override the one field that changes and leave
everything else untouched. It copies the schema, properties, query text, SQL
configs, current namespace, current catalog, query column names and schema mode
— but not the typed `viewDependencies` field.
This PR carries `viewDependencies` through, consistent with how the other
nullable fields (`currentCatalog`, `schemaMode`) are already handled.
### Why are the changes needed?
All three callers of `viewInfoBuilderFrom` are metadata-only mutations that
do not change the view body:
- `AlterV2ViewSetPropertiesExec` (`ALTER VIEW ... SET TBLPROPERTIES`)
- `AlterV2ViewUnsetPropertiesExec` (`ALTER VIEW ... UNSET TBLPROPERTIES`)
- `AlterV2ViewSchemaBindingExec` (`ALTER VIEW ... WITH SCHEMA ...`)
Each rebuilds the payload and calls `ViewCatalog.replaceView`, so after any
of them the catalog receives a `View` whose `viewDependencies()` is `null` and
the previously recorded dependency list is silently lost. Dependency lists are
a first-class field on `View` rather than an encoded string property precisely
because their nested structure does not round-trip through flat properties, so
a catalog has no other way to recover them.
This is reachable today through metric views, which are the only producer
of dependencies (`CreateV2MetricViewExec`). Creating a metric view records its
source tables, and a subsequent property change drops them:
```sql
CREATE VIEW mv WITH METRICS LANGUAGE YAML AS $$ ... $$; -- dependencies
recorded
ALTER VIEW mv SET TBLPROPERTIES ('k' = 'v'); -- dependencies
now null
```
### Does this PR introduce _any_ user-facing change?
No, in the sense that no released version is affected — the `View` API and
these ALTER VIEW execs are new in the unreleased line, so this is a fix within
master rather than a change in behavior users have depended on. Catalogs that
persist view lineage will now keep it across a metadata-only ALTER VIEW instead
of seeing it cleared.
### How was this patch tested?
Two new tests, both confirmed to fail before the fix and pass after:
- `CatalogV2UtilSuite`: `viewInfoBuilderFrom` preserves a dependency list,
and leaves an absent one absent (guards the null path). The first fails without
the fix.
- `MetricViewV2CatalogSuite`: end-to-end `ALTER VIEW <metric_view> SET
TBLPROPERTIES` against the recording `RelationCatalog`, asserting the
replacement payload still carries the source-table dependency. Without the fix
this fails with `viewDependencies()` being `null`.
Full suites pass locally: `CatalogV2UtilSuite` (10 tests) and
`MetricViewV2CatalogSuite` (32 tests). `./dev/lint-scala` passes.
### Was this patch authored or co-authored using generative AI tooling?
Generated-by: Cursor (Opus 5)
Closes #57802 from szehon-ho/spark-view-deps-alter.
Authored-by: Szehon Ho <[email protected]>
Signed-off-by: Wenchen Fan <[email protected]>
(cherry picked from commit 4a471e41eb3608692ac35bd291fcc5f4797dbd96)
Signed-off-by: Wenchen Fan <[email protected]>
---
.../sql/connector/catalog/CatalogV2Util.scala | 1 +
.../sql/connector/catalog/CatalogV2UtilSuite.scala | 23 +++++++++++++++++++++-
.../sql/execution/MetricViewV2CatalogSuite.scala | 22 +++++++++++++++++++++
3 files changed, 45 insertions(+), 1 deletion(-)
diff --git
a/sql/catalyst/src/main/scala/org/apache/spark/sql/connector/catalog/CatalogV2Util.scala
b/sql/catalyst/src/main/scala/org/apache/spark/sql/connector/catalog/CatalogV2Util.scala
index 28aaae4a81ff..a29fe880b915 100644
---
a/sql/catalyst/src/main/scala/org/apache/spark/sql/connector/catalog/CatalogV2Util.scala
+++
b/sql/catalyst/src/main/scala/org/apache/spark/sql/connector/catalog/CatalogV2Util.scala
@@ -577,6 +577,7 @@ private[sql] object CatalogV2Util {
.withQueryColumnNames(existing.queryColumnNames)
Option(existing.currentCatalog).foreach(builder.withCurrentCatalog)
Option(existing.schemaMode).foreach(builder.withSchemaMode)
+ Option(existing.viewDependencies).foreach(builder.withViewDependencies)
builder
}
diff --git
a/sql/catalyst/src/test/scala/org/apache/spark/sql/connector/catalog/CatalogV2UtilSuite.scala
b/sql/catalyst/src/test/scala/org/apache/spark/sql/connector/catalog/CatalogV2UtilSuite.scala
index 4b9d55be07e3..f124186903e0 100644
---
a/sql/catalyst/src/test/scala/org/apache/spark/sql/connector/catalog/CatalogV2UtilSuite.scala
+++
b/sql/catalyst/src/test/scala/org/apache/spark/sql/connector/catalog/CatalogV2UtilSuite.scala
@@ -23,7 +23,7 @@ import org.mockito.Mockito.{mock, verify, when}
import org.apache.spark.{SparkFunSuite, SparkIllegalArgumentException}
import org.apache.spark.sql.catalyst.analysis.{AsOfTimestamp, AsOfVersion,
TimeTravelSpec}
import org.apache.spark.sql.execution.datasources.v2.DataSourceV2Relation
-import org.apache.spark.sql.types.IntegerType
+import org.apache.spark.sql.types.{IntegerType, StructType}
import org.apache.spark.sql.util.CaseInsensitiveStringMap
class CatalogV2UtilSuite extends SparkFunSuite {
@@ -113,4 +113,25 @@ class CatalogV2UtilSuite extends SparkFunSuite {
assert(a.toString.contains("timeTravel"))
assert(a.toString.contains("writePrivileges"))
}
+
+ test("viewInfoBuilderFrom preserves the dependency list") {
+ val dependencies = DependencyList.of(Array(Dependency.table(Array("cat",
"ns", "events"))))
+ val existing = viewWithDependencies(Some(dependencies))
+ val rebuilt = CatalogV2Util.viewInfoBuilderFrom(existing).build()
+ assert(rebuilt.viewDependencies() === dependencies)
+ }
+
+ test("viewInfoBuilderFrom leaves an absent dependency list absent") {
+ val existing = viewWithDependencies(None)
+ val rebuilt = CatalogV2Util.viewInfoBuilderFrom(existing).build()
+ assert(rebuilt.viewDependencies() === null)
+ }
+
+ private def viewWithDependencies(dependencies: Option[DependencyList]): View
= {
+ val builder = new View.Builder()
+ .withSchema(new StructType().add("i", IntegerType))
+ .withQueryText("SELECT i FROM cat.ns.events")
+ dependencies.foreach(builder.withViewDependencies)
+ builder.build()
+ }
}
diff --git
a/sql/core/src/test/scala/org/apache/spark/sql/execution/MetricViewV2CatalogSuite.scala
b/sql/core/src/test/scala/org/apache/spark/sql/execution/MetricViewV2CatalogSuite.scala
index 9b75e89a1ad8..5036ab94c133 100644
---
a/sql/core/src/test/scala/org/apache/spark/sql/execution/MetricViewV2CatalogSuite.scala
+++
b/sql/core/src/test/scala/org/apache/spark/sql/execution/MetricViewV2CatalogSuite.scala
@@ -930,6 +930,28 @@ class MetricViewV2CatalogSuite extends SharedSparkSession {
}
}
+ test("ALTER VIEW <metric_view> SET TBLPROPERTIES preserves the dependency
list") {
+ withTestCatalogTables {
+ val mv = MetricView(
+ "0.1",
+ AssetSource(fullSourceTableName),
+ where = None,
+ select = metricViewColumns)
+ createMetricView(fullMetricViewName, mv)
+
+ sql(s"ALTER VIEW $fullMetricViewName SET TBLPROPERTIES ('k' = 'v')")
+
+ val info = capturedViewInfo()
+ assert(info.properties().get("k") === "v")
+ val deps = info.viewDependencies()
+ assert(deps != null)
+ assert(deps.dependencies().length === 1)
+ val tableDep = deps.dependencies()(0).asInstanceOf[TableDependency]
+ assert(tableDep.nameParts().toSeq ===
+ Seq(testCatalogName, testNamespace, sourceTableName))
+ }
+ }
+
test("SHOW TABLES on a v2 RelationCatalog lists both tables and metric
views") {
withTestCatalogTables {
val mv = MetricView(
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
